Uitdagingen en strategieën tegen bias in AI

In de tweede aflevering van de greatminds podcast staan Hildo van Es, solution architect en medeoprichter van greatminds, en Robin Smits, data scientist en eigenaar van LumiML Consulting, stil bij een van de meest prangende kwesties in de wereld van kunstmatige intelligentie: bias.

Bias in AI-systemen komt voor in vele vormen. Het gaat vaak om vooroordelen over gender, etniciteit of religie. Daarnaast spelen ook aannames over beroepsgroepen een rol. AI-modellen weerspiegelen helaas vaak menselijke tekortkomingen. Dit gebeurt vooral bij grote taalmodellen (LLMs). Ze trainen namelijk met teksten die via web scraping verzameld worden. Deze teksten bevatten vaak al de vooroordelen van hun menselijke auteurs.

🔑 Belangrijkste inzichten:

• Bias begint vaak al bij de inputdata — je model kan niet ethischer zijn dan je bronmateriaal. 
• RLHF is een krachtige methode om modellen te corrigeren op basis van menselijke feedback. 
• Bias volledig uitsluiten lukt (nog) niet, maar je moet het actief proberen te beperken én te blijven monitoren.

📱 Connect met onze gast en host: 

Robin Smits | Hildo van Es 

⏱ Tijdstempels:

00:00 – Introductie en definitie van bias 

00:59 – Wat wil je juist voorkomen? De ethische uitgangspunten 

04:06 – Regels & richtlijnen: EU AI Act en Microsoft framework 

06:37 – Garbage in, garbage out: data bias door webscraping 

10:07 – Evalueren van modellen en het belang van transparantie 

14:54 – Praktische maatregelen: datakeuze, filtering, privacy 

16:17 – Reinforcement Learning with Human Feedback (RLHF) 

18:28 – Monitoring en feedback na livegang 

19:53 – Afsluiting

Transcript

Hildo van Es 0:12

Hello, welcome to the Great Minds podcast. I am Hildo van Es. I am a solution architect and co-founder of Great Minds, a knowledge domain about everything that has to do with architecture and so also AI, and we do a series of podcasts about things we encounter in AI. And next to me is Robin. Robin, please introduce yourself briefly.

Robin Smits 0:36

Yes, welcome. I am Robin. I work at NBWI as a data scientist and in addition, I have my own AI consultancy company, LumiML Consulting.

Hildo van Es 0:47

Okay, super. Last time we talked about evaluation of LLM models. And this week we're going to talk about bias, and there's a lot to do about bias also in the news, for example, we had the between people based on gender, skin color and actually every form in which that manifests itself in your data and can have consequences if you train your model with it.

Robin Smits 1:57

You just don't want that.

Hildo van Es 1:59

Yes, so actually every form of unethical behavior in your data a form of unethical behavior in your data. So your data is set up in a way that unethical consequences are sucked into your data. Yes, yes, yes, of course we don't want bias in those data sets, because we want to have AI that behaves ethically. Can you explain what kind of behavior we want to prevent? What kind of starting points do we have, for example, based on? Well, we've already talked about ethics. Can you name some other points?

Robin Smits 2:41

Yes, what you see there is that, and now it's starting to come. Recently, the EU AI Act was introduced as a broad regulatory platform that has several starting points. The Americans are now coming up with frameworks to regulate AI Businesses, like bias To counter ethical things. I think one of the more beautiful examples is Although not everyone will agree with that, but well, microsoft has, for example, as one of the big tech companies, already in 2017. For example, they have set up their own framework To do a bit of self-regulation. Well, you can twist about to do a piece of self-regulation. Well, you can twist about it. How ethical is self-regulation at Big Tech? But hey, there's something there. And then you see that the starting points for Microsoft are that every AI model must be ethical, responsible, inclusive, reliable and safe, explainable, transparent, verantwoordelijk, inclusief, betrouwbaar en veilig, verklaarbaar, transparant, privacy en veiligheid. Dus, hele algemene uitgangspunten, maar denk wel hele belangrijke, en ik denk op zich dat het goed is dat dat nu ook gewoon tot op zeker niveau vanuit overheden, dat dat gereguleerd gaat worden.

Hildo van Es 4:05

Ja, dat lij to me too. The regulation. Is that comparable to GDPR? What does the EU write for, for example, where we should stick to?

Robin Smits 4:20

I must admit that I have not yet dived into all the details, but, for example, one of the apart of all those points that we now already call those in more or less size, privacy, have you said from gdpr, are guidelines for which data you may or may not use and how the distribution of that data is. In my opinion, that almost works one-on-one to your ai models because, because you don't want that privacy-sensitive data to be used to train a model, no, indeed, say that we're talking about a medical model and your medical data is in between. Yes, no, that can't be the intention, and you see that kind of points there and you see those kinds of points there. But also, one of the important parts that, for example, are taken up from that regulation is that the output of LLM models must be recognizable. Yes, so with a kind of watermark in it, that it is recognizable as output generated by AI.

Hildo van Es 5:25

Yes, that is interesting, especially now in the generative AI period where we are now, and I think that was recently that, especially during the elections here in the Netherlands, that pictures were generated by Frans Timmermans, who was in a jet, and the living, the life that was generated by AI. Yes, and as far as that is concerned, it would be an extremely good idea if that is indeed recognizable as generated by AI.

Robin Smits 5:59

Yes, but there you see right away, well, that whole hotmarking. There are also some hooks in the eye, but does that mean one of those models comes with biased output, discriminating texts, whatever form then you would through such a watermark is in any case the idea. You could more easily show it, because the idea of such a watermark is that every specific model is recognizable from there. Yes, yes, so that is, for example, one of the many ideas that are around regulation yes, but well, that is for this piece. That is also afterwards.

Hildo van Es 6:38

You want it, of course, preferably to prevent the front yes, precisely, you want to get the bias out of your data. Of course, yes, exactly, you want to get the bias out of your data. Can you maybe tell us something about how bias? Yes, we have trouble with that. How does that look? So, for example, if you set up an AI project, it consists of different phases. Where does bias mainly occur, or is that in every part of the process?

Robin Smits 7:11

You can have trouble with that in several points, but it actually starts mainly at the beginning With your data collection. Everyone knows the saying garbage in is garbage out. Yes, certainly. I fear that with AI models, especially LLM models that generate text, this will just go to extreme extent. You have to imagine the majority of the datasets that are used to train those LLM models. They are made on the basis of web scraping. So for years, yeah, yeah, recent law from the New York Times against OpenAI. Hey, this is copyright material. Yes, I read it. Yes, so that is not always appreciated. I think that law is still running, if I understood it correctly. So I don't know what the final conclusion will be. But web scraping, we collect that data online. Yes, and if there is something that people are, then it is biased. Yes, certainly. So in that respect, you could say that that data is representative for humanity. But, yes, do you want that? Not in your AI models?

Hildo van Es 8:46

Yes, I have also heard people say that AI is less biased than we people ourselves, because we are certainly not free from any prejudices for established opinions and AI has no opinion.

Robin Smits 9:00

In a certain sense, that's right. You train an LLM model on the data you put in. If that data contains a certain bias, then your model unfortunately learns that. On the other hand and that is now precisely the crux if we can prevent that that training dataset also contains only a single form of bias, then we actually train a very nice neutral model. Yes, and that is what we want. Yes, that is also practically very difficult. There are a lot of methods with which that data is cleaned up. No double texts. A lot of texts are automatically, also automatically judged with AI models. There are, for example, ai models that classify text and see if there are in that dataset. Are money words in the dataset? Is there racism in it? Is there sexism in it? So it's filtered with a particularly large accuracy, but it always goes on.

Hildo van Es 10:09

Yes, and we've talked about that last week with the evaluation of models, had about it with the evaluation of models. By testing. Well, you can, for a part, recognize bias, go out, filter, look at your data again and maybe start training your model again.

Robin Smits 10:50

Yes, well, in the end, yes, but then you still want to know for sure, or at least a certain amount of guarantee, that the model is trained on a dataset that is as pure as possible. And even with an open AI, you see that there is still a limited form of bias in a GPT-4. Yes, exactly. So we can limit it, but we can't reduce it to zero. Exactly Vorige week bracht Google zijn text-to-image model uit, en daarin hadden ze echt het maximale gedaan, om bias terug te brengen.

Robin Smits 11:45

Waar resulteerde dat? In? Dat, als jij vroeg om een plaatje van een Duitser in de Tweede Wereldoorlog, dat jij niet alleen zeg, maar echte Duitsers in de Tweede Wereldoorlog kreeg, maar ook plaatjes van andere nationaliteiten, that you not only got real Germans in the Second World War, but also pictures of other nationalities. So that went back to. It is perhaps a bias, but it is historically incorrect. So that is also a very complex one. Well, they just put it out then. So I think that's another extreme example, where the other side of on the other side, but it indicates that you want your datasets as clean as possible.

Hildo van Es 12:23

Yes, yes, but what should we pay attention to? So, when we want to have that dataset as small as possible, what are some examples, for example, of what you should look specifically?

Robin Smits 12:40

What we said earlier privacy, anonymizing data. You don't want private data in it. If you train a model on it, it will remember that.

Hildo van Es 12:52

So then you would, for example, if that wasn't the case, you would be able to ask ChatGPT, for example, what is the phone number of my neighbor?

Robin Smits 13:01

If that data is not filtered on it, then he would be able to know. Then you could also go to Telefoonboek if he doesn't have a secret number. But yes, that's possible. Then you're also back to the problem. You want to filter out privacy data from ordinary citizens. But what is it historically important, being people? Well then you can look at which website it comes from. So that is an important one. Any form of bias based on gender, religion, sexism, racismksisme, racisme dat wil je allemaal er niet in hebben. Maar ook bijvoorbeeld geen bias op basis van zeg, maar demografische gegevens. Beroepsgroepen Recentelijk toch een paper langs zien komen, waarbij ze an LLM to set job descriptions so invoice texts, and then you saw that the salary differences you see then return in the generated texts. So women have, according to an LLM, a right to a lower salary than a man at a comparable function. That kind of thing. You may not stand by it directly, but you see that too.

Robin Smits 14:23

And that is really inhumane behavior from the LLM yes, only unfortunately, you see that again we store human data in. Those are again invoices, texts that have been picked up online with web scraping and that are used.

Het tegengaan van Bias in AI

Hildo van Es 14:41

Yes, you understand, but anyway, yes, certainly unwanted behavior and that we don't want bias. That is logical. What could we do to eliminate bias? What methods can you use to handle that?

Robin Smits 14:59

Yes, really eliminate. I think what we said there are. You can use a lot of methods to select your data, to filter data. I think that you can use that, especially because it's an active corner for research. You can do the, especially because it really is an active corner for the research. You can do the maximum to limit it, but eliminate it. You assume that you can bring it back to zero. No, exactly, and I think that at this moment, unfortunately, that's not possible.

Hildo van Es 15:28

No, but we can of course try to do as many things as possible to make it better possible things to do, to do it, of course, and then we can do things with the right selection which data we select? Well, things like preparing data and training a model, for example, we can do things there. Can you tell us something about that, how you can adjust the training of the model so that you are as less biased?

Robin Smits 15:58

as possible. Yes, really Well. A nice example what you see when we have selected that data, filtered it prepared it, done our best to limit it to the absolute minimum, then we are going to train a model's, what we call pre-training. We train them standard, but what they actually do the last two, maybe three years, what they do Is then we go model alignment and that is as far as I know With ChatGPT and the predecessors of that Is that first really on a large scale, then we let the output of a model be judged by people and then we give this is a desired response and this is not.

Robin Smits 16:47

And what you then see is that and still again, that does not eliminate bias, but it does bring it back further, that you can really yes, yes, yes, genereren, en we geven dan aan, welk antwoord wel gewenst is en welk niet. Dus een beetje zoals je een kind bijvoorbeeld iets leert, ja het soort vorm van corrigeren, en je traint het model opnieuw En dan in de hoop, dat dat goed genoeg is. En dat zie je dus, om die bias te verminderen, om te zorgen, dat het model zich ethisch gedraagt, maar ook wat praktischer zaken we to ensure that the model behaves ethically, but also more practical things. We learn from the model that it will not answer questions about bombs and chemical weapons because we don't consider that to be ethical. We don't want to make that information widely available.

Hildo van Es 17:50

Yes, and then that is actually a question that I always ask in our trainings, or a topic that I then deal with is then you have built something beautiful and then you take it into production and then how do you keep this safe? How do you keep safe that you can recognize bias, because indeed that 100% possible then you probably have to continue to monitor on outcomes and on behavior.

Robin Smits 18:27

Absolutely, that model is, once trained, aligned. But what we did in the previous podcast, you can't test everything. So an important aspect after such a chatbot is in production is that you keep monitoring it. That will also be largely automated with models that check the output on is it well aligned? So that is one step that certainly at Big Tech I expect that. Yes, let's pay attention, you can. At every chatbot you can often give a thumbs up, a thumbs down. Yes, well, that's the human feedback you can use Super. You could still say that, yes. So if you want to fake that model, then you just stay on the opposite side. Then you give a good answer that it is wrong, but they will also look at it again. Yes, answer that it is wrong, but they will also look at it again. And even if you do that as a company yourself with your own trained model, you can just apply those techniques Even more. So I think you have to do that, but that is often also a matter of resources and budget, I realize Super interesting story.

Hildo van Es 19:53

Bias is one of the biggest problems I think that we have within machine learning, within AI, because on the basis of bias, social problems can also arise. We have now talked about how it arises and why it is so bad to have bias in your model. We have discussed what you could do about it. Yes, Next week we are going to talk about how you could successfully use AI in your company, so that is a slightly less technical story. Next time, yes, nice, I also hope that people will listen. For now, I want to thank everyone for listening to this podcast and see you next week.

Robin Smits 20:43

Thank you and see you next week. It was fun again, thank you.