Hybrid Society 1. Whose Values Shape AI?
Hybrid Society is a provocative podcast exploring the uneasy co‑evolution of humans and artificial intelligence.
Hosted by Joshiya Mitsunaga, with co‑hosts Prof. Catholijn Jonker (Professor of Interactive Intelligence at TU Delft) and Prof. Frank van Harmelen (Full Professor of Knowledge Representation & Reasoning, Vrije Universiteit Amsterdam), each episode unpacks the political, ethical, and philosophical tensions at the heart of smart technology.
From algorithmic injustice to value misalignment, we confront the hard questions Silicon Valley would rather you didn’t ask.
Hybrid Society 1. Whose Values Shape AI?
Whose Values Shape AI?
What happens when artificial intelligence learns our values, or ignores them completely? In this episode of Hybrid Society, hosts Joshiya Mitsunaga and Catholijn Jonker talk with researcher Enrico Liscio about the promises and risks of hybrid intelligence, the clash between freedom and regulation, and what it means when machines start negotiating human values.
What happens when AI is programmed to achieve a single goal without taking human values into account? And if AI adopts our human values, can we trust it? Welcome to Hybrid Society, the podcast where we critically explore the future of co-evolution between humans and machine. My name is Yoshia Michinaka, and beside me is my co-host, Katelijn Jonker. Welcome, Katelijn. Thank you. And I read on your website that you have the best job in the world. And of course, I'm curious to know why. Can you tell a little bit about it?
SPEAKER_02:Oh, my job allows me to really follow my curiosity in research. And I'm so much interested in intelligence as a phenomenon, both human intelligence, animal intelligence, although humans are also animals, of course, and artificial intelligence and all kinds of forms of artificial intelligence. And my job allows me to do that. How beautiful is that? So it's all around intelligence. Intelligence and the collaboration between humans and artificial intelligence. Which then is
SPEAKER_01:called hybrid intelligence.
SPEAKER_02:Yes, in the sense that we really are looking for a way that artificial intelligence can augment human intelligence and the two can learn from each other. So a co-evolution, as you were saying, between humans and machines in a way that we are getting smarter all the time. Nice. That's a bit of a dream.
SPEAKER_01:Well, Cathalijn will be joining us in the upcoming episodes to help explain and elaborate on the questions raised by our guests. And that brings me to today's guest, Enrico Lisio. Enrico, your LinkedIn profile mentions that your curiosity keeps you alive. Can you share where that curiosity has led you so far? Yeah.
SPEAKER_00:Yeah. Thanks, Jos. So at the Curiosity led me to be curious about how can machines, how can artificial intelligence in this case, learn about us? Can AI learn what we are, what we believe in? And precisely that has led me to Catiline's office a few years ago, five years ago, I believe. And she explained me essentially that what I am looking for, understanding humans means understanding human values. Those are like the deepest drivers, deepest motivations that we have, concepts such as safety, such as personal freedom, like respect for traditions, this kind of concept. So that means really understanding humans.
SPEAKER_01:So today we'll talk about human values and AI. You've conducted extensive research on this topic and it seems like you've come a long way. So before we dive into the findings, could you explain how you conducted your research?
SPEAKER_00:Yeah, so I've conducted my research, especially in the beginning, the most important part through hybrid intelligence. So what I mean is that certain aspects of understanding humans cannot be purely left to machines, right? But at the same time, if we want to understand as many humans as possible, then it's just too much work for humans to understand other humans, simply. So artificial intelligence machines can help scale and humans can help give the soft, abstract, difficult interpretation.
SPEAKER_01:Can you give an example for that?
SPEAKER_00:Yeah. So the example, the application that I started my research on was a was COVID-19, of course, because back then that was, you know, the thing in our lives. And it was the end of the first wave of COVID. Now we know that it was the end of the first wave. And the government wanted to relax regulations, right? And so it wanted to ask the population, the Dutch population, what regulations should we start relaxing from? You know, reopening ORECA, allowing immune people to go around freely and so on. But simply asking people this question. to vote, for example, A versus B, only gives that amount of information to the policymaker, right? But instead, really asking the citizens, why would you prefer A over B, gives a lot more information for the upcoming, say, second, third, fourth wave of COVID. But processing this, you know, it's easy to process votes, you know, you count the votes, but it's difficult to process these explanations that humans give. You know, point in case, this survey was performed among citizens of the Netherlands and there were 60,000 answers. Now you cannot have humans read all of that and somehow make sense of it. It's just too much. So 60,000
SPEAKER_01:open fields, it was an open answer option.
SPEAKER_00:Yeah, exactly. So 30,000 people answered in total and each gave on average two open field text box questions justifying explaining why they, for example, wanted ORECA to reopen or why they wanted immune people to be freely going around. So something like that is just too much to analyze for humans, right? Right. So that's why we need artificial intelligence to be able to go through those scale of data. But on the other hand, artificial intelligence, even today, you know, back then for sure, but even today I would argue, cannot really understand something so fine-grained like human values and the value trade-offs that we make, especially in a new situation like this one, right? You know, it was the first time in our lifespans that we were relaxing regulations after a pandemic, right? So it's just too difficult for artificial intelligence to understand the trade-offs that we humans make in such a new situation. That's why we need humans to interpret that. So what we did is that we created a hybrid intelligence methodology that is that we combine the scale that AI allows and the human intuition of humans. The intuition, the abstract reasoning abilities of humans. Essentially, what we did is that we want humans at the end of the day to judge these value trade-offs, to judge what humans say. You know, for example, they say that for them caring about mental health is really important, even more important than their physical safety, right? So these kind of concepts, they are really difficult sometimes to read into text, but we know that humans can read just that much, right? They can read a few hundred, maybe a thousand at most. So what we did is that we used AI to basically pretty much create a representative sample so that humans could only read selected answers that would be really representative of groups of answers and judge only the and go through only a few of those, but making sure that that few answers that they go through, a few hundred, a couple hundred, are the most useful, the most important to read and to judge. So
SPEAKER_01:you teach the system or you teach the AI which values are hidden in the text and then from there you scale it up?
SPEAKER_00:Well, yeah, so we didn't do exactly that. So we did mostly, what we did is that we did not train the AI. there, but what we did is that the AI was looking for sort of diversity of opinions, we can call it, in a way, and diversity of content mostly, based both on the content itself of the surveys and what was input by the humans. So if a human read, said something, this or this is a concern about, say, economic stability, financial stability, right? So then the AI would try to search for comments that are not about financial stability, because they should and already read something about financial stability, right? So maybe let's try to find something different and that different could be, for example, about mental health, could be about loneliness, could be about elderly care and so on.
SPEAKER_01:So it was picking out the different types of values in the data set.
SPEAKER_00:Yeah, mostly a different type of contents and then the human would interpret the value in there.
SPEAKER_01:Ah, got it. So, Katalin, this approach is typically what you would call hybrid intelligence. So is hybrid intelligence always related to data labeling and the assembling data and interpreting data? Or could it also be, are there also forms of hybrid intelligence on the front end, on the user end?
SPEAKER_02:Well, that depends on what type of intelligence you're using. So if we're talking about data-driven AI, so the machine learning approaches, yes, then the data is typically part of it, but not all AI is. And the point where the humankind is, as you heard also from the story that Enrico was just telling, is the context. So if you're talking about an AI system that is kind of general in its approach, like these large language models that we have, then those are trained on all kinds of data, but not specifically understanding the context in which you are now trying to use the information that is stored within the large language model. So that context, as Enrico was saying, at that time we were for the first time in a situation where you relax COVID measures. So do you actually already know what is part of that? What is important? What is not? That information is not available to large language models, so we wouldn't be able to say something about it. So in those cases, yes, then the human is the one responsible for bringing in that context. If you're talking about other situations might be different might be something where the human knows the overall applicability of a rule in a situation also typically context related but the artificial intelligence might have say a Bayesian network behind it which knows what is the most probable right way of going about this and you can still overrule so it doesn't have to be about large language models, but it is about bringing in your own expertise from both sides, whether one is maybe trained on data or has expert knowledge stored in it and the other, say the human, understands where the question comes from and is the one that also has in the end to make sure it's applied right in the context.
SPEAKER_01:So hybrid intelligence always needs two forms of intelligence then, right? At least two, yes. At least two. So I'm just trying to define that concept of hybrid intelligence. So for instance, if I use GPT to write an email, am I then working in a hybrid construct or is it a hybrid workflow?
SPEAKER_02:It's not yet a hybrid flow. It's close to it, but it's not. It's more like you're also doing with your browser. You give it a query, right? I give it a prompt, you get an answer. Then because you start thinking on it, you might change your prompt and do it again. In the meantime, the LLM is not learning anything. It just is, again, that same bit of machinery that if you give it a prompt, it gives an answer. And in that sense, it's just a tool. It's like a hammer. You give it a whack on a nail and the nail goes in. If you give it another nail, you whack it again. Yeah, then that's what it does. Nothing more. It doesn't challenge you. It doesn't come back with, but isn't this more, why are you asking this? Or tell me more about the context so that I can give you a better answer. answer. So it's not learning something back and it's not trying to understand the context specifically before it gives an answer. So it's not learning at that moment from you.
SPEAKER_01:Is it related to the depth of the dialogue? So answering questions back and forth then?
SPEAKER_02:Yeah, like we're doing now. You're trying to learn something and because the way you're asking it makes me think. And so I also learn just already being in such an interview. You start thinking, hmm, is that actually, what is then the essence of that duality, that augmentation goes in both directions.
SPEAKER_01:Interesting. Very interesting. And well, we will learn more about that hybrid intelligence way of thinking in other episodes as well. To go back to your research, Enrico, you trained AI in collaboration with humans, but when it comes to values, prioritizing them is deeply personal, right? So if the model is trained with humans, doesn't that make the training subjective? And would it also reflect primarily Western values. So in short, what values should the AI ultimately be trained on?
SPEAKER_00:Yeah, that's a million dollar question. I'll try to give my answer. So first, let me give a quick background on the idea of values, like a theoretical idea of values. The idea is that there's several different values that are relevant to a conversation. Say again, we talk about immigration or COVID-19, so let's take the COVID-19 example. So the values of safety is important, the value of personal freedom is important, and there's a number of values that are important at the same time, and each one of us gives different priorities, gives a different way to these values, basically, right? Some of us consider safety more important than freedom, some others consider freedom more important than safety, right? And in, you know, liberal, western, democratic societies, societies allow for a variety of value trade-offs, right? It's a right for, say, that we take immigration as well. It's a right for some to consider immigration, to judge immigration morally right, to judge immigration morally wrong. There are both views that are allowed within a democratic, liberal society. What societies do is to put external boundaries, what we call, what can be called guardrails, for example, which are legislations, constitutions. Essentially, what these constitutions and legislations do is that they say, okay, here are the boundaries, you cannot go beyond it. You cannot kill. But within those boundaries, you're allowed to have different views of society, different views of life. So this is pretty much how I view Western liberal societies, and this is how I would view the training of artificial agents and values in artificial agents. It's okay that different artificial agents have different value trade-offs. That they align for example with the trade-off of the people that they interact with or the users if you want to call them and that's alright granted that they don't go beyond set boundaries they don't help someone enslave for example they don't help someone kill right so that's those would go beyond the boundaries and we can find some we can actually make sure that they don't do that you know from a coding point of view right but within that they can have different value trade-offs different views of society And as we humans do in liberal societies, we have different views and we come together and we discuss. We have politics, right? That's what it is. And we take a common decision. We do compromises and so on. That's my view for artificial agents, that they should start as a whiteboard, right? That they don't have a preloaded set of values, but they can learn the values of the citizens, people they interact with.
SPEAKER_01:But these guardrails that you talk about, how tight should they be? For instance, if you look at the current deep-seek model, it has very tight guardrails when it comes to questioning China as a government. So to what extent isn't this just a more technical approach of censorship? And yeah, that can also be very dangerous,
SPEAKER_00:right? Yeah, definitely. So definitely, yes. But that's something that to me I see as... as part of political life, right? So unfortunately, yes, there are also some countries that are more illiberal, where these were the sort of the allowed view, where the allowed value trade-off variation is much more restricted. And these boundaries are much smaller. Yes, so if you ask certain questions to DeepSeq, they won't answer, or certain questions that they would be answered in, say, CHGPT, they are not answered in DeepSeq. The opposite, of course, happens with chat GPT, right? If you ask chat GPT to how can I build a bomb, chat GPT will not answer. It will tell you that they cannot answer this kind of question, etc., etc. So there are boundaries also in Western models. But
SPEAKER_01:is it then that we don't notice that because we are more aligned with those values?
SPEAKER_00:Yeah, definitely. So definitely that's the case, I think. So we notice those less because we just, I guess, naturally we wouldn't ask those questions to ourselves, but we since there's something that is culturally different from us, Chinese values, for example, different from our Western European values, then we notice those much more. But on the other hand, those are, you know, it's a bit of a thin line of political acceptance, I guess, acceptance of political stances. It's something that is, I would say, I would argue beyond technology is the decision of these boundaries. So I think the technology should allow for these boundaries because, I believe in legislations, I believe in constitutions, but where and how tight those boundaries is, I think is something that is beyond the technological and something that is a broader societal discussion.
SPEAKER_02:Well, you're talking about the boundaries, but on the other hand, there is also the stereotypes that are hidden in the data that you feed it with. And we also all know that our Western culture is really different from, say, more Eastern countries. And you see that ooze out of every basic answer that's the Western-based or Western-trained LLMs would give you. And those stereotypes are clear in the sense of ask for something about a lawyer or ask you to draw something with a lawyer in it. You'll see male people dressed in suits. And whatever you try to do to change it, forget it. So those kind of stereotypes are part of the model trained into it. So yes, all the cultural aspects that you see and all the biases and discriminations and etc. that you have within a society, if you put that into the data, and of course it has it, because it's part of us, and you train that model on it, then it has that. The problem is that it doesn't know it has it and therefore also cannot be transparent on it, except when we tell it to be transparent on it. And like also it used to just give an answer on this bomb making. Now we said, okay, we need to put some boundaries on it. So yeah, you could do that for as far as you realize that the model has this bias as you could train it or tell it, instruct it to make people
SPEAKER_01:aware of that. So it's also a way of reflecting on our own culture and how we think on and how we draw things, right?
SPEAKER_02:Well, that depends if you're sensitive to that and as a woman being discriminated upon regularly I am sensitive to it so I do see it but if you're not it's subtle enough it's not always that easy to recognize it and certainly not if it's beyond the scope of your normal thinking so I think it's also part of research and basically academic research to make sure that we pay attention to that we do look into it
SPEAKER_00:I mean at the end of the day it's also a question of do we want to train a model that reflects us as we are, or do we want to train a model that reflects what we wish we were?
SPEAKER_01:Well, that brings me a little bit to the perspective of the more complete, liberate perspective that, for instance, comes from the US with Elon Musk saying, no, AI should never be regulated. It's a technology. It should be solely objective and any form of regulation or it should be considered censorship and censorship is bad because it's freedom of thoughts and speech and so what's your take on that?
SPEAKER_00:Yeah so I have a very you know personal take against that which is to cite Voltaire that your freedom to move your fist ends where my nose begins right so pure freedom doesn't really work pure freedom essentially means that the strongest will win against the weakest will dominate the weakest. I strongly believe and this and AI per se will in two ways facilitate that. One is by, as Katalin was explaining, by sort of reiterating existing biases and these biases do come from the majority view, the stronger, against the minority view, the weaker. And the other is that these are tools that if unregulated will just be used by richer people, by the more powerful that just simply will have access to those whereas the weaker the poorer will not so I do believe in regulation especially to moderate these biases to unveil these biases and to make sure that everybody can use it in the same way
SPEAKER_01:if we talk about hybrid intelligence and this topic also triggers that thought in me that we are also making AI smarter we are also teaching it more about ourselves right if it can interpret that values and can that also be used against me I mean you also hear a lot of people that say well AI is smart enough maybe as it is so if we start working in a more hybrid context or more hybrid workflows, it will learn more about us? Yes. And isn't that also very dangerous?
SPEAKER_02:Yeah, essentially, of course. I think that's true. On the other hand, it can also, by doing that, do more good. So the whole point, and I think that's also why Enrico is bringing it up, is that you do want to have some constraints on it. So there's this nice term called human oversight or meaningful human control basically meaning that we want to be able to control and be able to intervene when necessary when AR starts doing things that we typically disagree with right so you want to be able to override it intervene teach it about the potential impact of its decisions as a negative impact on say democratic values personal values of people. And if you don't do that, then it wouldn't learn. Still, once it starts learning, you can abuse it. Yes, that's unfortunately the case.
SPEAKER_01:So isn't that the human oversight, human control, then more of an ambition that it's a feasible goal? So how sure are we that we can keep this control? Because if we're adding all this knowledge and we're making these models smarter and smarter and smarter, in the hope that we can keep it under control, then should we do that? Should we aim for that?
SPEAKER_00:I would say yes, but with moderation, in a sense that we first need to think on how to control it before going on with making it bigger and bigger. There's a lot of research on that, of course. And a way, as Catalan would say, is to make the AI more self-sufficient. to have more self-doubt, right? To be less confident in a sense. So in that sense, it can more like leave on the human and let the human kind of make sure that the AI is doing the right thing.
SPEAKER_02:Yeah, and so far the agency of it in terms of being able to set your own goals in life is typically what humans do, situated in life, in our environments, within our friends' companies. all the people that we interact with. The way that people are able to set their own goals in their own lives, coming from the obstacles they meet in life, they want to achieve something, they set their own goals in that way. AI cannot do that as yet. And part of that is because it's never situated in our environment. It's never embedded in our society in the same way. If we solve that, then maybe things will be different. But for now, it's not. And that means that in the end, it's always humans that tell AI what to do. So they give it the challenge. They set the goals. They set the criteria for optimization, which is, of course, typically what AI is also used for. And then it starts acting on that. So having the reins on it is something that comes also from knowing where you want to go. Otherwise, you cannot steer in the right direction. pretty simple from that point of view. So if you turn that around and say artificial intelligence starts setting its own goals, it should have a goal to start with in the first place, and it doesn't. It's just trained to answer these questions. It's trained to do a computation. A Bayesian network is trained to do the computation, give an answer. That's basically
SPEAKER_01:it. should the end goal be that humans and artificial intelligence or human and artificial intelligence always will need each other to operate or is the end goal to have a twin of the human brain in an artificial way which can operate autonomously and act autonomously?
SPEAKER_02:Well, my goal is not to have those things. On the other hand, maybe people do want to make such twins as you're calling them, digital twins. However, these digital twins are not embedded in the same way in our environment as you are. You have your bodily synapses, you have your neurons, your nerves. They give an input to our cognition that these AIs don't have. So if you go to robots to do that, and that's another form of embedded embodied cognition, then it's related to the point where where that body, the robot body, is interacting directly with the world and the world is having an impact on that body. Now, then we go into the whole subject of feelings and the feeling of feelings. Related to that, a very interesting topic, but I don't think one for today.
SPEAKER_01:Not for now, no. So if your research are potentially building blocks for some dystopian AI future, then why taking
SPEAKER_00:all this risk? Yeah, good question. So to this, I would give the answer of a famous example from an AI philosopher, Nick Bostrom. So I think almost 20 years ago by now, he made the famous paperclip example to explain why understanding human values is really important for artificial intelligence. The paperclip example goes that in the future sometime there will be an AI that is in control of a factory of paperclips in this. A very mundane application. But if this AI is sufficiently powerful and has sufficiently access to a lot of resources and is given the goal by the owner of the factory to make as many paperclips as possible in the show shortest amount of time that you can. If the AI is given this order and has really a lot of power, a lot of access to resources, and doesn't really understand the human, but only listens, only understands the order that is given, the goal that is given, as Katalin said, then it might actually deplete the whole world resources, use all of the iron in the world in order to make as many paperclips as possible, because that's the order that it was given. Essentially, this example tells that machines, in order for machines to be beneficial, to do what we really want, they must not only understand our orders, our instructions, but they also must understand our values, our true goals. And the true goal, of course, sort of the value trade-off that is in there for the factory owner is make as many paperclips as you can, but without harming anyone, but without the pleading the world's iron resources so these kind of things go unsaid for us humans like obviously you know you shouldn't use all the iron in the world if you make paper clips like this is something that goes unsaid for us humans but that's something that's not something that is not obvious for an AI agent and that's why AI agents should be able to understand our values what we really care about what we really mean deeply in order to truly align with our real goal and really to understand this.
SPEAKER_01:Got it. So... In this same example, if you would have two paperclips factories, they would end up having a war for the latest iron resources, right? So there's no boundary, exactly what you were saying with the safeguards. If AI understands more human values, would it be helpful in taking more ethical decisions or provide more ethical outcomes?
SPEAKER_00:Yeah, definitely. So it would at least provide what we in the office consider ethical, obviously. So yes, definitely.
SPEAKER_01:Somehow this also reminds me, or this AI technology also reminds me to Bitcoin. And at some point Bitcoin was there and people tried to regulate it and they say, well, it's a good system, but we need to regulate it. And somehow it's intangible, right? So we have Bitcoin as a blockchain being everywhere. Then you had all this side products and coins and now it's everywhere. And you see institutions around the globe trying to to get a grip on it, but it simply can't because it's online and it's everywhere and it can be done locally and remotely. And to what extent is this regulation then actually feasible in an online global scale?
SPEAKER_00:This is really, really difficult to answer for me. I would say it's still feasible in the sense that AI models are trained by someone on certain servers, right? They are not, at least so far as they are right now, they're not just trained by everyone. We don't all contribute with our little bits of training, but they're trained by someone in a certain location. And the data that they're trained on, we can have that information. You know, like... the trainer, the companies or the governments that train these models, they know what data they are feeding to the model, they know how the model is written, they know what the model then will become after being trained, so we pretty much can have the oversight on how this process works.
SPEAKER_02:This is a difficult one. There's of course the whole peer-to-peer network structure underneath it and there's some deep democratic or well, I'll say distributed ideas behind the whole Bitcoin portfolio etc which makes it very interesting and they also try to make sure that even though you don't know who owns what that the people that own the Bitcoins themselves also put in their own regulations and that seems that you have a grip on the whole system and that it's a system from by the people, from the people, for the people, so to say. But it also still has this power problem. So the people who have more Bitcoins have more sales, have more votes. So the rich, in that sense, can still become richer again. And of course, the whole first point on mining these Bitcoins was in the hand of relatively few people who got very rich of it and still therefore also have the power to invest more in machinery to make more, find more. So there's still a power problem. This is something that we should keep for another podcast, I would say.
SPEAKER_01:To our listeners, thank you for tuning in. Subscribe to this podcast to get an update on new episodes. Enrico, thank you so much for being here. Best of luck with your research. Thank you, Jos. And Katelijn, thank you as well. See you at the next episode. See you next time. Thank you. Okay, bye-bye.
UNKNOWN:you
