Exploring AI Matters
Our mission is to help the policy community understand the breadth and richness of AI and the potential for such technologies, wisely applied, to augment all sorts of human endeavors.
Some AI tools are able to assist humans in performing tasks faster, more accurately, or more efficiently. Some, however, are inaccurate and unreliable. Who or what we hold accountable for these flaws, and what incentives we do or do not create for their correction will influence AI’s hand in how we work.
In this series we will refine, sharpen, and clarify your understanding of AI.
Exploring AI Matters
Episode 12 - Nothing About Us Without Us
Use Left/Right to seek, Home/End to jump to start or end. Hold shift to jump forward or backward.
We all agree that new technologies are cool, but unintended consequences can be very hard to reverse. Today we will be talking with Dr Christina Colclough, a political economist with a PhD in Sociology from the University of Copenhagen. She has been engaged for over a decade in the global discussion of digital technology as it relates to work and workers across the world. [2023-07-10]
Welcome to Exploring AI Matters. This podcast series, previously known as Mind the Gap Dialogues on Artificial Intelligence, will continue to appear in the ABA series to the extent that, in addition, all of the episodes, old and new, will now appear under our new podcast name, Exploring AI Matters. Thank you.
SPEAKER_04Some applications of technologies, such as AI, have already shown harmful unintended consequences. It is becoming clear that governance of these emerging technologies must extend to a broader audience. Today we'll be talking with Dr. Christina Kokloff. Dr. Kokloff is a political economist with a PhD in sociology from the University of Copenhagen. She has been engaged for over a decade in the global discussion of digital technology as it relates to work and workers from a vantage point of the trade union movement across the world. Dr. Kolklough is the author of Trade Union Movement's first statement of principles on workers' data rights and the ethics of AI. Kolklough is a fellow of the Royal Society of Arts in the UK and former board member of the Global Partnership of AI. Trade unions continue to influence business practices across Europe. For instance, in many European countries, the staff of enterprises domiciled there are entitled to representation on the boards of corporations. In recent years, Dr. Cole Clough founded the Why Not Lab to examine the interaction of work and technology, studying the ethics of application of AI techniques, and analyzing models of governance for AI systems. Welcome to Mind the Gap Dialogues on Artificial Intelligence, Dr. Klo Clough. I am Charles Donner, a computer scientist.
SPEAKER_01And I'm Mama Adams, a national security lawyer. We are your host for this episode of Mind the Gap Dialogues on Artificial Intelligence. In addition, we have two more hosts.
SPEAKER_05Hello, I'm Roland Trope, a national security lawyer. And I'm Mark Donner, a computer scientist.
SPEAKER_04Each episode will be led by two of us, with the others adding impromptu questions and comments as the spirit moves them.
SPEAKER_01Well, welcome to our podcast, Dr. Callcloffett. It's wonderful to have you here. And to start us off on our discussion, I think it would be great to hear a little bit from you about your journey from political economists to being a thought leader on the future of workers and sort of the intersection with digital technology and AI.
SPEAKER_00Well, thank you both for all for having me here today. It's really, really an honor. So my journey was one of coincidence, really. And that I think is one of characterizes my life. Everything has kind of happened and stumbled into things. But after my um university degree, I took a master's degree in the UK, I moved to Denmark, and got employed by Copenhagen University in their labor market research center. And there were several things there which really kind of, as a Brit who lives in Denmark, they really kind of made me think. Now, firstly, every time I interviewed shop stewards, so union people in the companies, they would always say we about the company. We have to restructure, we have to reinvent, we have to do this and this. In many, many other countries across the world, and not least in the UK where I'm from, it's very much they. So that difference between the we and the they, you know, they being the employers, kind of got me wanting to dig deeper into this whole thing about the Scandinavian model, uh, which is very, for many people in the United States, uh, kind of bewilderment, really. You have very strong cooperation between the unions and the employers. The employers are organized in employers' federations. You have a so-called peace agreement. So in the period between the collective agreement has been negotiated and it runs, you know, there the companies know what they have of expenses in wages, working conditions, and so on. And the workers, of course, have negotiated those rights. So it's very much a consensus thing. Where, you know, in the UK and many other parts of the world, including in the US, it's more of a boxing. So you have the dancing model, you know, in the in the in Scandinavia, where you might step on each other's toes now and again, versus the boxing model uh of many other countries, uh, England, United States, but also many others. So, you know, I was researching on that. I did my PhD on the role of social capital uh and innovation in companies, tracing how Danish companies, when they became multinational, did they somehow bring that Danish model, high trust model with them uh abroad? And they did, and this was very interesting. I was in Poland and Russia actually doing my field work. Anyway, well, the unions got to know me through this work, and when the opening uh came for general secretary of what's called the Nordic financial unions, I applied and I got that job, so left academia and became a unionist. The short of all of that is that when I did that, it was like a whole new world opened. I could take what I intellectually knew and combine that with my values uh in actually, you know, fighting for, working for uh much better working conditions for workers across the world. So since then I've been working for unions in the Nordic region, European, global, before I left and created uh the Why Not Lab, so I can dedicate all of my time to this digitalization of work and workers and help unions in all regions of the world.
SPEAKER_01As you were talking through sort of that construct between we and they, I never really thought about it in that way, in that framework. And it's it's certainly, I think, here in the United States, something that we sort of that we and they is something that you hear very commonly in sort of the verbiages people talk about workers versus corporations. Um, and you noted that much of sort of this transition and shift of focus in your career sort of happened by circumstance. But I'm wondering, is there a particular circumstance or coincidence that um led you to kind of gravitate towards focusing on sort of the ethics of AI in particular?
SPEAKER_00Yeah, and again, that was a really low-key thing. I had bought a Fitbit, I had had an operation, and after that operation, I needed to get a little bit more fit. So I had bought a Fitbit. And at this time I was living in Belgium, and my neighbor in the building, he worked for the Belgian IT ministry, and he looked at me and he said, You do know now that that company knows exactly where you are, what you're doing, what time you get up, uh, how long you go exercise or how long you sleep and all of these things. And I was like, And of course, so true. And this was this was in 2014, so it's not that many years ago that it was like the penny dropped for me. And then I started digging into what is this datification of work and workers, how's that happening uh and taking place, how's that visible in companies? Uh, and that then led me to this whole debate about the ethics uh of artificial intelligence. So, so again, it was by chance, but it has uh really taken off from there.
SPEAKER_01Yeah, and I'm curious, you know, before we dig dig deeper into sort of the ethics of AI, from your perspective, particularly as sort of the lens of sort of the worker's perspective, you know, what do you see or how would you characterize the potential upsides of AI versus the potential downsides of AI, right? Because there's lots of conversations about the good things that AI can do. Um, but there's also more conversations about the potential risks and downsides of AI.
SPEAKER_00Well, let's take the potential upsides first. It's always good to start on a positive note. Now, what do we first mean by AI here? I will, you know, stretch the interpretation to any computerized um rule-based system of somehow using algebra to create some truths. Now, this there's all sorts of versions of this, but what we want to focus on this is mathematics statistics, something that most of us, not you on the call, but really uh did not like in school. Now, what uh is happening there is we see a quantification of work. So workers and the act of working is being turned into numerous data points. And these can be data points, for example, around sick leave. This can be around staff turnover. This will enable companies to work and say, hmm, you know, why in this division do we have uh particularly high sick leave amongst the employees? Or we could also focus on, for example, the redistribution of work. Some workers work too much, some are underemployed. How could we redistribute that? The problem with those things, I mean, I also have a dream of an algorithm that kind of trawled through a company's behavior to ensure that it was in accordance with the collective agreement, for example. This would be easy, relatively easy to design. But the problem is they don't exist. And what we see is that a lot of, you know, how do we be good? There's not a lot of venture capital in being good in the workplace. But then you have more automation, for example. We see across the world this dire lack of healthcare personnel. Japan is leading the way in the automization or robotization of care, for example. But also on a small scale, this could mean you know taking away heavy lifts from uh the care personnel. So there's lots of potential upsides, but I want to underline the word potential. Now, on the negative side, this is where, of course, uh it's easier to focus on because what do we observe? And what we observe is the effects of many of these systems being deployed. It could be automated hiring systems, automated scheduling tools, whatever. What we observe here is a range of negative impacts on workers or indeed certain groups of workers. The first one is, and I really like uh for calls panoptica. The fact that we might be surveyed, we might not be, but the fact that we might be is changing workers' behavior. So, for example, the always-on culture that we talk about now, where work-life balance is all going sort of into mushy mushy into one. Do you answer that email at nine o'clock this evening or do you not? You know, the productivity emails from Microsoft, how creepy are they really? Um, who else is seeing those uh productivity emails? How is this changing when you log on to your computer, when you reply to emails, uh, et cetera? But then we have the biggest and largest and most heartbreaking of consequences, and that's bias and discrimination, especially discrimination. Most AI systems, and I'd love to hear your opinions on that, uh, on the call here, but most AI systems on trade and on the white man, i.e., the consequences of these are very often that they marginalize the already marginalized. They discriminate against people of color, against any anomaly to the data set. And we see this really popping up all over the place uh in workplaces across the world. So another downside of this is, and and another debate we should have today, is this is moving the debate from ethics. Is it okay to deploy this system? Is this system good, to a question of actually of human rights? Our human rights, number one in the article one in the Universal Declaration of Human Rights, every human being is born free and equal in dignity and rights. You can say that's already being violated by many of these systems which are non-representative. So I could go on and on about the negative uh impacts, but I think the ones that we see the most here is the discrimination, it is the work optimization, and then the third one, which is very alarming, I think, is a sort of a de-skilling. It is taking away the professionalism, the autonomy of many, many workers.
SPEAKER_04Yes, I I feel uh a kindred spirit with you because when I first started preaching to corporate boards and executives about security, uh I was met with uh uh interesting responses because they uh were too excited about the new capabilities and techniques that were now available to them, new ways to reach customers and so on. And the security thing was a blanket, a wet one. So when you talk to corporate boards and executives about AI ethics and AI and human rights, what's been their response?
SPEAKER_00So a very interesting one. Uh, and I would say one of first uh a room in silence. I've had many opportunities to discuss with corporate board members, but also managers on lower levels, and I asked them typically one question. I said, Do you understand and govern the technologies that you are deploying in your company? And that's when the room goes silent. What is happening is something I call the managerial funds, that these technologies they're either brought in because the board can see that the competitors are using this and this system or the market somehow is trending in that way. So a lot of companies are buying in these tools, they're deploying them without having the necessary transfer of knowledge and without having the very important clear division of labor between those who are deploying the system locally, centrally, the IT department, etc., etc. So in general, uh the the impression across the board is that they do not engage in governance. If they do, it's some of the big companies who have ethics boards, for example, but those ethics boards are not representative. They don't include representatives of the workers of various people of various color, uh, age, and so forth. So sadly, and I want to give one example. I was uh giving a speech to some employers in a very digitalized sector, and I asked this question so do you govern uh these technologies? And he looked at me and he said, No, we are on average introducing a new system every four months. We don't have time. So the whole point here is they are deploying systems which are often third-party systems, which are often therefore designed or developed, not in the location where they're being deployed. They are then being deployed blindly. And here we have uh sort of what we've spoken about, digital colonialism is at stake because the values embedded in how do you find a good employee in the automated hiring system, you know, a good employee that has a different definition in Minnesota than it does in Denmark, than it does in Japan or South Africa. But you see, these systems are being deployed without question. And this is something that concerns me deeply.
SPEAKER_01So it sounds as if one of the risks that you're seeing in this, and and please correct me if I'm not phrasing this correctly, is perhaps an over-reliance on third parties. You know, these companies are coming in, bringing in these new tech, cool technologies that they don't have control over. Maybe they don't fully understand how it was put together, and they're putting it in across or using it across their enterprises or businesses and dealing with the data that they get from that. But outside of that, there can be risks when you're completely over-relying on another party to put in a system to help process this.
SPEAKER_00No, you're you're absolutely right, Emma. And and what Charles did, I think was interesting around the security of what you experienced there. It's the narrative. How strong the narrative of, oh, use this tool and your productivity and efficiency will be increased tenfold or a hundredfold. You know, if you and we did this in some of the training I did for public services unions across the world, I looked at what is the narrative in the local newspapers and you know, the media in those in all regions. And the narratives were exactly the same. And you know, we have really fallen for them. It's it's incredible how those pushing the narrative, not least big tech, but then you have all the consultancy companies underneath who are doing theirs and doing a good job about it, about pushing these um these uh narratives of productivity, efficiency, fairness, and so forth. So, so yes, it's it's very much narrative driven, and it's very much driven then by a sense of urgency. There's almost a sense of panic in some companies. We must you know have this system, otherwise, we'll lose market shares. And then they don't have you know, the consultants there, they don't come with the governance model, and they don't come with what to watch here uh to ensure that harms are not um inflicted upon workers or citizens.
SPEAKER_04Well, again, drawing on my experience with the security or no, um occasionally I'm met with um I don't want to say hostility, but outright um disagreement. And so I suspect you've had a lot of silence, as you said, but you may have also gotten into some very uh argumentative uh discussions, at least on their side. Um what what are they arguing for?
SPEAKER_00But that is that belief in no, no, I'm gonna turn that around. It is so interesting because you're absolutely right. You know, I I get into very heated debates with people. Also, some workers, you know, uh, but I'll I'll go into why in a minute. But what they mainly do is if you express some kind of reservation towards technology, you are equal to a Luddite, right? There's no gray area here. Either you love it unconditionally, or you are a Luddite. And some unions, some workers have even said to me, you know, Christina, you're either you're rather dystopic, you're dystopian around the future of this technology. And I say to them, I would rather have you get a very critical lens on this and then turn out to be pleasantly surprised than the other way round. So it's incredible. If you take the narrative, you take this again narrative, if you don't like technology, you're a Luddite. Uh, then you have a total uh highway into the free implementation of a lot of these technologies, which interestingly are taking away the power of the deploying companies.
SPEAKER_04Yeah, yeah. Well, we could go on about that, but I I'm really curious about this why not lab. Uh why did you set that up and and how does it operate?
SPEAKER_00So, you know, as as I started progressing through helping build uh Uni Global Union, which is a global union federation, their whole department on the futures of work, the future of work, and I started digging deep into this thing called data, AI, and all of that. I very soon realized that if the union movement in relation to the world of work, if it's not from the unions, where would the resistance or the lust for change come from? If they didn't get on top of this, then I feared and still do that in 10 years there would be no need for unions because the power imbalance would have grown so large between those who hold the data and those who don't. So, you know, I really wanted to dig deep into this. And of course, you know, unions are not uh necessarily finance, so you could have a nerd of that kind. And so I was working on all sorts of other things, and I thought I have to dedicate myself to this and dedicate myself thereby to the workers. So I took the leap and it was very scary. Um, but it's it's worked out uh extremely well, and it's it's just a joy to work with organizations and people that I believe in.
SPEAKER_02Let me ask a couple of follow up questions to what you were discussing earlier. Bias, I agree, is a serious problem, but if people are serious about correcting it, it's a correctable problem. And in fact, a better output of data would have a More diverse training set. There's no question about that. What I'm concerned about though is that even if you correct bias, there would still seem to be some potentially questionable uses of AI that have to do with AI enabling us to aggregate data that we didn't think would be relevant. We had an earlier episode in which we were discussing lending decisions, and it was explained to us that now SAT scores can be used because it's been found that there is a correlation between a higher performance on an SAT score and a higher creditworthiness, but that doesn't give the person whose data is being used and who is seeking the credit an opportunity to improve their credit worthiness because the SAT score may be years earlier. You and in an earlier conversation that we had offline, I I recall talking about your first experience in Belgium with a health issue. Could you elaborate on that? And then could I ask a follow-up question?
SPEAKER_00Sure. So yeah, I mean, this again was one of those decisive moments in my career, so to speak. I was living in Belgium. Uh, there were rumors that the owner of the flat I lived in wanted to sell it. So I went to the bank and to ask whether I could get a mortgage big enough to potentially buy this flat. And I, you know, I had a good wage, no debt. And um after a while, uh, the person in the bank frowned, pushed back from the chair and the desk and said, sorry, we can't offer you a mortgage. And I was like, Well, pourquoi, you know, why? And um it slipped out of her because of your health file. Now, of course, that was illegal even back then. This was before the GDPR, but it was still illegal. So I of course went into investigation here how the bleep had they gotten hold of my health file. And various experts kind of believe that it was as simple as that in in Belgium you have private health insurances that are obligatory, so private and private, but then you can buy aggregated health data about, you know, what is the state of affairs of the health of the Belgian citizens, you know. And most likely they have taken a data set like that, triangulated that with when I'd use my credit card to pay pharmacy bills and when I'd used it to pay hospital bills. And that they could show almost to the degree that they could disaggregate the data and find out that I had had a brain tumor. So I was too high a risk, you see. And this is again one of these moments which has really, really got me thinking about anybody who for any reason is outside of the norm, because they're overweight, because they have the wrong skin color, because they're too old, too young, uh, undereducated, whatever the norm that's been defined usually by the white man is. So this this was one of those moments where I have to say I it really put a rocket uh behind me to really start investigating all of this datification of work and workers.
SPEAKER_02Right. And the the follow-up for that is that again, assuming that the bias problems are corrected and main, you know, the correction is updated routinely, it doesn't change the fact that the digital users continually trade convenience for data that they don't realize will be used in ways that if they had known, they either would have said, you don't have permission to do that, or I don't want to give the data. And the flips, the other side of that is what about the workers who themselves discover they're using data this way, and they may be just quite uncomfortable when they see that this is what they're doing with everybody else's data, but it's part of their job description now. Could you comment on both of those?
SPEAKER_00Yeah, and a really, really sharp remark there, Roland. And I want to bring in Tim Wu here. And back in 2018, he wrote an op-ed in the New York Times called The Tyranny of Convenience. And I really recommend everybody to get hold of that article. It's not very long. But I think the the word you said convenience there, this is how we're being played, gamified. And I really want us to realize the extent to which our addiction to social media likes, our addiction to the ease, the convenience of some of these tools is what we're paying with there, is our privacy, our human rights, our data, uh essentially. But that goes back to the narrative thing, right? It has really, I think if we counted how many psychologists uh were employed by big tech, then then we will see the degree, you know, how how many people they have there to continuously hold us addicted uh and and sort of at awe uh on these technologies. So so that is is a key uh thing, and and and one of the main reasons why the resistance to the technology hasn't been there. You know, everybody on this call, if I may, I think we're all above 40. You know, we slapwalked. Now, some of you, of course, specialists that you probably didn't. We kind of slapwalked into this defecation. We never really asked, hmm, how is this working? Although we do all realize that there's nothing called a free lunch. So it was not until Facebook Cambridge Analytica scandal that across the board people started going, hmm, you know, what's going on here? So we need to be more critical, we need to ask the questions and on whether bias can be um somehow um lessened through uh the working of the data set. Well, yes, but you need to know what to demand. And if there's any word that describes the current power imbalance between, I would say, the developers of these technologies, then the deployers of them, and then the workers is you don't know what you don't know. You don't know what you should be demanding because these systems can have this and this and this effect. So, really, that saying you don't know what you don't know is is keeping us in this trap of convenience and more and more and more, and as Alma will say, this data hoarding. Uh, but we don't know what questions we should be asking or what demands we should be giving to tidy all of this up. We just don't know.
SPEAKER_02But are you troubled more by the use of our personal private data or by how workers may discover later that they have no power in their company outside of, let's say, the trade union movement in Europe, to influence the way a company decides to enter into agreements for purchasing and using data in their AI machines?
SPEAKER_00Well, both and, right? I mean, I would say they're two sides of the same core coin here, that if you have a hoarding of data, at the same time you have an unknown consequence of the aggregation of this data into influences profiles, then again, you're kept in the dark. You just simply don't know what you don't know. And I think Shusanna Subov uh from Harvard Business School, she she really is strong on her demand that we end markets in human futures. And for her, that means end uh the markets in uh in buying and selling uh data sets and profiles and influences uh on people. Now, this is what is happening behind the scenes. This is what is happening constantly, that we're measured against this uh mathematically defined norm. And this is closing doors for some people, opening some for the others, but we just don't know.
SPEAKER_01So, part of this conversation we've been having has been talking about, and understandably so, kind of on the perspective that corporations and workers bring to the discussion. But you know, there is a third player, third actor in this conversation, you know, what we broadly call the legislator or the regulator, um, and interested in kind of hearing, to the extent you've been involved in those types of conversations, what they've been about, and sort of the response that you're hearing to some of these topics that we've been walking through.
SPEAKER_00Yeah, that's another really interesting and big question. So, yes, I have been very much engaged in the digitalization of uh public services. Uh, this could be in everything from predictive policing, predictive law, could be to workflows in municipalities or whatever, whatever, on the whole, across the whole range. What we see there is that the vast majority of digital tools being used in public services are third-party systems. So you have the same dynamic. You have a developer who has convinced a government or local authority uh of the wonders of this tool. Um, and very often what we see, although I haven't done an academic quantifiable study of this, but what we see very often is that the developers in cohorts with maybe the consultants say, you know what, here's the tool, it will help you figure out traffic flows through your city. Um, we will analyze the data for you as part of a free service and send you the analysis back. So, what we're seeing is the public services who are strain for money, uh, let's face it, COVID has been devastating for many, many public services. But what we see is an increasing digitalization of those public services through public procurement, through privatization, but then also a private analysis of this data, which then you can then say, well, what's happened to democracy if if the public service is dependent on private sector analysis, then you can ask whether the public service truly can govern in the public's interest when the interpretation of said data comes from a private company. And then you see a long-term effect of this. If you don't have the competencies in-house, but you have companies offering you to do the data analysis, well, then you never get invest in those competencies. And then you know, you have uh a catch-22 where the dependency factor is is increasing across the world. So this is very, very clear in public services and something again, which is accompanied by the lack of governance. You know, nobody is governing these tools to ensure that they do know how.
SPEAKER_05I I just want to weigh in with a comment. I I some years ago, when I was uh uh full-time employed, I um I needed to get an account on a government uh website here in the US. And I went to the thing and I said, please register. And it said, and I filled in all sorts of stuff and did various things, and it said, sorry, you can't register. Call them up. And so I called them up on the phone and waited a long time. Finally, I talked to a person and they said, Oh, you have terrible credit. That's why you can't sign on. And I said, What do you mean I have terrible credit? I have excellent credit. Well, it turns out that they had outsourced the validation of are you really who you say you are, to one of the credit bureaus. And the credit bureau only knew people who had mortgages. And as it happened, I didn't have a mortgage. And so, as far as though they were concerned, I was unverifiable. And that meant not that I was unverifiable, but rather that I didn't exist. And the only way I could actually register with them was to present myself in person to one of the few places where they existed with my passport to prove that I existed. So you're absolutely right. This this outsourcing, uh, the the lack of sort of thought about this outsourcing process is is and can be quite catastrophic.
SPEAKER_00Wow, um, yes. And so, I mean, your your case, and you probably had the social capital to fight back on this, right? Many, many people wouldn't have had that. They wouldn't have known where to start. So they would have been left in the dark, and your data reaped, but no benefit for that. Uh no, it's shocking. I mean, one thing I think with public services that we should keep a real sharp eye on is the increasing use of predictive systems. So we've had numerous examples across the world where the use of or deployment of predictive algorithms have really, really caused dramatic and negative effects on the citizens. I mean, for example, in Holland, the whole government had to step down because they deployed an algorithmic system to predict whether people were cheating with the child benefit system. If they were flagged that they were, the children were removed from the house and/or the uh child benefit uh was stopped and the payment was stopped. So a lot of people they were ended up in hunger. They ended up having to leave their house. They were devastated, of course. And then it wasn't until after 1,700 children were removed from their homes, forcibly, that they found out the algorithm was discriminative. It targeted and flagged the likelihood of non-uh ethical white uh Dutch people as being likely to cheat. I mean, you know, I mean, awful. And then you have in the UK, and this is on a less dramatic scale, but still dramatic because this is the future of the students, this automated um grading system of students. Now it turned out that it up uh sort of up-averaged uh children from rich areas and downplayed uh the grades from children from poorer social economic areas. We've had in Denmark, we've had uh an algorithmic system that was aimed to predict whether somebody would survive a heart operation. Over 500 patients were maltreated because the algorithm was wrong, etc. etc. I mean, in your country, predictive policing, you know, we could go down there and talk about that for hours. But what is stunning to me, or really one of these sort of like how, why, is we suddenly believe there is something called a glass ball. We suddenly believe we can look into the future and predict behavior. And this very shift in the norms is is for me one of the most interesting, but also one of the most dangerous uh aspects of the digitalization of public services right now. We somehow believe that we can predict negative behavior. And why are not more people demonstrating against that and say prediction belongs to either religion or something else, right?
SPEAKER_01Those examples that you just shared are, I think, are sort of salient sort of case studies of what Charles, when he was sort of starting us out on this discussion, you know, sort of the harmful unintended consequences, right, of these what seemingly be to be very cool technologies or technologies that perhaps will have potential upsides. Um and sort of listening to you walk through those those experiences in uh the UK and in Holland and other places, it made me think or question are legislators or regulators equipped to kind of deal with the governance challenges, right, um, that these AI technologies present? Um, the example you gave sort of predictive and analytics, right? You know, AI is constantly changing. That's one thing that we've certainly learned in talking to many of our guests. It's constantly changing. How are legislators able to sort of keep up with these issues? A, sort of understand the technologies, B, the harmful unintended consequences, and then try to figure out, C, sort of what are the criteria that they should be using perhaps to put some governance structure around this? Because as you were sort of discussing and laying out, there really is to some degree no quote-unquote governance, right, over some of these technologies.
SPEAKER_00Uh again, thanks. I mean, great, great, great question. The short answer is no, they they don't have the competences to do this. And this is extremely, extremely scary. Now, let's fly up onto the highest level and and look at one of these narratives again. About a year and a half, two years ago, I started noticing in the Global Partnership on AI and the OECD, where I'm part of their expert group, the mention of standards and certifications. This started copping up more and more and more. This year I was addressing the G7 in their political priorities for 2022 standards and certifications. In the EU, US Trade and Technology Council standards and certifications. Now, this is not good news. Uh, standards and certification bodies are 99.9% industry-led. They are private bodies. So this is a de facto privatization of regulation, if if anything at all. It's a non-representative um set of bodies. But also, and this is a question that I would love, yes, AI is changing, but there are some fundamentals which should and could, which really should be included in if we have any standard and certification, we should have a mandatory requirement to periodically govern these technologies. Now, I don't, I mean, I correct me if I'm wrong here, but I don't see the changing nature of some technologies as changing that requirement, a mandatory requirement of governing and then inclusively governing. How will you know whether the system is fair? One of the big principles in the OECD AI principles, if you're not listening to the voice of those who are subject to these systems. So, this is one thing in public services, this would be a member of the public, members of the public. If it's a workplace tool, it should include the workers. So this is not happening. My last very eerie feeling around this fallback, it's like there's a growing political consensus towards standards and certifications, is um the very notion, how can you certify something that by nature is changeable? Yeah, this for me, and I've asked the politicians this doesn't can you kindly explain? You can certify, like they do in Europe, like they're discussing in the EU, US, the Trade and Technology Council, is market access. So we all certify this automated hiring system, a one-off certification. It enters the market. Yet we all know these systems are fluid and changeable. If you do not have a requirement and a mandatory requirement of ongoing governance, then literally what our politicians are doing right now is handing all power and decision making to a thing and not to uh regulators. Very, very dangerous. Uh, what's happening?
SPEAKER_05A term that um a phrase that is very much suggested to me by some of the things you said has resurfaced recently in the uh current political environment, uh, which I learned in the process of studying it dates back to the early 1500s in the Polish independence movement, which was nothing about us without us. Um, and it's a very powerful, very simple framing of the key, of the key theme that I think emerges from a number of the things you've been saying.
SPEAKER_00Exactly. And I it's funny you should say that because I have that on lots of my slides, you know, nothing about you without you. It's almost common sense, but the world is pulling in the totally opposite direction. Luckily, there are some some you know attempts uh of citizen assemblies, for example, increasing democracy, increasing having a say over particular um changes. But our world needs re-regulating in order to really make democracy thrive. Democracy first, profits second. At the minute, we're heading down the total opposite. And this is my sense of urgency in the why not lab is to raise enough critical awareness around these things so that people can start tabling an alternative uh digital strategy.
SPEAKER_02Can I ask a follow-up on that? You've spoken several times very convincingly about the importance of narrative. How optimistic are you that even if your recommended changes were considered seriously, that they could be introduced, given the power of the narrative, against adopting measures that would make things inconvenient? I mean, uh if you imagine a Pandora's box and we open it and all these crystal balls for predicting come out, the the thing that they all seem to have in common is they're cool, captivating, and convenient. How do you sell companies or masses of individuals to give those things up when the companies themselves are pushing ever more rapidly down that road? Would any regulation actually be effective? I don't mean to be so skeptical, but I'd like to know as somebody who's trying to recommend regulation, I assume you you have to think in your mind that there's got to be some point of leverage that would be tolerable and effective at the same time. But is that really true?
SPEAKER_00Love that question. This is the one I go to bed with at night, and I wake up in the morning trying. Figured out the answer to. Yeah, my answer varies from day to day, Roland. And this is this is very much uh a question of uh the tyranny of convenience that Tim Wu once said, the power of our institutions. So the market. Now, if we want to change how various systems work, be it a digital system, be it a climate, whatever, whatever, we're gonna have to change the market. We're gonna have to start rewarding behavior that goes in that direction of being more environmentally friendly or digitally putting human rights first, for example. Now, our market is human-made. Our market was, and all of the institutions around that were made by uh people before us. We can change them. This is my uh sort of, I'm clinging on to that hope, I'm clinging on to that realization that we can change it, but then you need a number of people, the majority, to demand that change. But as long as you're kept in the dark, uh, as long as you sort of, oh, you know, the too many of convenience, well, this is great, you know, although there's a nudging feeling something is wrong, uh, as long as we're kept there, then resistance won't grow. And I know, and I'm deliberately using the word resistance here, and that's probably provoking a lot of people, but this is what we need. We need to say no. You cannot hoard my data, you cannot make all these inferences, you cannot determine my life before I've even had a chance to form it. This is, you know, um, again, a breach of human rights. So am I optimistic? Sometimes no. Am I optimistic? Sometimes yes. But I know that if if people like me and all of the others who are critical towards this, if they shut up, then we definitely will never uh get the changes we need.
SPEAKER_01In all these discussions that you've been having, right? I think it would be interesting to hear are you seeing uh generational differences to how people are approaching these issues? You know, you mentioned resistance. Um are you seeing that resistance from particular spectrums of the age group, gender? Just sort of curious if you could give us a snapshot of where the differences in terms of this, and as you reference resistance is sort of coming from.
SPEAKER_00Yeah, uh great question. Um yes and no. Um again, I want to paraphrase you don't know what you don't know. So what I can see is if I have been giving workshops or training for specifically young workers, then you can see they start questioning some of the things, some of the truths that are in their workplace. You know, there's the use of an automated scheduling tool. Well, then they might start asking, well, what data are you using? From what sources? How long is it kept and all of that? But you see, we still need to have confidence in asking those questions. And although it is mind-baffling, you know, how getting that confidence. I know enough to start asking these questions that is still not embedded in the wider labor movement or especially in the young ones. But they are interesting because they are making my generation look like uh we've been slaves to the labor market, right? They walk in and say, we don't want to work evening shifts, we don't want to work weekend shifts. They have much higher demands, they want a high wage than an ascot, you know, after 30 years of service. But sometimes they don't criticize the technological thing, and that is they are the generation of technology. They have never lived before the smartphone like the rest of us have. So, again, you don't know what you don't know. This is my mission. We should make sure more and more people know so more and more people can start asking those necessary questions.
SPEAKER_02I want to share an anecdote with you that uh with a digital generation versus a you know digital immigrant like myself. Uh, about six, seven years ago, I was giving a guest lecture up at Columbia Law School uh on copyright. And near the end of the lecture, after they had sort of exhausted the questions, they wanted to ask, they asked, you know, what can we do to improve our chances of getting a job in New York City in a law firm? Uh they didn't ask about government, they didn't ask about not profit, but they wanted to know about law firms. And I said, well, um leaving aside your performance at law school, one of the things I would urge you to do is go to your Facebook and other social media pages and scrub them. That won't stop people from entering, you know, using Time Machine or other previous website viewing models uh to see what you had before, but get rid of anything there that might make an employer say, is this really the sort of person we want? And they said, Why? They don't look at those things, do they? And I said, you know, just because we who are older than you don't use these technologies frequently for our social engagements doesn't mean that we're not smart enough to use them as intelligence about the people we're possibly interviewing. And they were shocked and dismayed. And I said, Well, but this shouldn't surprise you. Now, AI is now doing this more and more, but it coincided with that summer, um, an associate working at a firm who made a statement on his Facebook page that he didn't intend to stay at this firm if he got an offer for more than two years and then he was off having paid his bills. And they did not make an offer to him chiefly because of that disclosure. And I think one of the few ways you have of leverage, you say you don't know what you don't know. One of the most powerful things is to tell people this is what could be an impediment to you or uh a means of blocking your choice in the future because this information was aggregated from your use of four different websites, none of which told you that they could be used to triangulate something adverse against you.
SPEAKER_00Yeah, totally. Totally. I mean, I totally agree with you. And that's also, I wrote a little article around my top 10 tips uh for protecting your identity in this digital age. And one of them was, you know, to daily or to ensure that your browser deletes cookies when you close your computer. Another one was you know, to use a VPN or whatnot, whatnot. But you see, again, uh some people say, oh, but my computer gets slower, my mobile phone, my cell phone gets slower when I use this system. So you're up against this tyranny of convenience again. You're up against, oh, but it used to be so fast, and now it's a little bit slower. No, no, no, I don't want to do that because you know the ease of it before was better. So yeah, I mean, I do believe that we can by mentioning some of the most sort of common uses of um social media data to profile us. The more stories we tell about that, the more people will realize, well, I won't necessarily be thin and sporty and and and healthy the rest of my life. Maybe I should start protecting my identity. Or as I usually say in my trainings, you know, a lot of people say to me, I have done nothing wrong, so who cares if they take my data? And you know, then I have to say to them, this is not just about you, this is on the country, you know, all of the profiles being made on us comparing um somebody with your education, your age, your location, your postcode with economic performance or whatever. So there's a lot of myths that we have to break still, and uh that's that's what's keeping me busy.
SPEAKER_04Well, this has indeed been fascinating. I I'd I'd like to close with one more question. And you've touched on you've you've danced around it and touched on it here and there. Are there specific groups in society that might be more at risk if these issues aren't addressed at least in the future? You've mentioned human rights groups, but you know, sections of society or groups of society that might be really uh facing some trouble.
SPEAKER_00Well, anybody on the margins of society, any minority group. Now, this could be a minority group in economic terms. So, you know, the blue-collar worker or something. It can be by gender, it can be by ethnicity, it can be by a range of things, but we are marginalizing the already marginalized. And this is a particular, particular concern for me. Um, that it is the majority, uh sort of the hegemony. This is why I talk about digital colonialism, it's the hegemony who's deciding whether you are fit for a job, alone, uh, whatever, whatever you are, whether you are a communist because they've been scraping your Facebook and seeing you you're a union member, for example, right? The consequences of that can be enormous in out in the degree that you have never imagined in your life. So, you know, I work with several scholars. One of them, she works with the concept centering the marginalized, and this is something I think we should really consider is how do we, through governance, bring the voices of those who are subject to these systems into the core of the governance of them. What we have to there realize is that we're up against the promise of speed and efficiency of these systems and the slugginess of governance. It takes time. And here, you know, we humans are pretty lazy, right? If we can get away with not doing anything, then we'd rather do that than take the necessary time to ask the necessary questions. This is why I think we need regulation from a national state level, uh, in your case, um, to demand that they take this time.
SPEAKER_02It almost sounds like they're suggesting also that civics, which at least in the United States, used to include the teaching of a little bit about the constitution, a little bit about the rule of law, should include teaching what an emerging widely deployed technology like AI is and what its upsides and downsides are, because if we're familiar with the digital divide between schools that could teach computers, could teach coding, could could have computers available. I would have thought the AI divide will be even more pernicious if we don't include in as a fundamental teaching what AI is and its risks and benefits of the kind that you've been discussing with us today.
SPEAKER_00Totally agree. I mean, I totally agree. But they there you have to also ask the question, you know, that you have the freedom of expression, you have the freedom of assembly you have enshrined in both the US Constitution but also in human rights laws. You have those. And how can you have the freedom of thought, for example, if you're continuously being manipulated? If doors are closed to you or certain products are advertised to you and certain not because the algorithm thinks that this is what you need. Do you then really have a freedom? You know, we we have the right to be free from manipulation, but having that discussion, you gotta get people into the whole notion of law, the rule of law, human rights, uh, the US Constitution, we've got to make that sexy again. So apologies for the expression, but this is this is very much needed.
SPEAKER_01Thank you very much, Dr. Kolkoff. You know, this has been a very insightful um and thought-provoking conversation. And I think it is also uh a voice and a lens and a perspective that I think it's important for our audience to hear. So we thank you very much for your time today. It's been a wonderful discussion.
SPEAKER_00Thank you really for the discussion and for having me. And uh, let's continue uh discussing.
SPEAKER_04We thank the business law section of the American Bar Association for their generous sponsorship of the production of this podcast. We welcome questions and comments from listeners. Send email to comments at mind of the gap dialogues.com. We read all comments and questions and will try to respond in the letters section of a future episode. If you're writing about a particular episode, please mention the specific episode number. And please also include pronunciation tips to help us properly say your name when we reply in a subsequent episode. See you next time on Mind the Gap Dialogues on AI.
SPEAKER_03Thank you for listening to the AVA Business Law Sections Podcast Series to the extent that the section offers a robust collection of content. To explore more about this topic or to learn about joining the section, visit ambar.org slash bizlaw. That's B-I-Z-L-A-W.