Mystery AI Hype Theater 3000

Uber for Military Surveillance (with Niamh McIntyre), 2026.03.30

• Emily M. Bender and Alex Hanna • Episode 75

Share episode

Use Left/Right to seek, Home/End to jump to start or end. Hold shift to jump forward or backward.

0:00 | 53:44

The people who make automated translation possible are often low-paid gig workers. Usually, they don't even know who they're really working for — and it might be the US military. Reporter Niamh McIntyre joins Alex and Emily to dissect how one data labeling company presents its work, based her investigation into the experiences of East African employees.

Niamh McIntyre is a senior reporter at The Bureau of Investigative Journalism (TBIJ) in London, covering AI, labor, and surveillance tech. She was 2023 AI Accountability Fellow at the Pulitzer Center, and prior to joining TBIJ, spent four years as a data journalist at The Guardian.

Find tickets to our April 30th live show here!

References:

Also referenced:

Fresh AI Hell:

Check out future streams on Twitch. Meanwhile, send us any AI Hell you see.

Find our book The AI Con here, and MAIHT3k merch here.

Subscribe to our newsletter via Buttondown.

Follow us!

Emily

Alex

Music by Toby Menon.
Artwork by Naomi Pleasure-Park. 
Production by Ozzy Llinas Goodman.

Emily M. Bender: Hi everyone! Before we get started this week, we wanted to let you know that we’re going to be doing our first ever live taping of the pod, on stage in New York later this month! I mean, even more live than a Twitch stream, since we’ll be physically co-present with our live audience. We’ll be at Starr Bar in Brooklyn on Thursday, April 30th at 6:30pm.

Alex Hanna: As anyone who’s joined our Twitch streams will know, this podcast is even more fun when you’re watching it live! You can find tickets on the Distributed AI Research Institute’s Eventbrite page, and we’ll have a link to that in the show notes. It’s $10 for students, $20 for everyone else, plus fees. All proceeds will support DAIR’s work.

Emily M. Bender: Can’t wait to see you all there! Okay, onto the show.

Alex Hanna: Welcome everyone to Mystery AI Hype Theater 3000, where we seek catharsis in this age of AI hype. We find the worst of it and pop it with the sharpest needles we can find. 

Emily M. Bender: Along the way we learn to always read the footnotes, and each time we think we've reached peak AI hype, the summit of Bullshit Mountain, we discover there's worse to come. I'm Emily M. Bender, professor of linguistics at the University of Washington. 

Alex Hanna: And I'm Alex Hanna, director of research for the Distributed AI Research Institute. This is episode 75, which we're recording on March 30th, 2026. Our guest this week is Niamh McIntyre, a senior reporter at the Bureau of Investigative Journalism in London, covering AI, labor and surveillance tech. She was also a 2023 AI Accountability Fellow at the Pulitzer Center, and prior to joining TBIJ, spent four years as a data journalist at The Guardian. Niamh, thanks so much for being here. 

Niamh McIntyre: Yeah, thank you for having me. I'm really looking forward to our conversation. 

Emily M. Bender: We're gonna have some fun. We wanted to talk to Niamh because she just published an amazing investigation into the experiences of gig workers in East Africa who work for the data labeling company Appen. Specifically, how these workers may or may not be secretly employed by the US military. You can find the article online at thebureauinvestigates.com. 

Alex Hanna: Of course, this is a topic that researchers at DAIR have been keeping an eye on with the Data Workers' Inquiry. So today we wanna look closer at what some of these data labeling companies are saying about their work. Companies that have contracts with the US military, and are also hiring contingent gig workers. So let's get into it. 

Emily M. Bender: Here we go. Okay, so first artifact, Appen's blog post here. "Beyond do no harm," colon, quotation mark. Weird. "Why AI must be ethical and responsible." And this is by Carl Middlehurst, from February 2023. A little bit older than the artifacts that we usually go through, but we're gonna see why this is highly relevant today. So I have to say, aside from the really bad punctuation at the top here, where like, why is that quote mark outside the colon? This little pull quote at the beginning is very weird too. It says, "The pace of technological change is increasing. The long-term uptake of AI will depend not only on society's willingness to embrace new technology, but also on the-" and it just- 

Alex Hanna: Yeah, it just stops right there.

Niamh McIntyre: I feel like some parts of it look like they got AI to write it, and then some parts of it they really could have done with putting it into AI to check that it's a complete sentence or something. 

Emily M. Bender: Yeah. Or at least a grammar checker. 

Alex Hanna: Yeah. 'Cause I guess this is just after ChatGPT came out. So that concludes, so it's, "The long-term uptake of AI will not only depend on society's willingness to embrace new technology, but also on the willingness of those developing the technology to embrace all of society. Generative AI is the latest breakthrough in AI, and like many cutting edge technologies, generative AI is facing its own teething challenges." So, just a few things here. This is weird to say generative AI is a breakthrough in AI, which again, not coherent. And also teething? Weird child metaphor. 

Emily M. Bender: There's the bat signal for Anna Lauren Hoffman, right?

Alex Hanna: Yeah. And then we're starting to get into the kind of aspect of the humanness. So, "Society sets a higher standard for AI than it does for humans," which, weird declaration there. "It's not sufficient for it simply to do good. It must also not do bad, and there are no allowances for quote 'human mistakes.'" So yeah. Thoughts on that, Niamh? 

Niamh McIntyre: Yeah, it's a strange construction, because the point of this blog is justifying the kind of services that Appen offers. And it's a company that sells training data, but also has a gig work platform where you can acquire contract labor to help improve your AI models. So I think it's setting itself up to be like, with our help, your AI system can meet these really high standards. But, yeah, it is a strange construction to say that society sets a higher standard for AI and doesn't allow human mistakes. When it's like, we see mistakes that AI makes constantly, as the point of this podcast records.

Emily M. Bender: Yeah, exactly right. And it's just a category error to say that we set a higher standard for AI than for humans. We expect, for example, if we're gonna use a knife to cut something, that knife is going to be effective as a knife. And not comparing it to, I don't know, tearing the thing with your fingernails. But because AI is mythologized as something human, they get away with saying weird things like this, that really don't make sense. And you're absolutely right, Niamh, that the mistakes are all over the place. And it's weird that they say "human mistakes" with scare quotes there, as a way to talk about system errors.

Alex Hanna: Yeah, it's certainly a bizarre thing, given what the service is. I'm wondering, Emily, do you want to take this? This one with "AI is already doing a world of good." 

Emily M. Bender: Yeah. So headline here, or the major section heading, "AI is already doing a world of good." And then, "Media buzz and headlines aside, AI technology has been working behind the scenes to make sure we have a better, healthier, and more equitable future." And then they get into these examples, which seem pretty far afield from their core work. So, "AI can help end world hunger, one crop at a time." And they're talking about something to do with data processing in the context of planting. And we can read that if we want. Second one is "The role of AI in the fight against climate change." Which we have a lot to say about, but I think here they're pointing to how to handle maybe balancing energy demands or something. And then finally "Helps the world communicate," which I think, if I'm not mistaken, is Appen's core area of business? 

Niamh McIntyre: Yeah. Or it certainly was in the early stages of the company, 'cause it's been around for a really long time, since the nineties. And it was founded by linguists, and it started to build linguistic data sets, and serve companies that were building translation, transcription tools, all kinds of natural language processing. And then over the course of the 2010s, it's broadened into doing other kinds of AI training work. But, yeah, this point, I think, is more related to its core work. And Appen's marketing its work as this sort of altruistic thing, which is all about helping us communicate, widening access to information, et cetera. Which, yeah, when we started reporting on the company, they were building and selling languages for other purposes, too. But yeah, we can get to that, I suppose, in a bit. 

Emily M. Bender: Yeah, for sure. So I wanted to jump into this one about "helps the world communicate."

Alex Hanna: Yeah, go for it.

Emily M. Bender: "It's important to recognize the significance of translation. Without it, most of the world would be unable to read texts written in unfamiliar or forgotten languages. This could result in the loss of important historical and cultural events, or important scientific and medical discoveries only being shared within specific groups. Language connects people from all around the world, but inadequate translation can lead to inequitable access to information. One of the way-" again, I could have proofread this- "AI helps to resolve this is through chatbots and other conversational AI powered platforms that are trained on diverse data sets that are free from bias. Gathering data from people around the world, from various demographics, ages, religions and cultures speaking different languages and dialects helps train computers that broaden the world's access to critical communication." So there's so much that's making me mad in this, but the first thing is, it's not like translation doesn't exist without machine translation. Like translation's an activity that people have been engaged in ever since the first groups that spoke different languages encountered each other, like that's a thing that we do, that we learn how to do. And I'm not denying that machine translation is useful, like I certainly make use of it. There's a time and a place for machine translation. And I think Appen is really trying to guild the lily here by saying they are solving all of these problems, including, by the way, getting access to text in forgotten languages, which, yeah, that's not gonna happen. I don't care how much machine translation compute you throw at it, if the language is forgotten, the language is forgotten. So this is very much, we're the good guys, we're the heroes. 

Alex Hanna: Yeah. Niamh, I'd love to also bring in some of your reporting and kind of the ways in which, the particular languages and the particular people that you were talking to in your reporting, and how Appen was engaging with them and those data workers.

Niamh McIntyre: Yeah, absolutely. So Appen has this gig work platform, and a lot of the work on there is linguistic. So it might entail giving a worker a certain amount of audio in a particular language, and they'll transcribe it, is often what the work entails. And a language that we looked at a lot in our reporting was Somali, which is quite an interesting one, for a number of reasons. It fits into this, the framework of the blog post, in that Somali is what's called a low resource language. Low resource language, it wasn't a term I knew before I started working on the story, 'cause I'm not a linguist, unlike Emily. But it just means there's not a large amount of readily available material that you could train a model on. And so companies like Appen, if you are trying to build a system that can parse Somali language, will sell you a data set, or they'll connect you with workers to help you achieve that. And some of the data sets Appen has, you can just buy off the shelf, you'll pay them a certain amount of money, and you can do what you like with it. Or sometimes they'll build a bespoke one for you. And we found that in a number of cases, these Somali language data sets were sold to various US military units. And obviously the workers who are doing this work, would've had no idea about that. It's quite standard practice for this kind of gig work that you aren't explicitly told who the client is, you are just logging onto the Appen platform. And Somali, obviously, is spoken in a place that has seen a conflict that's persisted for decades. So there's a lot of internal and external displacement. So a lot of the people who were doing this work for Appen were refugees in Kenya. Either first generation, or they might be the children of refugees. And also because of all these structural issues, it meant it was also very difficult for Appen to pay people in Somalia, so they had to recruit from Kenya. But what that ultimately meant was that in some cases, Somali refugees who are living in camps in Kenya were, without knowing it, ultimately working for a military that was quite actively involved in conflict in their home country. And you see the way in which this marketplace for low resource languages overlaps with US military interests quite substantially.

Alex Hanna: Yeah, for sure. And I think it's, even the term low resource is such a weird one. I don't even know if it's- I'm curious. I hate it. I think it comes more from the natural language processing field than linguistics, per se. 'Cause I feel like linguists are not, like yeah. But I'm curious what- 

Emily M. Bender: Yeah, no, I can speak to how that term is used. And it is a contested term in natural language processing. So basically, it refers to languages that might have very few speakers or very many speakers, but for which there are not very many digitized linguistic resources. So we're talking dictionaries, we're talking morphological analysts, we're talking just corpora, and then in the machine translation context by text. So you've got, typically, especially the US military wants English on one side and this other language on the other. And then transcribed and aligned corpora where someone has spoken. So all of those are considered resources. And part of the contestation within NLP is that you have, I think, three kinds of languages that get lumped under the low resource language banner. There are languages that for various reasons have military interest, especially to the US military. And for a long time, Pashto was a very highly focused upon language, let's say. You have languages where there's a very large market in terms of the number of people- this is like many of the languages of India, for example- but relatively little language resource development. So widely spoken languages, standardized lithography, just not a whole lot in the way of digitization. And then you have languages that are, the languages that have been stolen from indigenous people, basically through the process of colonization, where you may have very few speakers, you may not have a standardized lithography. Many of these languages, anybody who speaks it is also going to be bilingual in English, Spanish, French, Russian, whatever the colonial language is in the area. And there's a lot of work that is actually funded by US military to work on low resource languages, but dresses itself up in this white savior mode of "we're gonna help the endangered language communities." So that's where some of the contestation comes in. 

Niamh McIntyre: Yeah. And that was something that, it definitely came up in Appen's work, too. Agencies like DARPA fund these low resource language programs, and say that they're for humanitarian purposes. It's like, oh, there's an earthquake in a country, and we need to be able to pick up linguistic information that's coming out of that country in order to help with disaster response, or whatever. And that's what gets posted in these sort of promotional blog posts, and not, the case that we ended up zeroing in on was a subset of the contracts that were sold to a unit that upgrades a type of military spy plane called a Rivet Joint, which is a really foundational part of aerial warfare.

Emily M. Bender: Yeah.

Alex Hanna: Yeah. Maybe we should get more into the article. So, "Getting it done, AI responsibility by design. There have been many public examples where generative AI models were hallucinating and inventing facts, which might be okay for creative endeavors like fiction writing-" sure. I guess these people have never really written fiction- "but not okay if someone is looking for facts or local up-to-date information like a search engine. Because these large language models are trained using currently available written data, mostly from the internet, it's very hard to filter out the source data that eventually leads to incorrect outcomes or bias in areas like gender or political preference. This is due to humans, no matter how hard we try, being inherently biased themselves, or due to a skew of who is creating the content based on a number of factors such as access to the internet. Individuals and companies are currently having to go through a process to understand the potential risk posed by currently flawed generative AI." And weirdly, there's a capitalization of generative AI here, as it wasn't before. I don't know. "There have been many responses, from it being embraced-" this is so poorly written- "from it being embraced as a work tool of the future to outright rejection and bans. Regardless, the implementation could have serious impacts to society good." I guess that means like common good? "Where done responsibly, but potentially negative where hallucinations and mistakes are permitted to permeate AI that is used to impact the lives of everyday people without appropriate safeguards." God, that was so poorly written. 

Emily M. Bender: So I think that last thing was "serious impacts- the impacts can be good ones or they can be negative ones," I think is what they were trying to do in that sentence, but this whole thing makes no sense at all. And the "currently flawed," capital G generative AI sort of suggests this is gonna get better, right? The teething pains, this is still growing. We're in that space. But also, they seem to think that you could fix this with having better data. I think that's where they're going, which is just like, synthetic text is synthetic text. And it doesn't matter what it's trained on, it's still gonna be paper mache of its training data. And I guess it matters to the extent that if you don't train it on Somali, you're not gonna be able to generate Somali. But it's not like you can end up with something that is going to be factual and reliable just by putting in better training data. 'Cause that's not how this technology works.

Niamh McIntyre: Yeah, no, this was really hard to get through. I do love just putting "good" as a clause by itself. That's maybe my favorite bit of this blog post. I think that in the heading for this, the "responsibility by design," I guess it speaks to the phenomenon of like, responsible AI, or AI safety, just not really having much to say about the ways in which AI is used by the military, or for surveillance. It's much more about alignment and is the model doing what we expect and want it to do? Or, could it be exploited by bad actors? And it's not really, yeah, it is a definition of safety and responsibility that ignores those cases where it's being used for, to gather intelligence ahead of an airstrike or something like that. 

Emily M. Bender: Yeah, absolutely. The AI for good and NLP for good framing always drives me up the wall, because it's basically people saying, we're gonna look at all the positive things we can do with this tech. And refusing to grapple with both the harms in its use and the harms in its production.

Alex Hanna: Yeah, absolutely. So they talk here about "designing AI for good." "Answering the call for responsible AI requires an approach called responsibility by design, similar to the privacy by design concept championed by privacy experts."

Emily M. Bender: I wanna jump in there, if you don't mind. 

Alex Hanna: Sure. No, go for it. 

Emily M. Bender: Privacy by design means you've designed a system where privacy is an inherent property, right? Or the protection of privacy is an inherent property. And I think it makes sense to talk about privacy in that way. But responsibility by design, the responsibility doesn't live in the system, right? It lives with the people who build and use the system, and that can't be therefore designed into the system. Like, this doesn't make sense. 

Alex Hanna: Yeah. And responsibility is such a wishy washy word. Responsibility doesn't really make any, there's no reference to this. It's like, we will just be- yeah, I'm responsible. Trust us on this. And they say that they're doing this, but that actually means absolutely nothing. So where they say, "This requires embedding responsible AI practices into the design specifications of technologies, business practices, and physical infrastructures from the beginning. Doing this upfront is far better than trying to do so retrospectively." And I'm like, okay. And then they say, "It may be that responsibility by design is legislated, and we've already seen the first attempts out of the EU-" I'm assuming that's referring to the EU AI Act- "but it may simply be that it makes good sense." And I like this comment, these pair of comments from magidin who says, quote, "Our objective is to design polluting industries that create nuclear weapons for good," capital G. And then the second comment, which is, "But are they pinky swearing it will be used for good?" Which seems to be what the design signifier is inserted to do here. 

Emily M. Bender: I think the second half of this one is worth commenting on, too. And then the next one. So, "What we believe we will see in the future is less focus on the algorithms and training models themselves to prove that the AI is responsible, but more focus on the underlying data sets, human feedback, and increasing scrutiny and guardrails that are provided for the outputs. Incorporating feedback from real people with real world experience across a diverse set of backgrounds is the best way to have models trained to act more like humans. If the feedback is diverse and expansive, the models have fewer hallucinations and bias." Simply not true in the end there. But also there was a very strange turn from responsibility by design being inclusive, to therefore we're trying to make this act more like humans. And I'm wondering, Niamh, how this lands, if you're thinking about those Somalian refugees in Kenya who are doing this annotation work for Appen. How is their real world experience improving the responsibility of the models, would you say? 

Niamh McIntyre: Yeah, it's a really good question. I suppose Appen would say, if a model understands- or not understands, but you know, can interpret Somali, it's less likely to make a mistake. But it really just shows the absurdity of the construction, really. What's happening is the Somali refugees are getting paid small amounts of money, to work on data sets that will enrich Appen, and serve a US military objective. But it is hard, because there's obviously just such a degree of opacity around military technology, that it's hard to know, was the work they were doing in any way gonna meaningfully improve the accuracy of the system? I am probably not gonna shock you if I say that it's quite hard to get people to tell you what kind of software they have on military spy planes. So we have a broad sense of what the Appen contracts related to. We know that one of them related to this particular kind of plane. But yeah, in terms of the actual functioning of the systems, we have hypotheses, but couldn't pinpoint. But yeah, I'm sure it was all very responsible, whatever it was, in summary. 

Alex Hanna: Right, everything they're doing is so responsible, by design.

Emily M. Bender: Do you wanna take this next bit, Alex? 

Alex Hanna: Yeah. These two graphs, and then there's definitely this, "Appen's role in AI for good," which we gotta get to. But this first one before it is, "AI needs to work for all," comma, "equally." So, "AI needs to help people, and to the best extent, all people. When built responsibly, AI is more successful and works in a way that benefits everyone regardless of race, gender, geography, or background. Large language models are generally built on English language data, which accounts for the majority of online content, yet less than 20% of the world speaks English as a first or second language. Limiting language inputs not only leaves a tremendous language gap of users underrepresented in the models of today, but also a cultural gap. It is widely known that when you are not represented in the underlying trading data, it is more likely the AI won't work for you. Creating ethical AI that responds to, respects, and reaps benefits for everyone means not only-" hold on. "For everyone means those involved in initially training it and later refining it needs to not only reflect the diversity of people it ultimately serves-" holy run on sentence- "but those creating it also understand the critical role they play in impacting those around them."

Emily M. Bender: Okay. So just object lesson in why you should always read something out loud.

Alex Hanna: Yeah. My questions are like, okay, what does it mean to say there's a cultural gap of what a model is doing, as if it is encoding culture?

Emily M. Bender: There's a lot of people who look into that in NLP, and it is very easy to show that there are US culture and generally more Western culture based biases in the systems. But that also presupposes that you're using them for something in particular. 

Alex Hanna: Yeah. It really depends on what this use is. And if we're talking again about military targeting, I guess in that case, that is very, that is US culture. Who says the US doesn't have culture? 

Emily M. Bender: On that point, ElizabethwithaZ is in the chat. "But will they be humanity centered?" And magidin replies, "Absolutely! They will have humanity centered right in the middle of the cross hairs." 

Alex Hanna: Yeah. And abstract_tesseract says, "Slaps roof of person. This bad boy can fit so much geography in it!" 

Emily M. Bender: This is, yeah, because people have geography? 

Alex Hanna: I suppose people have geography. Is that what they mean when they say "I contain multitudes"?

Emily M. Bender: So the last bit of this also really got me. Because they say, at the end of that long run on sentence, "This requires those creating it also understand the critical role they play in impacting those around them." And that led me to wonder, who does Appen think is actually creating this software? And do they count the data workers in that group? 'Cause Niamh, you told us that they don't tell the data workers what they're working on. 

Niamh McIntyre: Yeah. I think what's very apparent in this passage is like, they're talking about it as if it's like a cooperative endeavor and everybody has an equal role in it. But yeah, these workers really have so little stake in the end purpose of the technology. Like they don't even know who they're working for in the way that I know who my employer is, I know broadly what the objectives of my work are. So it doesn't really seem accurate to posit in this way, that we're all having, taking an equal responsibility in creating the technology. And I think in this bit, again, it's just the sort of slippage between, we want Somali speakers to be able to access information in their own language for altruistic purposes, versus some of the end use of the training that they were doing. 

Alex Hanna: Yeah, it's really giving, like, we want this technology to work for everyone, and by everyone we mean our variety of clients. And it really gives the game away. And when you're suggesting, this is a model that we want to understand your language and your culture, but in this case it is for a very particular type of deployment. It makes me think about the different projects that we often see discussed when it comes to training data sets, and diversification of those data sets. When I was at Google, there was an app called, I think it was called like Crowdsourcing or something, but it was effectively, you could download it and you could take pictures of things and label them yourself. And then, but that's just to make Google's product better. Like, why would I want to invest in this? Yes, I do want Google's proprietary model to work better for me, and so I'm going to do this work for them. I'm doing my part. And it's just such corporate double speak in this whole piece. 

Niamh McIntyre: Yeah. But was that for, as in that was like, anyone could download that app? 

Alex Hanna: Yeah.

Niamh McIntyre: Yeah, okay. Because it reminded me, there's a great story about a company called Premise Data, which is very relevant to the Appen story, in that it was exactly that kind of product. Like you would, they'd get people to download it and go out and take pictures, and often for market research, go to your local supermarket and take a picture of what products are on the shelves. But then, if I'm recalling the details correctly, then the US military were getting those people that were signed up to the app to go to, I can't remember which country it was, but somewhere where they had military objectives, and go take pictures of like, random roads where they didn't have good satellite coverage or something.

Alex Hanna: Wow. That's wild. 

Niamh McIntyre: Yeah. There is often that kind of crazy dual use stuff where it's, yeah, some people are taking pictures of what kinds of cereal are on the shelves of a supermarket, and other people are gathering information for the US military. 

Emily M. Bender: Yeah. So this last bit remains a rich text in many ways, so I think we should go through it. Their final section- final? Yes- is "Appen's role in AI for good. Appen's role is significant in responsibility by design, as our diverse 1 million strong team of AI training specialists-" which is I guess what they call their gig workers- "spans 170 plus countries and speaks 235 plus languages and dialects. This underpins an ethical and diverse AI supply chain that helps ensure AI works and is relevant across societies and cultures." So they keep going. "We are in the data business. Crucial components in the AI lifecycle are data sourcing and data preparation, collecting, annotating, and evaluating data. We create data sets for companies all over the world, and we couldn't do it without the support of our global Crowd."

Alex Hanna: And crowd is capital C. 

Emily M. Bender: Yes. And just the fact that, that "our," that possessive there is really icky to me. There's no indication here that Appen really has a sense of a responsibility, of care, to these extremely vulnerable people that they're employing. But they brag about them as AI training specialists. Impressive that already in 2023, they're calling it that. I have to say they were maybe a little bit ahead of the curve there.

Alex Hanna: They were ahead of the crowd. They were innovating. 

Emily M. Bender: Yeah. And later on they talk about a "Crowd code of ethics." 

Niamh McIntyre: I also love, were you gonna say, did you try and click and you got a 404?

Alex Hanna: Yeah, it's a 404. 

Emily M. Bender: Oh, no! 

Alex Hanna: It's not found anymore.

Emily M. Bender: No, but I was really wondering, is this ethics for the people participating as gig workers or is this ethics for how they will treat those people?

Alex Hanna: Yeah. I'm also trying to, I'm trying to do an archive.org backwards lookup. Although our producer Ozzy did find some subsidiary called CrowdGen, that is a subsidiary of Appen where they do have a "Crowd code of ethics." And they've got things here like "fair pay, communication, wellbeing." Which they have also, those are the things that they have linked on a blog post, which is the, "without the support of our global," capital C, "Crowd." And they've got this "Fair pay" program.

Niamh McIntyre: Which to be fair, they do say, they have said publicly previously that they try and benchmark contractor pay to minimum wage in that country, like a minimum hourly rate. Weirdly, they didn't engage with us at all for the story, so they didn't tell me that directly. So I don't have a, they didn't respond in detail really to any of our points, but I know they have had said that previously. 

Emily M. Bender: So what they're saying is they make sure to do the legal bare minimum. 

Niamh McIntyre: Yes. 

Alex Hanna: And they make a claim in this other piece with the, "without the support of our crowd," in which they say- first they have a stock photo of diverse people. Great. Good sign already. And then they have a survey in which they say that, "According to our November 2021 survey, 16% of our contributors were living under the global poverty line. As of today, 77% of those contributors are living above that global poverty line since the beginning of that work with Appen." I mean it's, whether there's actually any proof, we need to see the pudding. 

Emily M. Bender: It's also interesting to me that they frame all of this as "code of ethics" rather than commitments to fair treatment of their workers. It's just a surprising framing. 'Cause usually a code of ethics is, you associate that with a professional society, or like, there's a code of ethics for journalism that you follow if you are being a journalist. And here's Appen saying, here's the rules we're making for ourself. 

Alex Hanna: And I'm also curious, too, on what this has, as the market for data workers has changed since this was published in February 2023, and we're seeing that market change so dramatically. And we've seen a lot of articles on the massive demand for the data and what kind of market effects that's had, too.

Emily M. Bender: I was really surprised about this here, where they went from 16% in November 2021 to 75% today. And I was like, wow, they kept workers for two years! And the answer is no, this one was actually published in June '22. 

Alex Hanna: Yeah. And I think that's also a good point. I'm like, I'd be also curious on how much drop off they have there, in your panel data. 

Emily M. Bender: So let's do this last paragraph, maybe. 

Alex Hanna: Yeah. So they say, "We have greater hope now, and do see significant efforts to reflect on the impact that new technologies are having on society. As tech pioneer Bill Joy said, 'We have to encourage the future we want, rather than try to prevent the future we fear,' end quote. As we explore the possibilities of what generative AI can do for humanity, we are committed to being an ethical, responsible part of the AI supply chain, powering AI for good." Wow. How lovely. What a lovely sentiment that they have. 

Emily M. Bender: Yeah, so them saying, "We are doing this for good," Niamh, makes me wonder, have they reacted to your piece now that it's out in the world?

Niamh McIntyre: No, it's actually, I was really surprised by their lack of response. I think since I've worked at TBIJ, for the last three years, whenever we've written about a publicly listed tech company of any sort of medium size, they'll always engage with you to some degree. But no, they didn't answer any of our questions, and they haven't publicly reacted since. I don't really know what to read into that. I guess they hope they'll be able to ignore it, and just carry on as usual. 

Emily M. Bender: Yeah. And keep the mantle of AI for good. 

Niamh McIntyre: Yeah. 

Alex Hanna: And I'd imagine many of the large data work firms are rather mum on these types of things. And we've seen this kind of callousness from Scale intensely, which is, and I guess Scale has been bought by Facebook or Meta. And many of these companies, think they can act with impunity, or point to something like an ethics code as a foil.

Niamh McIntyre: Yeah. It's just unusual, like normally you'll get a sort of one sentence summarization of this blog post, at least, as a response. But we didn't even get that. They're also quite interesting in that their fortunes have declined as these newer companies like Scale have entered and been more successful at getting military and government contracts. There's been a lot of change of leadership, and it might be like, this is new leadership and that was a different era type of thing. But yeah, certainly like, Appen was quite an early mover in the online gig work space, and Scale has largely supplanted it, I think.

Emily M. Bender: Yeah, there was something that I saw as I was getting together the Fresh AI Hell, about, so it wasn't gig work, it was this set of teenagers who put together a company so you could do focus groups with fake people, and they got tremendous amounts of funding. So I think a lot of the venture capital in the space is excited about- one of the things, like there's two kinds of companies that are making money, right? The ones doing the data contracting and the ones selling the chips. And so I think there's a lot of energy there, and it might be that Appen was a little bit too long in the tooth to really excite people at the point that wave came through. 

Niamh McIntyre: Yeah. 

Alex Hanna: All right.

Emily M. Bender: All right. Should we get onto, I have a transition prompt for you, Alex. 

Alex Hanna: Oh, okay. Hit me with it.

Emily M. Bender: So you are giving a TED talk like presentation on behalf of something like Appen, about how you're doing AI for good. Except someone, let's say it was actual you, came through and swapped out the slide deck for lots and lots of evidence of all the harm that they're doing. And this can be non-musical or musical as you like.

Alex Hanna: Sure. Let me think of the most droll TED talk introduction. Like... people. Community. Wellness. Fair pay. These are all principles that we could all live by. We are living through a time of incredible technological change. That's why what I think about, is how we can improve that. Click. And it's got, it's got like, Karen Hao's reporting on how AI follows crisis. And I'm just like, that's why at Appen we're really dedicated to- and then seeing people in the crowd murmuring, and I'm like, click. And then the next one is like, Data Workers' Inquiry, like, Fasica's story of content moderation. And then, and I just click through. Anyways, I'm gonna write an actual TED talk that's on this now, and then just make it. I think there's definitely, it would be such a great, like a Yes Man style thing if somebody said they were one of these CEOs and set it up, and then just did something like that. Anyway, free idea for anybody in the audience. Make a fake company. Do something like this. 

Emily M. Bender: That'd be awesome. I have to say, Alex, I am both impressed and distressed at how well you can mimic the TED talk presentation genre. We've got ElizabethwithaZ in the chat saying, "How can something be so hilarious and so painful at the same time?" And magidin says, "Mission statement of the podcast." 

Alex Hanna: Yes. Again, used to work at Google. It's terrible. So this is a tweet from Marc Andreessen, our favorite venture capitalist. And the original post is, he is quote tweeting himself, and the original post says, "TLDR, there is no inner self. You're chasing an imaginary concept. The end." And then the quoted tweet, he said, "You are a 15 second sliding context window with the working memory of a goldfish. Your long-term memory is mainly fake, and it's a minor miracle you can get out the door in the morning." So yes, very much one of those cases in which the tech bros are dehumanizing themselves to make a larger claim about all of humanity, and how they move through the world.

Emily M. Bender: Yeah, and sjaylett says, "Offensive to goldfish." All right, so next, this is a really cool call from Rest of World, and the headline is "Contribute to Rest of World's labor by tech reporting." So this is a third year of a program underwritten by the Ford Foundation, and they are offering a fee of a thousand US dollars per story, for people who can write stories about "how new technology is transforming existing industries in unique ways, and creating unexpected new pockets of employment. How technology is leading to significant job gain, job loss," and so on. There's a bunch of topics here. They say, "Unfortunately we are unable to commission reporting in India or from writers based in India for this series." I don't know why. But I just thought it would be good to signal boost this, 'cause Rest of World does good work.

Alex Hanna: So this is, I'm gonna see the journalist who alerted me to this. So this is a website called detrans.ai. It was sent to me by Madisyn Parisi, who's a journalist at the Backbone and I think Rewire News. And it is a large language model, also apparently a RAG system, where it is an interaction, like it's supposed to be a chatbot, basically, to talk about detransition. And so it, I think, fine tuned on the r/detrans Reddit. And it says on the page, "Detrans.ai, talk to 50,000 plus detransitioners. Perspectives from the other side," as if detransitioners are beyond the veil of mortality or something. So it's, yeah, and there's, in kind of grayed out text, it says, "Please don't share any personal information that could be used to identify you. Chats are public. 50,000 plus refers to the members of the r/detrans subreddit." Whew. Just a lot to process here. 

Emily M. Bender: I have a hypothesis, Alex, that most of their chats are actually transphobes playing with this toy. That most of the data they're collecting is that. 

Alex Hanna: It could be. It's also interesting, like, I poked around a little bit at the detrans subreddit, too. And it is, as any kind of subreddit, it's curious as a place where there are people who are exploring things, and I don't know the dynamics of that very well. But it is at least doing the work, not to defend r/detrans, but it is actually situating stuff in discussion, ostensibly. But I don't know how much of that is transphobes, or your JK Rowlings, or whomever. 

Niamh McIntyre: Do you think that whoever's created this has just scraped that whole subreddit without their knowledge, or is it a kind of project linked to the subreddit itself? 

Alex Hanna: I think it's just one person. It's like someone that's scraped that subreddit. 

Niamh McIntyre: Yeah. 

Alex Hanna: From what I understand. And it's, the GitHub is available by this person named Peter James Steven, and his name is on the, their name is on the subreddit. I don't know their pronouns. But it's, yeah, so I'm assuming this is without the consent of all these different people who are posting there. 

Emily M. Bender: So we're gonna collect data non consensually, and then mash it up and make it, and advertise this as you get to talk to these 50,000 plus people. Which is just not true. Okay, that's horrific. Let's keep moving. So this is a C-SPAN clip of Melania, like, catwalk runwaying into a room in the White House with this humanoid robot. And it's at some kind of an international event. And later on, so it's not in this clip, but the tweet here from Brian Allen says, "Melania Trump this morning proposes an AI system, Plato, as a replacement for teachers. 'Literature, art, science, mathematics, history, the entire corpus of knowledge at home.'" And this robot here at this point is saying welcome to the assembled guests. And the thing that was hilarious to me as a linguist is that, for the languages that I speak, at least, it was very clearly an American English accent in those languages. And it's like, echoes of C-3PO, who sounded like he was a Brit no matter what language he was speaking. But also, there was no need. It could well have been that they created this multilingual thing by using an English text to speech and just putting in the words from all the different languages. Or maybe they just had really bad text to speech. Anyway, I was just really knocked off my chair by how bad that was. And then of course, appalled at Melania's suggestion here. 

Alex Hanna: It's like the trope in US shows where to signify foreignness, you have the people speak with a British accent. They're French, don't you know? They have British accents! 

Niamh McIntyre: Maybe they need Appen to do some kind of localization.

Alex Hanna: Oh, lord. Yeah, apparently. Geez. So this is from Yahoo Finance, and the title is "Nadella-" so this is Satya Nadella, CEO of Microsoft- "paid 650 million to recruit his AI chief." And I thought this originally said AI chef, which I thought was funny. But after two- 

Emily M. Bender: Barky barky bark! Sorry.

Alex Hanna: How dare you do this, Swedish chef? "After two years, he's quietly pushing him aside. These brutal numbers are why." So the first paragraph, "Microsoft CEO Satya Nadella announced the sweeping reorganization of the company's AI leadership on March 17th, unifying its consumer and enterprise co-pilot teams under a single executive and quietly sidelining Mustafa Suleyman, the former DeepMind co-founder he paid 650 million to bring aboard just two years ago. Here's the stunning data showing why." I'm curious on what these data are. Where are the numbers? I was promised numbers! Here, adoption problems. So "Microsoft 360 has more than 450 million paid commercial seats. After roughly two years on the market, it has converted approximately 15 million of them into paying users." Holy shit. Oh man, I didn't read this beforehand. Schadenfreude has just gone intense. "That's a 3.3% conversion rate, at $30 per user per month, generating only roughly $5.4 billion in annual revenue. That is less than what Microsoft spent on infrastructure in a single quarter." 

Emily M. Bender: Yeah, oof. All right. And abstract_tesseract in the chat says, "Notably not sidelining him because of history of abusive behavior or anything like that." 

Alex Hanna: Yep. 

Emily M. Bender: All right. I think you wanted this one, Alex. 

Alex Hanna: Yes. We'll try to make this brief. So this is, there's an original post by Andrej Karpathy, formerly of, where was he, OpenAI, all these different companies. Also was in Fei-Fei Li's lab as a grad student. So he says, as a bullet list, "Drafted a blog post. Used an LLM to meticulously improve the argument over four hours. Wow. Feeling great. It's so convincing. Fun idea. Let's ask it to argue the opposite. LLM demolishes the entire argument and convinces me that the opposite in is in fact true. LOL!" And then, "The LLMs may elicit an opinion when asked, but are incredibly, extremely competent in arguing almost any direction. This is actually super useful as a tool for forming your own opinions, but just make sure to ask different directions and be careful with the, sycopanthic- sycophancy." Sorry, I was missing the H. And then the poster that quotes it is like, this is such an important thing to consider! And I'm like, this seems to say more about Karpathy and his lack of discernment and argumentation and anything about the LLM. 

Emily M. Bender: Nothing in that story is him forming his own opinion, either.

Alex Hanna: Yeah. It's like, oh, it demolished it! And it's just, this is why we need humanities folks. 

Emily M. Bender: Yeah. All right, two quick chasers. One, Washington State Nurses Association. "Sorry, AI, you can't call yourself a nurse. The Washington State legislature passes a bill to restrict nursing titles to licensed human professionals." One of those things that we shouldn't have to do, but I'm glad we got it done. And thanks to the WSNA for their part in making that happen. And then Alex, you wanna do the honors on this one real fast? 

Alex Hanna: Sure. This is from a local Fox News affiliate, Fox News 10. 

Emily M. Bender: It's in Oklahoma. 

Alex Hanna: In Oklahoma. Thank you. So it says, "Fire department turns down $250,000 Google donation amid data center fight." And it's got a picture of a guy in a cowboy hat that's a, that's a video preview, and he is pointing at someone at a community meeting. The journalist is Burt Mummolo, KTUL via CNN Newsource. "Sand Springs, Oklahoma. An Oklahoma fire department turned down a significant donation from Google. At the end of State Highway 97 is where Charley Pearson could be found, taking care of his cattle in the peace and quiet of the Rock community," et cetera. But yeah, basically yay for many people fighting against data centers, especially in places that we have not found the same resistance, or purported resistance, to AI. This data center fight is really, really cross partisan, cross national, cross geographies. 

Emily M. Bender: Yeah. I love it. And I also love that these people weren't even having it, that donation was not gonna buy them off, which is amazing. 

Alex Hanna: Yeah. All right, that's it for this week. Niamh McIntyre is a senior reporter at the Bureau of Investigative Journalism. You can find her latest report on TBIJ's website at thebureauinvestigates.com. Thanks again so much for joining us. 

Niamh McIntyre: Yeah, thank you for having me. 

Emily M. Bender: It's been wonderful, Niamh.

Alex Hanna: Our theme song is by Toby Menon. Graphic design by Naomi Pleasure-Park. Production by Ozzy Llinas Goodman. And thanks as always to the Distributed AI Research Institute. If you like this show, you can support us in so many ways. Order The AI Con at thecon.ai or wherever you get your books, or- 

Emily M. Bender: But wait! 

Alex Hanna: Or request it at your local library. You jumped the gun there, Emily! 

Emily M. Bender: But wait! There's more. Rate and review us on your podcast app, subscribe to the Mystery AI Hype Theater 3000 newsletter on Buttondown for more anti hype analysis, or donate to DAIR at dair-institute.org. You can find our merch store there, too. That's dair-institute.org. You can find video versions of our podcast episodes on Peertube, and you can watch and comment on the show while it's happening live on our Twitch stream. That's twitch.tv/dair_institute. Again, that's dair_institute. I'm Emily M. Bender.

Alex Hanna: And I'm Alex Hanna. Stay out of AI hell, y'all. 

Emily M. Bender: But wait, but wait! 

Alex Hanna: But wait!

Head Shot

Alex Hanna

Co-host
Head Shot

Emily M. Bender

Co-host