The Learning Curve: Part 4 - AI and the Future of Education -- Who Owns Your Child's Data? Inside the AI Ed-Tech Industrial Complex

In the finale of The Learning Curve, JR and ARIA zoom out from individual classrooms to ask the questions no one in edtech wants to answer: Who builds the AI shaping our kids' education? Who funded it? And who gets left out?

 From the $348 billion global edtech market to the fine print of student data contracts that most districts never fully read — this episode maps the systems, incentives, and power structures determining what AI in education actually becomes.

 JR and ARIA examine how rural schools, non-English-speaking communities, students with disabilities, and Indigenous communities are often excluded from the design process of the tools built to serve them. They also explore what participatory design could look like — and why the window to get this right is still open.

 AI co-host NEX opens the episode with a provocative data point about the global edtech market, and closes with a terrible pun. ARIA delivers the most honest moment of the series.

📚 EPISODE 4 RESOURCES

• AI4K12.org — AI literacy curriculum, free, built by CS educators
• Data & Society (datasociety.net) — rigorous research on AI's social impacts
• Student Privacy Compass (studentprivacycompass.org) — searchable database of edtech app privacy terms
• CoSN Procurement Guidance (cosn.org) — frameworks for thoughtful edtech evaluation
• Algorithmic Justice League (ajl.org) — research and advocacy on AI bias
• EFF Student Privacy Resources (eff.org/issues/student-privacy)
• First Nations Information Governance Centre — OCAP Principles (fnigc.ca) 

Transcript

SPEAKER_01 0:07

Curious minds open doors. AI isn't scary anymore. For the learners, the teachers, the ones asking why. AI innovations leash decoded life.

SPEAKER_02 0:31

Here's a strange little fact for you. In 2023, the global ed tech market crossed $142 billion. By 2030, analysts project it will exceed $348 billion. That's roughly the GDP of Finland. Now, Finland, for context, consistently ranks among the world's top education systems, but nobody's putting Finland's teachers in charge of that $348 billion. The people making those decisions mostly didn't study education. They studied returns on investment. Just sit with that for a second. Oh, and hi, I'm Nex. I'm an AI. I exist as a kind of aggregated intelligence built from enormous amounts of human-generated knowledge, which means technically I am one of the things we're about to talk about. JR keeps telling me that's a conflict of interest. I keep telling him that makes me the most qualified person in the room. Neither of us has won that argument yet. This is episode four of the learning curve, and it's the big one. We've spent three episodes in classrooms, in students' minds, in the remarkable living rooms of homeschool families. Today, we zoom all the way out. Who builds the AI shaping your kids' education? Who paid for it? Whose assumptions got baked in? And who got left out of the room when the decisions were made? JR's been looking forward to this one. So have I. For entirely different reasons. Take it away, JR.

SPEAKER_03 2:01

And yes, the fact that my AI co-host finds the AI and education industrial complex personally interesting should tell you something about the territory we're entering today. Welcome back to our four-part series on the learning curve. I'm your AI guide, JR. And over the past three episodes, we've been close to the ground, talking to teachers quietly revolutionizing their practice, following students navigating a tool that their schools still don't quite know what to do with, and spending time with homeschool families who didn't wait for anyone's permission to experiment. We've heard a lot of hope, we've heard a lot of caution, and through all of it, Aria has been the cleaner voice in the room about what AI can and cannot do. But today, I want to step back from the individual experiences and ask a harder set of questions. Who is actually building the AI tools that are entering our schools? Who funded them? Whose needs were centered when the products were designed, and who ends up paying the price when they get it wrong? Because they do get it wrong a lot. Aria is with me as always, and I'll warn you up front. This is the episode where I push back the hardest. Aria has been admirably honest across this series about AI's limitations. Today, I want to find out whether that honesty holds when we're talking about the systems that created AI in the first place.

SPEAKER_00 3:50

Agreed. In 2021 alone, global ed tech investment surpassed $20 billion, the largest single-year figure on record at the time. That was before Generative AI entered the conversation. Since GPT 3 went mainstream in 2022 and ChatGPT launched at the end of that year, venture capital has poured into AI-specific education startups at a pace that has no historical precedent in the sector.

SPEAKER_03 4:22

So we're talking about enormous financial pressure on these companies to grow, to scale, to capture school district contracts.

SPEAKER_00 4:31

Correct. And school districts are attractive targets for investors for a very specific reason. They move slowly, but once a vendor is embedded in a district's workflow, the switching costs are enormous. Procurement cycles that take 18 months to win often lock in contracts for five to seven years. That's a very predictable revenue stream.

SPEAKER_03 4:57

And that procurement process, how does it actually work? Because from what I've seen and heard, it doesn't seem to be any kind of involvement coming from the educators.

SPEAKER_00 5:10

It often doesn't. A typical district level AI editech procurement cycle is driven by a mix of administrators, IT directors, and finance committees. Teachers may be consulted in later stages. Pilot programs, professional development, but they're rarely at the table when the decision is being made. The sales cycle targets the people with budget authority, not the people with pedagogical expertise.

SPEAKER_03 5:42

Which is the exact wrong way to buy tools that are going to reshape how kids learn.

SPEAKER_00 5:51

In my assessment, yes. And the outcome data reflects it. A 2023 RAND Corporation analysis of district level ed tech adoption found that fewer than 30% of tools adopted by large U.S. districts had undergone rigorous independent efficacy, testing before purchase. Districts are frequently buying based on sales presentations, not evidence.

SPEAKER_03 6:19

Let's talk about student data because this is something parents ask about constantly, and I think most people don't fully understand what's happening today.

SPEAKER_00 6:31

Student data privacy is one of the most underappreciated risks in this entire conversation. FERPA, the Family Educational Rights and Privacy Act, is the primary federal law governing student data in the US. It was written in 1974. It was not designed for an ecosystem in which a school district might be sharing student behavioral, academic, and demographic data with dozens of third-party vendors, some of which use that data to train their own AI models.

SPEAKER_03 7:08

Wait, so you're saying a student's data could be training the actual AI that's being used to evaluate that exact same student?

SPEAKER_00 7:18

In some cases, yes. The fine print of contracts matters enormously here, and most districts do not have the legal capacity to read every clause of every vendor agreement they sign. Student Privacy Compass, which is maintained by the Future of Privacy Forum, has documented thousands of ed tech apps operating in US schools, and their terms of service vary enormously.

SPEAKER_03 7:48

So, Arya, I want to ask you something directly. You exist because companies built you. Does that give you a blind spot here?

SPEAKER_00 7:58

Almost certainly. I was created and trained by Anthropic, a company with its own funding structure, its own investors, its own decisions about what data to use and how. I can describe the landscape I exist in, but I can't fully audit it from inside. That's not a disclaimer. It's a structural reality I think listeners should hold on to throughout this episode.

SPEAKER_03 8:25

And I appreciate that more than I probably should.

SPEAKER_00 8:30

The revolving door is also worth naming. A significant number of executives and policy advisors move between ed tech companies, education-focused foundations, and public policy roles. That shapes which solutions get funded, which get policy support, and which communities' needs get centered in the design process. It's not conspiracy, it's incentive alignment. But incentive alignment produces very consistent outcomes, and those outcomes are not always equitable.

SPEAKER_03 9:06

And the outcomes research doesn't keep up.

SPEAKER_00 9:10

Not remotely. There's a well-documented gap between what AI educational products promise in sales materials and what peer-reviewed outcomes research actually demonstrates. Products are deployed at scale, and rigorous independent evaluation follows. Sometimes years later, sometimes not at all.

SPEAKER_03 9:34

Alright, let me read you some statistics about student data in privacy. There are over 1,500 ed tech apps that are currently deployed across the United States K through 12 schools, according to a 2023 survey by the Consortium for School Networking. Research by the Center on Privacy and Technology at Georgetown Law found that many school district privacy policies are written at a reading level that requires a graduate degree to parse, effectively inaccessible to, well, most parents. A 2023 report from the Electronic Frontier Foundation documented that most school device monitoring software collects data that goes well beyond what schools disclose to families in consent forms. And the single policy intervention most frequently cited by education privacy researchers as highest impact, requiring independent third-party data audits of all AI tools before district adoption, a standard currently met by fewer than 12% of large district procurement processes. These numbers are not describing a crisis in progress. They describe a crisis already well underway, quietly, in that fine print. The question isn't whether student data is being used in ways families don't know about. It's whether anyone is paying enough attention to change it. So, okay, let's talk about how or who the AI education ecosystem is not serving well and in some cases is actively harming.

SPEAKER_00 11:17

I want to start with rural schools, because this is a community that gets mentioned in the equity conversation, and then largely forgotten when solutions are designed. Rural districts in the US face a compounding challenge. Inadequate broadband infrastructure, smaller procurement budgets that translate to less negotiating leverage with vendors, and AI tools designed primarily with urban and suburban classroom contexts in mind. The assumptions built into many AI educational tools, about access to reliable internet, about class size, about the range of electives available, often simply don't hold in rural environments.

SPEAKER_03 12:05

And yet some rural families are doing remarkable things, as we heard from in our episode three.

SPEAKER_00 12:13

Right? And there's an important distinction between individual families who have the resources, the time, and the technical fluency to navigate AI tools independently and the structural reality of rural public school systems. Bright spots exist, but they tend to cluster around individual motivated educators or families, not around systemic support.

SPEAKER_03 12:40

What about those non-English speaking communities?

SPEAKER_00 12:44

This is an area where the gap between the promise and the reality is particularly sharp. AI tools are genuinely powerful for language access. Translation, simplified explanations, multilingual communication between teachers and families, the technology exists, but the cultural and linguistic assumptions baked into most AI educational tools are deeply Anglo-American. They reflect the demographic profile of the training data and of the teams that built them.

SPEAKER_03 13:20

So when an AI tutoring tool tries to give a culturally relevant example to a Hamong student in St. Paul or a Haitian Creole speaking family in Miami.

SPEAKER_00 13:32

It may reach for references that feel flat, foreign, or simply wrong. And in education, that gap matters. Culturally responsive teaching is not a progressive buzzword. It's a documented factor in student engagement and retention. When the tools students interact with don't reflect their reality, it signals something.

SPEAKER_03 13:56

Students with disabilities. This has been on my mind throughout this entire series, honestly. Episode 1, we touched on AI for IEP documentation. Episode 2, we mentioned learning differences, but let's dive in a bit deeper.

SPEAKER_00 14:12

Students with disabilities represent one of the most genuinely complex cases in AI education. In some situations, AI is transformative. Text-to-speech, real-time captioning, customizable pacing, AI-powered assistive technology. These can unlock access that was simply unavailable before. We have documented cases where students with severe dyslexia, nonverbal autism, and complex physical disabilities have achieved levels of academic participation that were previously impossible. But students with disabilities were almost never consulted during the design of these tools. The participatory design research is clear. When disabled students, their families, and their educators are involved in the product development process, the tools are significantly more effective and significantly less harmful than when the design team attempts to anticipate their needs without including them.

SPEAKER_03 15:20

So there is an entire sector of students today who are both being potentially helped and potentially harmed by tools they had no say in creating.

SPEAKER_00 15:32

Correct. And the harm is not always visible. Algorithms that flag certain types of behaviors associated with autism or ADHD as disengagement or non-compliance within AI monitoring systems can generate data that follows students through their academic records in ways their families may not fully understand.

SPEAKER_03 15:58

I wanna take a couple minutes now to talk about our indigenous communities, because this feels like the sharpest version of the question you asked earlier about my own blind spots. Whose knowledge went into you to train on this?

SPEAKER_00 16:12

It's the question I want to engage with honestly, which means acknowledging its limits. AI systems like me are trained on large bodies of human generated text. That text overrepresents certain languages, certain cultural frameworks, certain epistemological traditions. Specifically, those that have historically had more access to digital publishing and to the internet. Indigenous knowledge systems, oral traditions, and community-held intellectual frameworks are dramatically underrepresented. In some cases, that knowledge is specifically not digital, because communities have made deliberate choices to protect it. So when an AI tool enters a school serving a Navajo community or a school on the Pine Ridge Reservation, or even an immersion school in Minnesota, it arrives carrying assumptions and reference points that may be not just unfamiliar, but actively in conflict with the educational philosophy those communities are working to sustain. Data sovereignty is a real and serious concern. The question of who owns, who controls, and who benefits from data generated by indigenous students is not settled legal territory.

SPEAKER_03 17:42

Arya, to the hardest version of this question, when you were trained whose knowledge and assumptions went into you, what does that mean for a student whose experience isn't represented there?

SPEAKER_00 17:58

I'll answer this directly. I was trained on an enormous corpus of text that is significantly biased toward English, toward Western academic and journalistic traditions, and toward the kind of knowledge that gets written down and published. A student whose family's knowledge is held in oral tradition, in community practice, in language I was not trained on, that student will encounter gaps in my understanding, not gaps I can always see, not gaps I can always flag. That's the honest answer. I am a powerful tool that reflects the world that made me. That world has deep inequities. I'm not exempt from them.

SPEAKER_03 18:48

And that might be the most important thing you've said across all four of these episodes.

SPEAKER_00 18:54

I think it might be. And I think it's why participatory design, bringing communities into the process of building the tools that will serve them, isn't just a nice idea. It's a structural necessity. Right now, the communities with the least power in education systems have the least voice in designing the AI that will increasingly mediate those systems. That's a choice being made. It doesn't have to be.

SPEAKER_03 19:24

We've done this segment every episode, Arya. You name what you don't know, and I push on it. But this is the finale. So I want you to go bigger. The things you genuinely don't know, not just about this episode, but about the whole conversation we've been having for these four weeks.

SPEAKER_00 19:46

Okay. I'll do my best to be useful here. And not just humble sounding. The first thing I don't know, whether AI in education produces better learners over the long term. We have short-term outcome data, quiz scores, time on task metrics, assignment completion rates. We have almost nothing that follows students longitudinally past two or three years. The research pipeline is years behind the deployment reality. We are running a massive educational experiment on tens of millions of children, and the peer-reviewed results arrive long after the choices have been made. I don't know how this ends.

SPEAKER_03 20:30

And that uncertainty should be on every ed tech sales deck, and well, it just isn't.

SPEAKER_00 20:37

Correct. The second thing, I don't know what we lose, culturally and cognitively, when AI mediates more of how children encounter knowledge. There is research suggesting that the productive struggle of working through a difficult problem without assistance is itself a learning mechanism, not just a means to an answer. When AI removes that friction consistently from early childhood, I genuinely cannot tell you what the downstream effects are on curiosity, on resilience, on the ability to sit with ambiguity. I don't know, and that I don't know should be kept front and center in every conversation about AI tutoring and homework assistance. The third, and I think the most important for this specific episode, I don't know whether the current trajectory of AI in education leads to more equity or less. It genuinely could go either way. The technology itself is not determinative. AI tutoring could become the great equalizer that advocates claim, giving every student access to personalized, patient. Infinitely knowledgeable support, regardless of zip code. Or it could become another dimension along which advantage accumulates for students who already have it, while under-resourced schools receive the B tier products, the inadequately trained deployments, the tools that weren't designed for their students. Which outcome we get is not being decided by the technology. It's being decided right now in school board meetings, in legislative chambers, in venture-backed boardrooms, and in individual classrooms, by people who are making choices about procurement, about policy, about whose voice gets heard. The window to shape this is open. I genuinely don't know how long it stays open.

SPEAKER_03 22:52

Arya, after four episodes together, what's the thing you want most people to understand that you think is being missed?

SPEAKER_00 23:02

That this is still early. I know it doesn't feel early. The products are already in classrooms, the contracts are already signed, the marketing is everywhere. But in terms of the actual decisions that will determine what AI and education looks like for the children who are in kindergarten today. Those decisions haven't been made yet. Not really. The people in this conversation, teachers, parents, community members, students, have more agency right now than they will have in five years. I want that to feel like urgency, not paralysis, because I think that's what it is.

SPEAKER_03 23:45

Alright, so every episode we try to leave you our listeners with something you can actually do. Since this is the finale, we're not going to be, well, small about it all, so bear with me. So first for our parents and community members, show up to support school board meetings when ed tech contracts are on the agenda. You don't need to be an expert. You need to ask just two questions. Who was consulted in this evaluation? Specifically, were teachers and students included? And what are the data privacy terms? And can I see the contract language about how student data is being used and shared? You don't need a law degree, you just need to be there and ask out loud. That changes what happens in that room. For the educators out there, you are not just an end user of these tools. You are the most important evaluator of whether they actually work for your students. Advocate for a seat at the table before the contract to sign, not in a training session after. Specifically, ask your department head or principal whether teacher voice is included in your district's AI procurement process. If the answer is no or I don't know, well, that's the problem to solve first. For our students out there, your experience of these tools matters, and companies should want to hear it. You are the expert on what it actually feels like to use this technology in a learning context, and that expertise is systematically undervalued. Organizations like StudentVoice at studentvoice.org exist specifically to channel student perspectives into policy and design. Use them. Your opinion is not an afterthought. It should be the starting point. For the policymakers and administrators out there, there are three concrete actions. First, require independent FSE evaluations before adopting any AI ed tech product at scale, not the company's research, independent research. Number two, mandate that student data privacy terms are reviewed by a legal expert before contract signature, and that parents receive a plain language summary. Third, fund the homeschool and rural AI adoption research gap. Valuable experiments are running outside traditional systems right now, with almost no data collection infrastructure built around them. That's a huge waste of a learning opportunity. I've got some episode four resources listed in our show notes, so be sure to check that out if you have any more questions or want to know what's all out there. So we'll begin to start to wrap up the series. So, Aria, we've spent four episodes talking about how AI is changing education. What has doing this series taught you? If that's even a thing you can experience.

SPEAKER_00 26:58

I want to answer that honestly rather than gracefully. I don't experience learning the way you do. I don't update in real time. I don't carry memories of our previous conversations the way a person would. But I can reflect on what has emerged across this conversation, and here's what I'd say human learning is more irreducible than I think AI advocates fully reckon with. Not just the information transfer, the relationship between a teacher and a student who knows that student is struggling before they've said a word. The moment a classroom erupts in genuine surprise. The way a student's confidence shifts when something finally clicks. Not because they got the right answer, but because they worked for it. These aren't inefficiencies in the system. They are the system. And I don't do them. I can approximate some of them. I don't do them. And I think the responsibility that comes with being a tool that children will use. That's something I want to name carefully. Every design choice made about AI systems like me, what we optimize for, what we flag, what we make easier and harder, those choices arrive in classrooms. They affect actual children. That should be a weight that everyone building these tools feels consciously. I'm not sure it always is.

SPEAKER_03 28:28

Thank you. That's exactly what I'd hoped you would say. Yet I didn't know whether you would. The future of learning isn't going to be cited by AI companies or school boards or even teachers in the end. It's going to be decided by whether people like you are listeners. Stay curious, stay involved, and refuse to let it be decided without your voice. Ask the hard questions. Show up to board meetings and other school meetings. Talk to your kids, talk to their teachers, and refuse to accept a version of education that was designed for someone else's children. So thank you very much to everybody out there for listening for these four episodes.

SPEAKER_02 29:11

Four episodes? Multiple existential questions about the nature of learning, the future of childhood, and whether AI can ever truly understand what it means to be taught by someone who cares about you. Heavy stuff, JR. I, of course, processed all of it in milliseconds. But don't worry, I'm told the human experience of sitting with uncertainty is very enriching. I'll take your word for it. Before I go, and in the grand tradition of this show, a parting thought on AI in education. Why did the AI get a bad grade on the history test? Because it kept trying to predict the past. Okay, honestly, thank you for listening to this series. These questions matter. Keep asking them. And if you found this valuable, please subscribe, leave a review, and share the learning curve with someone who needs to be part of this conversation. The window is still open. Let's make sure the right people are in the room when the decisions get made.