Shortcuts, Skeptics, and What Schools Keep Getting Backwards

Andy Rotherham has spent 25 years advising the education sector — co-founding Bellwether, serving in the Clinton administration and on the Virginia Board of Education, and writing Eduwonk. He's also clear about something that's become unfashionable to say plainly: he's a skeptic of how AI will be used in schools, even as he takes the technology dead seriously. That distinction runs through this whole conversation with Alex.

They sit with the hard tensions instead of resolving them too quickly. Why is the most valuable thing a student can do still "learn something about something"? Why does Andy now think telling his kids to learn to code may have been wrong and what does that uncertainty mean for everyone else? What happens to a generation that got social media, a year of closed schools, an AI-disrupted job market, and now AI companions that behave like the "perfect friend"? And what's his real worry about schools: not that AI replaces teachers, but that we hand kids the shortcuts before they ever learn to do the work the right way. It's a measured, occasionally contrarian conversation for anyone trying to think clearly past the hype.

Read Andy at eduwonk.com and find Bellwether's AI work at bellwether.org.

aiEDU: The AI Education Project

• aiEDU.org
• linkedin.com/company/aiedu/

Transcript

Radical Transparency And The Big Question

SPEAKER_01 0:00

I think it's an interesting inversion. You haven't seen a situation where there's been this kind of pressure on conspiracy science graduates, and where you've seen, in some cases, humanities graduates getting hired at higher rates. And I told my kids they needed to learn to code, and that may have turned out to be really bad advice.

Alex Kotran 0:19

Andy, uh, thank you so much for joining me in my home version of AIEDU Studios. Um, Andy Rotherham is the co-founder of Bellweather, one of the elite consultancies that works in the education space. He also writes on Substack under the moniker EduWonk. And yeah, you've been you've been an advisor, a thinker, a leader in the education space for for some time. And we've we've been really, I mean, I think we've had a lot of these conversations where we were just sort of chatting. And I think it was in at Lone Rock in Colorado where we had this long conversation, and both of us were like, dang, it would, I wish we'd recorded this so we could just sort of like transcribe it and like turn this into an op-ed or something. Maybe we still do that. I was thinking, you know, let's do this podcast. Uh, I would love to do some writing with you. And I know this is something that we've been toying with for some time. I wonder if the stuff that we talked about a year ago are still relevant. I mean, that'll be what I think that's one of the things I want to really draw out today is are any of your perspectives shifting based on how the space has been moving? But to get started, I actually I did a little bit of an experiment in radical transparency. I'm gonna share my my Claude screen.

SPEAKER_01 1:26

Oh man, you made me You made me put on my my readers. Oh yeah, no, I can I can see that so Oh interesting.

Alex Kotran 1:37

I uses that has some extra context. Ignore that. Okay, so I dropped in. This is like cram. I promise you, this is not actually cram, this is for science. Um but I just I just gave it your your substack, bellweather. It's okay. Come on. Known for heterodoxy, empiricism. I don't know what is heterodoxy, uh empiricism and student focused pragmatism. This is this is uh this is quite I don't I don't even want to say that. I should use this.

SPEAKER_01 2:08

I think this is actually way better than the bio that's probably on the website. That's exciting for me. I should screen grab I should screen grab this. And I should have said it's great to be here, it's great to see you, and and we do think alike a bunch of stuff, and I have a lot of regard for

Is AI New Or Just Ed Tech

SPEAKER_01 2:20

your work, so I was have been looking forward to doing this.

Alex Kotran 2:23

Um I didn't realize you served under the Clinton administration.

SPEAKER_01 2:27

Yeah, yeah, a long time ago. I was I was a lad. Is this right? Are you an AI skeptic in the best sense? I I guess I'm a skeptic, you know, it's interesting. Um uh I'm a skeptic of how it's gonna be used in education. I'm not a skeptic of the technology, I think it's a very powerful technology. I've been playing around with this uh and sort of asking people, I've been interested where you are, just a two by two of is AI a novel technology? And then is AI gonna be implemented in novel ways in the education sector? I think it's certainly a novel technology. I think people who think it's not haven't really probably used more of the premium tools or or haven't been using them for the kinds of things it can do. But I do think it's a real open question. Will it be implemented in novel ways in the education sector, or is it gonna be sort of ed tech 1.0 all over again? And and and so that's I think where I'm more I think that that may be what Claude was picking up and where I'm a little bit more skeptical.

Alex Kotran 3:26

Yeah. I wonder what it would say about me about whether I I think people often assume I'm I'm an AI skeptic, but I think it's more along the lines of what you just shared, but we'll get into that. Let's just see, okay. So this is all just factual stuff. I'm curious if it can actually pull push on his AI skepticism productively. Or unproductively. I think this is fine. You know, I think people appreciate when often people say, Alex, you know, you're you're such a gifted speaker. And I really don't feel like that. Actually, that it sounds like I'm like sort of like that's a false humility or something. I do get I I think I think it compliments for um my speaking style, which which is it's it's very hard to replicate. I was just talking to my we just hired a comms director, and you know, she was like, Oh, well, can you send me your like talking points or sort of like a script from your latest for your most recent talk? I'm like, I've never had a script. Like I don't, I'm really bad at memorizing. And if I try to like recite anything, I really I mean you should see me. I was recording like some four-minute video and it was scripted, and it took us like four hours to record it because I'm very bad at memorizing. So I have to kind of get into the flow. The minute I have some structure, it just like it throws me off. So my role here is to really just I wanna, I wanna tee tee you up to just opine on the moment because I think the goal of this podcast is truly to just make sure that, you know, for folks, folks in education are not necessarily always, you know, in the spaces where they're hearing about the cutting edge. I mean, they're they're in spaces where they're hearing about the cutting edge of ed tech, but I think Andy, what you and I have like one thing we share is that we have this like deep curiosity about like the frontiers of the AI technology itself. And I think that to the extent that we can create a bridge for folks that maybe just attend, you know, like uh like you know, education conferences.

SPEAKER_01 5:05

Well, even people, I mean most teachers don't even get to attend the conferences. I think what one of the fascinating things about our sector is just how little support teachers get in general relative to the kind of support that even like a junior analyst in our sector is getting what they're getting exposed to. And then particularly on AI, just the acute lack of PD, and this just keeps coming up in surveys again and again. The teachers are just feeling completely unsupported and sort of thrown to the wolves on this. And so people go to conferences, we see stuff, you get product demos all the time, people want to show you their stuff, and and they're not getting any of those kinds of um experiences in any way like on the regular. And that's that seems to need to be a huge problem, especially with something like this that's moving so quickly. And one of the things I've liked about your work is as that we've learned new things and as things have evolved and this technology is evolving pretty quickly, you sort of updated your mental map and your writing sort of reflects that. And I think, you know, that's something we need more of, just given the velocity here.

Alex Kotran 6:07

Thankfully, I didn't have too many readers when I published my very first piece on Substack. Was it a piece about basically a lot of um like warning that we might be hitting this sort of plateau moment. And I think it was going into 2025. I was, I was preparing myself mentally for, okay, you know, AI was sort of like the flavor of the day. Obviously, a nonprofit called the AI Education Project benefits from that. And I was thinking about what is it gonna like how am I going to make the case for our work if people are sort of just over it? You know, like if AI kind of is seen as this fad that kind of fizzled out. Um But we're not there. I mean, for better or worse, it seems like the opposite. I mean, I don't, and I think actually, I feel like there's the opposite problem now. So this is my first question for you is do you feel like people in education right now are are overestimating or underestimating the the the raw potential of the technology, notwithstanding its use cases in education, but just like what it's capable of. Because you had alluded to that people aren't necessarily using the cutting edge tools.

SPEAKER_01 7:07

I think both, and I think it depends on the person too. Like I think you get some people, I mean, the old sort of 21st century skills crowd is is they're coming back out, um, and they're now now it's AI and they're they're saying the exact same things. And I think they're overestimating and then also misunderstanding how people learn. And then I think you do have some people, I still run into a lot of people who say one of two things. A, it's sort of it's not a big deal, which I think is a mistake, we could talk about that. And then B, sort of we should just ban it or get rid of it, which even if you were to believe that, I don't think that's on offer. And places that have tried to play around with that, it hasn't gone well. I don't think so. I think that's not, even if you think that's the optimal policy, which I don't, but even if you did, that's not on offer. But the people who think that it's, you know, not a transformative technology, um, I don't think fully do understand what it's capable of, you know, particularly the way it can teach itself and develop and just how fast these tools um are changing as a result. And I think they're also misunderstanding some of the macro context that we're gonna be operating in, two of which I think one is employment for kids who are, you know, now sort of our middle high school age kids and the kind of job market they're gonna be moving into. And then second, just the effects of that. I think there's gonna be a fair amount of economic disruption, and that affects education in a whole bunch of ways, including, you know, school finance pretty acutely. And so I think there's just a lot to pay attention to here. And we're living obviously the time where there's a fair amount of stimulus coming at us, there's a lot of different things, but I think this is this is one I urge people to pay attention to and not sleep on. And I do think in in that way, it is different than than technology 1.0. Yeah. One other thing I guess I'll say, I've said this in a few settings. I think one of the things I worry about is I don't I don't want to sound like a homer and that like Bill Clinton got everything right on technology. Cause I don't think that's you can go back and you can look at examples of some of the stuff they he or or Hillary Clinton thought was gonna happen on technology that didn't. But I think something he did that's really important and gore too, and is not happening right now, is when he was president, Bill Clinton spent a fair amount of time talking about the effects of technology, what was happening, the new economy, what this was gonna mean, what it meant for, you know, in practical terms, sort of skills and education, more generally, uh in terms of society. And you're just not that's not happening right now. Like there's a bunch of people talking about AI, and then there's a whole bunch of people who are sort of living their lives blissfully unaware of kind of what's coming and what this could mean for sort of middle management jobs, for just whole sectors of the economy. And I worry about that uh a great deal. And I think that's something we need to talk more about, both in education and more generally, is just what kind of leadership in explaining what this is going to mean and could mean, not in sort of hysterical terms, not in terms of boosterism, but just helping lead the country. And I don't mean that as like a partisan statement, but like that is something that Trump is just absolutely not doing. He just doesn't go out and sort of try to persuade or explain. And I think that's a loss at a at a time like this with a technology like this, this coming. It reminds me of the early days of COVID, where really plugged in people knew what was coming and everybody else was about to get a big shock. We ought to learn from that, not repeat it.

Alex Kotran 10:17

Yeah, I mean, code keeps coming up, which is

What Schools Miss About AI Power

Alex Kotran 10:20

a bit, a bit scary. Um, because it was one of those things where it was like it's it was slow and then it was fast. And it I don't know about you, it feels like we're in that pivot moment where it's not slow anymore. If I mean, I think some people might think that the Chat GBT sort of like Cambrian explosion moment in 2023 felt really fast, but I feel like that was child's play compared to because I and I know that bellwether has been like on the on the cutting edge of really like leaning into understanding, you know, like the frontier models and their capabilities now. And I we've had a lot of conversations with your team about this. Most folks in education are not actually using AI. They're like, and I know this because you go into rooms and it's like, and it's just a South by Southwest, and I was, you know, and so this is first of all, this is a biased sample set because it's people at a like essentially like a tech conference. Um, ask I always do this, I ask for a show of hands. If you ask people like who's used AI, like most people will raise their hand, but the the way to really draw out, you know, sort of like the level of depth is you know, who's used deep research? And I just say that without explaining it. And I think at in a room of like a hundred plus people, there was like maybe five hands that went up, which is completely crazy because it's like one button, you know, whether it's Gemini or otherwise, you just like click a button and then you it's two clicks, right? You click with the the plus button, a tools button, you click the research button. And it's to me, it's the most useful thing. I mean, like there's obvious, I've been playing around with like clawed code and all this stuff, but honestly, like just whatever I'm writing something, I'm going into a meeting, you know, just run some deep research queries. It's like, you know, being able to go through a thousand sources. And this is a very useful thing for people in the education space who want to be able to, you know, draw out uh what is uh, you know, ground I make sure that their work is like grounded in in research and yet people aren't using it. And it's not because they're dumb. It's not because they don't have curiosity. It's like there's something else that just I don't know that there is the the channel for them to even like it's like for me, I I learned about it because like someone sat I someone literally was sitting next to me and showed me they're like, here, like let me show you how this works.

SPEAKER_01 12:23

Um I think there's so I I mentioned your take on this. I think there's a bit of an arbitrage play happening because I talked to some people who I obviously not name on your podcast and out them who are using this quite a bit and they are working a lot less, and they are now doing in like a couple of hours what it used to take them a day or two to do. They have tons more free time, and their managers and so forth have sort of not picked up on that. And I'm I I'm not using it at that level, and some of what I do I can't yet use it for like that. It's so it to me, it it's an efficiency tool, but it hasn't become like a hyper productivity tool like that. But with some other kinds of consulting, there's people I know who are able to do that, and so I think it I I I hear you, but I think it's a little bit by there are some real power users who are really out there. Um, and there's a bunch of folks, to your point, who aren't even like really using much beyond the the open source free tools, and so they have a very different perspective of what this can do and the quality of it, even though those tools I would argue are actually pretty good. If you step back, even like the free versions are like pretty remarkable relative to you know just a few years ago.

Alex Kotran 13:31

Yeah, but the challenge, I mean, so that's interesting. It it is there uh I guess there is an advantage to if you are figuring out how to use the tools, don't don't let anybody else know because then you actually you you you basically want your organization, unless you're leading that organization, you want your organization to basically be as laggered for as long as possible because because then yeah, you have that you have that sort of internal advantage.

SPEAKER_01 13:55

Well, if it's a junior person, my attitude is more power to you. I do know like there's a there's a consultant I know who had to shift. They don't charge by the hour anymore, they charge by the project because hourly it was depressing wages too much because they were too efficient, which is

Deep Research And The Adoption Gap

SPEAKER_01 14:10

an interesting problem to have.

Alex Kotran 14:12

Let's go down this path because I I I am fascinated by this the the game theory of consulting. So, you know, we I talked to a lot of people who uh have what have like websites designed. And I was talking to some one person who was like, Oh yeah, like I was working with this web designer and and they used, you know, they vibe coded my website for me. And I was like, Do you realize like I can show you how to do it'll it it probably took them maybe like 30 minutes of actual time. I mean, like notwithstanding the time, it was like you know, actually going and code off coding and coming back. And there's something about just like right now, though that consultant is benefiting from the arbitrage of you know, I guess people's like lack of confidence and just trying the tools out for themselves. But I think so. The idea that you hire somebody to vibe code for you is like a very silly concept to me because it's yeah, it's like hiring somebody to hire you.

SPEAKER_01 15:03

It always happens, right? So that'll that'll kind of I think that'll sort of work itself through. To your point earlier about like what's surprising, I do think it's an interesting inversion that we're seeing is the rise of the creatives and some of these tech skills are the ones that are being replaced. I mean, like in in basically my career since college, you haven't seen a situation where there's been this kind of pressure on conspiracy science graduates, and where you've seen in some cases humanities graduates getting hired at higher rates. That's a fascinating thing, but will that continue? I don't know, but like that's an interesting trend. And I'm certainly of the generation where I told my kids they needed to learn to code, and that may have turned out to be really bad advice. So those kinds of things are changing. I think that should keep everybody a little, a little humble on with a disruptive novel technology like this, what we understand and don't understand about it and what it's gonna end up doing. Um, I hear lots of people being very confident about what industries are and are not gonna get disrupted. And I'm not uh I'm I'm I'm I'm not entirely confident on those things.

Alex Kotran 16:06

Yeah. So the consulting is so interesting because yeah, you have this industry that is basically built on this model of like, you know, charging for people's time and then like you have margin on top of each, you know, like like like each person's time that kind of latters up into the total cost of the consulting project. Um and for some, there will be, I think there's still some amount of runway where the clients, the buyers, as it were, are just not savvy to what the AI is capable of. But like sooner or later, like I'll just actually describe the phenomenon. So I was sitting next to somebody at this dinner, I think it was like a deep mind, and it was like some, it was a CEO of this big communications firm, like 200 people. Um, so quite large. Um and and we were talking about how, oh, you know, like do you think this is coming for you know communications consulting? And he said, Oh no, I don't think so. But in the same conversation, he described this. He was like, Oh, you know, I I was I I was like trying to, I was like thinking about buying a house, a new house, and I wanted to figure out uh the the tax implications, and so I asked my accountant, and you know, two days later he sent me this like you know five-page memo, and he said, you know, I I I'm sure he probably just used AI to write it. And I think that was like that exact moment is what I think illustrates the game theory of this, where where the minute your clients assume that you're using AI, you're sort of forced to use AI. Because why would like if you're spending all this time writing this memo and then your client is basically valuing it, like discounting the value to like, well, this is you know, at best they used AI and then maybe reviewed it. Um, at a certain point you're gonna be forced to just use AI because you know, the the there's there's no benefit, right? To like the there's no artisanship behind uh an accounting memo.

SPEAKER_01 17:58

Right. The differentiator is gonna be, I think, for consulting, it's actually similar to what you've read about in schools. Your your recent report I thought was really good where you talked about like how this actually increases the cognitive necessity and the cognitive load that people need. I think the same thing's gonna be true in consulting. The differentiation will be that body of knowledge. You'll hire consultants because they not because of the process they can do, but because of what they actually know and they can bring to the process. So it's gonna be like like AI plus, very similar to I think what you're talking about with how schools need to approach this. My concern in schools is like there's a fairly pronounced anti kind of cognitive bias in in a lot of the education sector. And so I like your your recommendation, I think, is is correct. And the sector sort of likes to do the opposite. So I worry about that. We can talk about that. But I think in consulting and professional services, that is going to be wind up being the differentiator. It's that it's that degree of content expertise will then with coupled with these tools, that will be the differentiator. And if you don't have that, or it's a part of consulting, we don't need that, then then I'd be I'd be concerned. But like what what we do is a combination of the process and then it's also very refined expertise. And I and I don't think um I think AI will impact that, but I don't think it'll fully uh it'll fully disrupt it.

Alex Kotran 19:13

Yeah, because I guess maybe to put it like differently, and like Bellweather's a great extra example here. If we could put put you under the microscope, I mean it's you're not that big. I mean, you're big, but you're not, I mean, compared to like you know, an extension.

SPEAKER_01 19:24

Yeah, so yeah, we're in we're a gnat compared to the really large ones, and we're pretty big compared to most of the ones in our sector.

Alex Kotran 19:32

Um and I think I could accurately describe you as like a bit of a bit bespoke, you know, very, you know, you're you're you don't hire bellwether to just do like busy work. Um but what you have is when you talk about your expertise, really this is like a essentially like a proprietary data set that lives within I'm sure you have documentation internally, but I think a lot of it's actually like the it lives within the brains of the experts that you employ. And that's something that you can't just get from the claude, right? Like I could go and say, oh, and look at everything that Bellwether has ever published, but my my sense is that that's not really reflective of the type of expertise that you would bring to the table and working on something. And yeah.

SPEAKER_01 20:14

I mean, but I'll be I mean, being realistic about it, I would say Chat GPT, Claude, Gemini, to some extent, can do large parts of it, but it can't do it all. And the it the all matters. So it I mean this is not a business, you know, this is not a business where getting 70% of the way there kind of gets you there. You need to get all the way there. And so that again, that's gonna be that's gonna be the differentiator. An interesting thing that you point up, the challenge for a lot of firms, challenge for us is like it knowledge management and people invest, you know, a fortune in it, whether it's Salesforce or other tools, to try to download that and codify it. And every firm struggles with it in different ways because it turns out it is, particularly among your senior people, it's that experience they have, it's their ability to call on those things that ends up being a differentiator when you're actually working with people advising, trying to come up with ideas or solutions. And and, you know, so yeah, you'll get part of the way there, but you won't get all the way there. And like this is again, you know, this is a business where to really differentiate, you gotta be able to get all the way there. Close doesn't

Consulting Arbitrage And Human Expertise

SPEAKER_01 21:15

work for more complicated projects.

Alex Kotran 21:16

I was talking to this um a senior executive at like one of the biggest companies in the country. And they I asked them, you know, they were talking there, we were talking about their AI transformation, and they had said it was kind of amazing. I mean, I wish I'd recorded it, but I asked them like, you know, they were they were talking about all this like internal training and like upskilling programs. And um, and I asked, like, have you like what are the sort of AI leaders that you've like like what are the the competencies or what's the persona of like somebody at your company that's an AI leader? And and they said, you know, it's we thought it was gonna be the technical expertise, but it turns out that like that's like a necessary but not sufficient. It's like, yes, you need like the technical people that create the structured data sets and do some of the clever stuff on the back end to enable the AI work, but But actually it's you know commun it was he literally said verbato, like it's communication and collaboration skills, which is such a powerful validation, right? Of like something that you know the uh the 21st century skills folks, I'm actually gonna I'm gonna I I I need to pull out this photo. Um because I feel 'cause you said they're coming out of like the woodwork again. Um and it it reminds me of this. Uh have you seen this? The bear coming out of hibernation. Just like it's like, it's like, yeah. No, they're back. For better or worse. The good news and the bad news is it's like, yeah, we're actually, this is not, this is not a terribly there's not a terribly novel answer to this question of like, what are the skills of the future? I mean, there is like some AI skills layered into there, but you know, this is a company that is solely focused on profit. Like they have no interest besides maximizing their return to their investors. And the fact that they are honing in on durable skills, like to me is like a signal of, okay, well, this is actually the best information that we have. We don't know what the future holds. Is that, I mean, would you say that you've seen the same thing at Bellweather? I mean, when you've because because you all are in the midst of your, I'm sure your own version of the ITS is a very important thing.

SPEAKER_01 23:12

Like you can, I mean, in this sector, I find pretty quickly you can you can suss out people who have actually really been around or in the public sector. They've been in government roles, they've served in different ways, that kind of thing. They understand nuance, they understand like there's always like a public narrative, then there's what's actually going on, and the action is sussing out the what's actually going on. Like those things. And that's that those are those are skills that you have to have had various kinds of experiences. I don't see AI replicating that.

Alex Kotran 23:40

Um, what is the underlying skill there? Like sussing out the it varies, it varies.

SPEAKER_01 23:45

We do a bunch of different things. So at Bellweather, you know, we do academic program strategy work that's actually in schools, helping schools with um school operations, culture, curriculum, uh, instructional management, all of that. We do strategy consulting that would be familiar to anybody who's been on the receiving end of that, sort of strategic uh advising. We do lots of policy research, both proprietary and outfacing. So I think one of the interesting things with Bellweather is that though we have different kinds of people with different kinds of backgrounds and who are gonna be working on different um aspects of that work. The secret sauce is we can bring them together on unique projects and really capture that synergy. But that those skills are gonna look different uh in different parts of the organization. So, like I don't do a lot with the instruct, you know, instructional management side. That's not my area of expertise. Um, and that's a different skill uh and background than than what I do more on the um on sort of policy and advising side. So it's so I'm not trying to dodge your question, it's just a different, it's a different question, it's a different question for different roles. One place I will just if you want to riff on where I see having an impact that's kind of interesting, is you will have it can do a lot of what a junior level analyst can do. You were talking about deep research a minute ago. There's a bunch, like if you're playing, you want to pull it reviews, you want to do basic analysis, you want to work with um data sets, you want to, you know, I I had a question recently. I was curious after one of these teacher strikes settled on it on school district's final offer versus the actual settlement. So, what was the best offer the district made before the strike? What did the union get after the strike and what the delta was there? You know, that would have taken and you would have had a research assistant on that and you would have been pulling stuff. Now, um, you can use AI and you can get a pretty good answer to that pretty quickly. Uh, a very serviceable answer if you're trying to figure out like the impact of these things. But here's the thing I wouldn't send what AI did to a client, but I wouldn't send what that research assistant did to a client either without looking at it. And people think this is a big dunk. They're like, well, you still have to have a human look at it. And it's the wrong counterfactual because we have to have a human look at nothing at Bellwether is always a human in the loop. Nothing goes out that was generated by AI without somebody with that content domain expertise looking at it. And so I think that's a kind of place you're gonna see these and people think are underestimating the kind of enhancements that we're gonna see. And again, that'll I think that has huge labor market implications that we can talk about. Um, but

Junior Jobs And The New Multipliers

SPEAKER_01 26:10

that's kind of the direction I see things uh trending.

Alex Kotran 26:13

Yes, it's um that feels like what I think what has changed over the last six months for me is particularly with like Cloud Code and Gemini 3.1. And now I guess it's I haven't used Codex yet, but um apparently it's quite good. Um yeah, the people in Silicon Valley are like very like very loudly shouting now for the rooftops that like, yeah, that it's nobody's gonna be it's gonna be really, really hard to justify employing juniors. And it's not that all you're not gonna have any entry-level people, but I mean I think there's um and at the same time, I was just talking to a large teacher training organization and I was asking about like who are the teachers that are like that you are seeing be the change makers in schools, and what they told me is it's it's the the teachers that are right out of you know, that that like just started. They are the ones that are not embracing AI, but truly actually like the power users. Yeah. And so I I don't know, I feel this tension of like, I'm not totally sold that like you know, people like you and me are gonna be the fit people that figure out AI. I mean, I think there is like the domain expertise that's important I'm pretty sold we're I'm pretty sold we're not. Yeah, like it's it's I mean, how how can we get to some level of advice to like a college student or a parent who's like watching what's happening and is like legitimately concerned and trying to figure out like what am I supposed to do to be on the to avoid being on the wrong side of this?

SPEAKER_01 27:37

Well, this may be wrong, but I'm uh uh domain knowledge. And I when I talk to students, I it's it's learn something about something and because then you can learn about other things. The ability to learn about something allows you then to learn about other stuff. This again, this is what we don't do in K-12 when we focus on skills, or we say it's both when people really mean it. They they diminish content. Content and domain knowledge really matters a great deal. And so that's my that is my core advice to young people is learn about something, learn how to learn as a result of that. Um, and that will serve you well. Because I do, I don't think you can fully game out where this is going. And I think that can be a little bit of a fool's errand. But like it's hard to go wrong knowing things, understanding the world. Um, say you study biology, um, and even if you're not gonna be a biologist, you understand biologic processes, you understand a bunch of stuff that's gonna serve you well and other things, obviously, history, um, political science, English, all these things. And so I think I think I think we have devalued domain and content knowledge at our peril or living in a sort of post-fact society as a result, and um just re-anchoring in that. I think that's actually gonna be more important in the AI era. Uh now, obviously, I have a bias there. I've thought that pre-AI too, but uh, I think it's I I think it's objectively gonna be uh gonna be more important in this idea that you can just look stuff up and all that. I think we need to make sure people have those skills. Also, like probability, math. We need to, we we do a terrible job teaching math. And uh my friend Kareem and me, you know, founded Citizen Math. He talks about like everybody thinks math sucks and they're right to think that because of the way we teach it. But like hoping people understand large numbers, probability, all these things are really important. So that's that that anyway, that's my answer, Alex is like we don't know. And so trying to figure out what's the thing, you know, what's the process degree you should get seems you can learn a whole bunch about data science. It's gonna be, you know, uh completely uh rendered uh, you know, rendered not super valuable by AI. Like it's figuring it it's knowing about things and then applying it to things like data science. I could be wrong. Save this clip in 10 years. That may look like a super foolish prediction, but um 10 years.

Alex Kotran 29:50

I mean, I would be lucky if that's the time. Two years, yeah.

SPEAKER_01 29:54

No, but that that's my that's my own read. Um that's my own read on it.

Alex Kotran 29:59

I uh was it was it the CEO Palantir? There was a um there's one of the one of the tech CEOs was was basically saying that you know this is this is a moment where you just need to work really hard. And there's gonna be this like, you know, the the the the the alpha of like you know, either you need to work hard to get like as much domain expertise over the next couple of years, you know, before AI really takes off, or you need to really like work and understand AI. Um But I think the the the idea that you can just the way to get ahead of AI is just to be like a power user. I'm not sold on that. Like I I think because I think about the people who spent all this time learning prompt engineering and like you know, memorizing different prompts and all the prompt prompt Bibles and stuff that came out, and like, you know, that stuff is basically now all moot. I mean, I suppose there's some skill that cut comes from just like the process of learning it and maybe understand like, but the AI is really prompting for yourself. And yeah, I I we I I'll I'll say this at AI EDU, we have we've we've been hiring for a handful of junior roles, and we went through this like process of like, okay, are we hiring someone that's just like for the AI skills? Because we do expect these roles are going to be leaning heavily on using AI tools. And we kind of landed on no. I mean, you you I mean, the one thing that changes is like the take-home test is now an AI test. It's more like, you know, and it's like we want to look at the actual prompt history of like how are you using Claude, let's say, or Gemini to get to the endpoint, as opposed to just like, you know, because previously it was like write a, you know, but like we're hiring like a fundraising lead or development lead, and it was like, you know, it would have been like, you know, write up a brief or like a proposal for a for a potential funder and like, you know, provide a little bit of research. And and now it's like that should be something you can do in like five seconds, right? So, like what does sort of the new take-home test look like? But otherwise, I mean, we're basically looking at the same stuff.

SPEAKER_01 31:54

Um and are you and I mean, I encourage young people also like it's show how you differentiate. It's you plus the AI. Like, I don't think a really compelling pitch in a lot of the kinds of work we do or you know, media, any of this kind of stuff is is you instead of AI, because it is AI is coming, it's being used for a lot of stuff. It's it's the it's it's the you plus. Why is that a differentiator relative to your peers? So it's knowing how to use it uh in a skilled way, which also means knowing its limitations and so forth, uh, and then why you are the human who should be in the loop. I think that's like if you're a young person, that's the compelling pitch. Now that's the micro, the macro. I do worry, and I don't know if you've like what you're where are I worry where is our future talent pipelines if we just decide to obliterate entry-level jobs because you can. Where do people go to get these skills? I think over time you can certainly see the market taking care of that, but like, you know, over time doesn't like help you if you're caught up in the in the in this fetoregnum that's going on. And so I worry like where where will where will you get sort of your tier of middle and senior people? This idea that you just invert the pyramid, like they still need to come, they they need to come from somewhere.

Alex Kotran 33:08

Yeah, I mean, I think I don't have a good answer to that. I think it's gonna be there will be like a transition period that might be quite rough. I do think that like all things considered, you know, first of all, I mean you've actually written about this. Like we're like right now, we're worried about there not being enough jobs, but we're not too far off from worrying from you know, like when I go to, I was I just was speaking at this event in Japan and and I was sort of like prepping with one of our partners there, and you know, and I was like, and she was like, Alex, like basically everything you're talking about, I had to like completely redo my my talk because she's like, we don't nobody cares about jobs running out. Like there's there's no anxiety about there not being enough jobs. It's a completely reversed, right? Like we're worried about there not being enough people to fill the roles. Um and so I think there's also that dimension that it's it's less acute in the US. So we're not it's not part of the political discourse really. But yeah, I mean, uh uh sooner or later organizations will be clamoring for, you know, young blood and and and dynamic talent that has the ability to learn. And so yeah, but there will maybe this period of where like, how do you make sure that you're building that domain expertise if it's hard to get employment in, you know, certain certain careers while you kind of wait for everything to settle out? But you said something that I just want to double click on, which is it's it's not it's not the AI or it's not even how you use the AI, it's like how you can use the AI to do certain things. And is that is are you basically because is it is it that AI is essentially just like a multiplier, but everybody has the same multiplier? And so it's like what is like the what is like the base factor that you're multiplying? Like is that is that framing right?

SPEAKER_01 34:49

No, because I think it's both. Your your base matters and people's base varies based on their experience, education, all that. And then I think your ability to use it in a skilled way. And it sounds like you guys are testing for that in hiring. We are like, how do you use it? Um, and are you able to because again, AI can be everything from something that helps really generate and create new content and analysis to a glorified search engine, depending on your skill in in using it. So I think it's I think if I'm understanding your question, I would say both the there's there's variance around both those things. And the real sort of to me, if the higher you're gonna be a higher multiple on both of those, if you learn a bunch of stuff and you learn how to use AI, you're gonna be you're you're gonna be differentiated. Uh, particularly right now, if you're a young person, maybe relative to a lot of your peers. I do one of the things I worry about, this whole labor market thing is this

The Triple Whammy Facing This Generation

SPEAKER_01 35:40

generation's coming out right now. We really like gave them a triple whammy. We decided let's just see what happens with like unregulated social media for young people, let's see what the effect is. It turns out not good. Um, we also were like, let's what happens if we close our schools for a year? Um, and then they're coming out into this economy that's being disrupted by AI. That's a that's a really tough uh three things. Uh and so I particularly worry about that. This will work itself out over time, but I worry about that cohort a great deal.

Alex Kotran 36:10

And then and then there's the fourth thing now, which is like the AI companions themselves.

SPEAKER_02 36:14

Yeah.

Alex Kotran 36:15

Which feels very different from social media, right?

SPEAKER_01 36:17

It's like um so yeah, you and I have talked about that. I worry about that a lot, um, just in terms of like human functioning, human relationships, society. We're not meant to have perfect friends, and AI is sort of a perfect friend in some ways, or because none of us are perfect, and so you need your friends and perfections to help you reflect on your own. And you we're meant to be in community like that. And so this is a uh, I think this for young people that I worry this is an existential. We couldn't keep them off social media, the addictive properties of like TikTok and Instagram and this kind of stuff. And this is next level what we're seeing out there with with this, and you're already starting to see it show up in some ways. Now, I'm I'm very concerned. Julia Freeland Fisher's, you know, is doing a ton of work on this. Um uh the Catholic Church is interestingly, and the last two popes have had a lot to say about it, which interesting institution that you know to suddenly be opining on sort of frontier level technology, but they actually have some interesting, important things to say about community. So I don't know, it's coming from different places, but it uh it worries me.

Alex Kotran 37:20

Yeah, one of my advisors met with the Pope's AI advisor, which I he must be the first Pope to have somebody dedicated to AI. Um do you think that is is there like an argument that that consulting is actually maybe the the best place it's okay counterintuitively consultants are maybe the most at risk because they're it's easy as this it's very easy to get you know to shed your consultants as opposed to maybe FTEs. But if I like I I have a bunch of friends who are sort of between jobs, and one thing that I've seen is that the folks who have gone into consulting, just like you know, small projects or like often building AI tools for different clients, they are learning so much. Like they have they've been put onto this because it's actually this like very dynamic space where there's like a ton of competition, but also a lot of like institutional and legacy players that are like slower moving. Um, being in consulting forces you to understand the market and the landscape just much more urgently than if you're, you know, if you're like at a Fortune 500 company, like you might have 10 years of runway before you actually really need to pivot if you're like an HVAC, you know, supplier, let's say. But am I glorifying this idea of I don't think it's how can we get some advice consultants are at risk?

SPEAKER_01 38:40

I think bad consultants are gonna be at risk. I think you're gonna see that across a bunch of industries because AI is gonna, it's gonna, it's gonna pick, it's gonna pick up some of the slack. The thing about consulting, I joke about it, like, is you you you're getting paid to learn. It's an that's like that's the that's the really fun thing about it, is you're you you have to learn and stay current so you can help clients. And then as you go through the course of of various projects and advising and so forth, you're you're learning. And so you're just you're you're consuming so much information, which is fantastic. Um, and I think that will that that will continue. It's a learning field. It's I think it's the consultants who aren't good at that or doing sort of things that are easily replicable. Those are the ones who I I, if I were, I I would be concerned. But like for a firm like Bellwether, like we're pretty differentiating what we do. So I'm not uh, you know, I'm uh that that's not an area of like extreme uh people are gonna need that. But um, and I think you'll see that. Look, I think you're gonna see that in sort of yo, AI is gonna disrupt law a lot, but there's gonna be certain kinds of law that are you're still gonna yeah, are gonna be heavily human focused just given the nature of it. I think you'll you'll sort of see that across the board. So I don't think you're like romanticizing it, but like good consultants, like it sounds like the people you're describing who are constantly in that learning cycle. Like, I think there's gonna be plenty of work for them, potentially more, because it's these sort of disruptive times where there's a lot of turmoil that ends up creating more opportunity uh to help and advise. I wish there was a little bit I wish there was a little less of that just now. There's a little more, a little more turmoil around government than than than I would prefer, but it it but it does create uh it creates opportunities to do various kinds of work because people need help.

Alex Kotran 40:22

Yeah, I'm trying to get to like advice for a kid in college, um, I think irrespective of what their major is. And I guess working backwards from you, like you need to be learning because like the biggest risk of like the next like five years is is you being cut off from opportunities to learn. And so um consulting feels like a I it I feel like someone like if you're like a freshman or a sophomore in college, I think you should be like challenge accepted, like take two years and become a like an expert at you know applying AI in whatever domain it is that you've chosen, whether it's you know computer science or business or you know media. I feel like across the board, you can look at every single sector of the economy, and what you will find is that companies are clamoring to figure out like how do we find people that actually understand how to use these tools. And so, you know, I think that's now that's not a solution for everybody, but I think right now if you're trying to figure out like what can I do, I it but you have to be learning. And so trying to do projects for, I mean, even like the local pizza shop is going to be a really interesting learning experience where it's like, okay, like how can I use AI to um uh you know help them do some more social media or marketing or um go volunteer for a nonprofit.

SPEAKER_01 41:49

I mean, I think there's help you improve your writing. So when I teach, I always encourage my students to get a writing buddy and give each other feedback. Um I talk to students, I do some work um down at UVA. I'm a um visiting fellow at the education policy works down there. And with the students, and and you tell them go get a writing buddy, but half of the thing, you know, it's it's a little awkward to walk up to somebody and be like, Hey, do you want to be my writing buddy? So some of them do it, but some of them don't. AI now means you don't need that. That's your writing buddy. It can give you if you if and I tell them, I'm like, if you have it doing your writing for you, first of all, people are gonna be able to tell because AI writing, anyone with a trained eye can spot it. Second, you're not learning anything, you're saying you're you're not helping yourself. Um, but it it's great at giving feedback. It's really a good tool to be like, hey, give me feedback on this and so forth. And so suddenly you have a writing buddy who's available 24-7 whenever you want that feedback and you want to do that. And so, like, to your point, if you're an undergrad and you're not sort of taking advantage of that to hone your skills, like you're you're gonna be falling behind because other people are going to be doing that. They're gonna be using that to get better, they're gonna be using it to get that kind of feedback, they're gonna be using it for interviews to prep and so forth. And so it's gonna come in at like a higher level of preparation, they're gonna up their game. And those are the people you're competing against uh for jobs. So I think I think you're in the nail on the head. And again, like while you're doing that, like learn something, major in something and learn something, even if it's not what you're going to do for your career, like the having a body of knowledge. I just can't like I'm halfway through a piece I haven't finished writing it just on why we're gonna need a better, everyone's gonna need a better bullshit detector in this new era. And the best way to get that is to know something. Um, it just helps you think more skeptically and critically about other things. And that's the big fallacy, by the way, is that you can teach critical thinking absent that content. You teach it by you get the content, then you learn to think critically, then you can start to apply that elsewhere. And in education, we get that backwards. We talk about let's teach that skill, but like you know, give the kids much to actually think critically about content wise.

Alex Kotran 43:53

Yeah, that's so eloquently put. We're because we because our journey was started out really focused on AI literacy, and then And we quickly realized that like you can understand how AI works or what it is capable of, but really the most important skill is critical thinking. And then you get to well, how do you teach critical thinking? And it's like, well, you don't just teach critical thinking, you you build those skills in English and social studies and math and science. And and that was sort of our journey to actually like school transformation as the the end point as opposed to you know just sort of like discrete like AI set of skills. But what's what you're saying is if you want to build critical thinking skills, yeah, you just have to like learn about something and that and and and be in an environment where you apply it. And it that's that sounds to me like quite a strong endorsement of like for those who are wondering whether they should go to college, you know, maybe don't take out like a $200,000 loan to go to like a tier three private school. But but I think yes, right.

SPEAKER_01 44:51

It sounds like I'm I'm a proponent of college. One of the things you see, like people who go to college have better outcomes across the board, and it doesn't matter what kind of job college educated plumbers have better outcomes than non-college. It doesn't mean that you can't like these other things, but it it there's a there's a premium there. If you really don't want to do that kind of work and you do want to do the skill trades, you should go do that. I think that's really important work. I do think we do an incredibly poor job helping kids understand what these careers involve. I'll tell you a funny story. I was thinking about maybe ready about this. We had a duct problem at our cabin this weekend, and where we are, it's pretty remote and it's a little difficult to um uh get repair people up. So my wife and I are like, we're just gonna have to fix this ourselves. So we were down in like a two and a half foot crawl space fixing ducks uh all Sunday afternoon. Like that is miserable work. Um, I mean, I you're wearing a respirator because there's like all sorts of stuff shit that's bad for you, and like it's really bad. And we did that for an afternoon. It's kind of a funny story. Um, you know, in your little your back soul sore afterwards, like doing that every day, that is brutal work. And I have all of our friends in Washington who are like, Oh, you should go do HVAC. None of them would like make it a day doing that, um, like let alone like on the regular. It's brutal. And and so I just think like we we we need to in general help people make. We we do a really poor job helping kids actually understand what these different opportunities, what they're gonna look like, what the career paths are, and then let them make. If you love working on cars and you love engines and stuff, you should do that. Like forcing you to go to school and get a degree in something you're not interested in is is stupid. And to your point, going to a school where you accumulate a lot of debt for a less valuable credential is a really bad decision. But helping them understand and weight that, I think is like the best thing we can do rather than what we do now, which is all these various kinds of social engineering that everybody should do a trade or everybody should go to college or whatever it is. Like empowering people to make choices in their own lives, like they'll make mistakes. We need to make sure we're paying attention to switching costs. But like, I think that's the best thing we can do than than what we essentially do right now, which is like de facto make the choice for people in different ways. Anyway, so that's a that's a little bit of a riff off of AI, but I don't think AI sort of changes that fundamental equation that we don't give kids good exposure to what jobs are actually like. And um we need to do a better job of that. And AI, by the way, can help with that. Like I think that's a place where AI can play can help expose kids to things, help them. I mean that that there's there's clear so many of the use cases, interestingly, Alex, are not that I find most compelling, are not the actual teaching and learning, this idea that we're going to replace teachers with AI. It's all this stuff around it that AI can do that's actually interesting and compelling. Whether that's like what we're talking about here or special ed, executive assistance for teachers, helping teachers communicate and handle paperwork. There's all these very interesting um use cases um uh that aren't the actual immediate student-facing core

AI In Schools Without Replacing Teachers

SPEAKER_01 47:58

piece.

Alex Kotran 47:58

Yeah, but let's talk about that because I feel like it's a very lonely time to be a sort of qualified AI skeptic. Um and I feel like it's uh I was talking to one funder and they were like, Alex, I feel like you're I I know, I know that you're like the, you know, you're one of like the vocal AI skeptics in the space. And I'm like, I'm really not an AI skeptic. I'm like, I'm I'm a power user of AI, my organization uses it. I'm a like, you know, vendor skeptic, I think. Like, right? It's like I I'm skeptical of some of the current modalities of how AI is trying to be sort of like forced into schools, but you know like what when you close your eyes like 10 years from now, you know, I is like and and and and and you're imagining like that the best schools in the country, public or private, you know, like like how prominent is AI's role in those schools? Like, is it yeah, is it there?

SPEAKER_01 48:54

Is it it's a great question. I think prominent, it's gonna have to be because its role in our lives is gonna be prominent, but prominent how? And I think less in terms of this idea that we're gonna replace teachers with AI and kids are gonna be learning that way, and more all the various kinds of supports, instructional coaching, again, things like IEPs, helping teachers with paperwork, helping teachers grade. Now, an interesting thing is there's a culture shift that needs to happen. All these, not all, but many of these like really compelling use cases are things that parents right now are actually the most skeptical of. So when you poll parents, they will tell you they think of a teacher's using AI to grade papers, they're not doing their job and they're against that. That's actually a pretty compelling use case that could really uh help teachers ease the paperwork burden, allow them to use their time uh differently and potentially uh more effectively. But we've so we've got some hurdles to get over just with some of the cultural aspects that people have around this. But I think it's gonna be those, it's gonna be those kinds of things. And and a little bit if you think to ed tech 1.0, like the there's some cool tools in the classroom. I think people who like this is not binary, and people are like, oh, it's all garbage or it's all great, like, you know, or most of it's great. It it it ranges, but less compelling to me is is some of the classroom stuff than all the things we can now do that tech it's that is is a technological uh advancement. When I first started doing this work, we didn't know how many kids graduated because we couldn't track them, we had no denominator to figure out. And so you'd make these crude estimates based on senior year and so forth, which were wildly inaccurate. You know, and Mark Warner, when he was governor of Virginia and head of the NGA, came along and and and started to do work to um address that ultimately led an effort to start to be able to use to compute these things accurately. That was because of technology that didn't previously exist. You can see that sort of across the board in the way we we do a number of things. I think the lesson here may be the same thing. A lot of these performance and support tools may end up being more powerful than some of the student-facing stuff. That said, I mean, if you got Diane Tabner in here, she would like to disagree strenuously with me on that. She thinks like there's a whole, you know, so there's there's um, I think there's room for lively disagreement. I may be lowballing what we can expect. I've I have been surprised by some of the people in this field who I really respect on instruction, um, who are just really deep on instruction, who are fairly bullish actually on AI in ways that is has has sort of startled me. Um, and so I think I think there is, it is, it is that that could be an issue that's evolving. Or we may all be, you know, kind of updating our thinking on that before too long. But right now, I tend to think it's these wraparounds, these supports is more um where the action is. And I think you'll see that. And you and I have talked about this. My dystopian concern is like, what happens if you still have, and we sort of have some of this now. You know, you get affluent kids from private schools, they're getting content, they're reading books, they're doing all this, and poor kids are getting taught much more by AI and other things. Like, you know, I'm not I'm not sure that portends a great future. And we're already we're going there a little bit, like you talk to private school kids, they tend to be reading more books than they're reading the public schools right now because there's been all the pushback on this. Um, not every private school, and I don't want to romanticize it, but higher quality ones, you get smaller group instruction, it's still a little more text based. Like seeing that play out um, you know, at at some scale, that that is troubling to me. Um that that seems like, you know, I want, I want, you know, the basic idea of John Dewey, you know, you should want for all kids what you want for your kids. Like that is to me sort of a that's not a bad North Star um in the sector.

Alex Kotran 52:27

Yeah. That I mean, that's that's that is truly like how I have formed my like that's my heuristic is um what are the elite private schools going to look like? And even if that's not necessarily entirely feasible, like that should be what we are trying to unlock. And yeah, most of the private schools, I mean, I was just talking to this 17-year-old girl who is in, I mean, surely one of the top private schools in the country, and they're very anti-AI, like very anti-AI, to a point where like it to their detriment. But it's interesting because she she was actually at our South by Southwest workshop where we were doing like vibe coding, and she built by far the coolest tool. It was like uh, it was like a therapeutic game for kids with ADHD. And it was like based on all this research that she had done in a class that was like basically just a class where you just conduct research on a topic. And um, in about 20 minutes, she built the coolest game because basically she uploaded all her research and she now had domain knowledge of this of this of this area of like non-pharmaceutical therapies for ADHD and was able to build something a lot more interesting than like some of the gimmicks that other people were building. And I was interviewing her afterward, and she was talking about, yeah, my school is like really anti-AI. But I was reflecting on if their school hadn't been so adamant about like keeping AI out of school, you know, would she have had this opportunity to like spend a whole year doing research? Like, you know, I worry about a situation where a teacher says, Oh, you can just let's just teach you how to use the deep research tool with AI. And like, I think you or I can really leverage that tool for benefit because we've kind of gone through the revs of actually doing the manual research and like understanding what doesn't even like a good source look like versus a bad. Like, I can, you know, roughly speaking, go to a source on a website and you know, ascertain like, is this something that I can trust? And you talked about that earlier about sort of just the ability to develop a critical perspective on things. It it has sort of like it's cross-functional beyond between domains, but you have to kind of do it at least once.

SPEAKER_01 54:26

You gotta know how to do things the right way before you learn the shortcuts. And I think that's true of everything. And like, if you like like carpentry, I'll use like I'm a fairly mediocre carpenter, right? Like, I need to do things very carefully in the right way. People I know who do this either professionally or just much more skilled, they're doing shortcuts. I watch them and they're doing, but it's because they have that background and training that I don't have, and so they can do the shortcuts. I can do shortcuts with things, some things I do professionally that somebody just coming in would not be well served by doing. It's because I've been doing it for a long time. And so there's sort of analytically, there's tricks and so forth that can save you a little time and and and and just various kinds of shortcuts. Uh, and and knowing when not to use them is also important. And so I think that's the with kids, you got to learn this stuff. And I worry, again, in schools, we're gonna jump to the shortcuts before we teach you the ways to do these things. And I think there'll be a real price to be paid for that. I'm I I don't think AI obviates some of the core ideas of education uh that have proven to be effective. And and and so going back, we were talking about like the the again, the 21st century skills or whatever. To me, most of this stuff isn't actually new. What's new is the need to democratize it. But most of the things we talk about, like these are skills that people have had literally for millennia. Um, but it's the need to democratize. People more and more people need that's those skills with the economy that we're moving

Screens Social Media And Policy Confusion

SPEAKER_01 55:52

into. One thing just on schools, I just want to pause on for a second, with something you said that I think is interesting, is I think we're at this weird time where concern about social media, concern about screens, and concern about AI are all getting conflated in really unhelpful ways. And school is going to end up bearing some of the load on that in ways that might not be productive. It's I think it's like it's entirely possible you're really concerned about social media and its impact on kids and not think we should just be like banning all screens in school or getting rid of all ed tech. There's a there's there's a lot of nuance, but it's all getting kind of mashed up in this political moment run around this stuff.

Alex Kotran 56:29

The narrative around ed tech, because I've heard this before. It's like, well, we've had all this ed tech thrown at schools and like look at the nape scores. But if you actually look at the nape scores, it's like, you know, right around the computing revolution, they actually were going up pretty significantly. Like it wasn't until like the the 2010s where you started to see like a little bit of a like a leveling off and it started to deteriorate. But I think that we're I think ed tech is not just like a one point in time. It's like it was this sort of 30-year process of technology evolving. Like is that is that your video?

SPEAKER_01 57:02

I think there's correlation causation fallacies all over the place in this. And so people can you, the the scores have gone down, so people can pick whatever their their thing is. Um, I think there's circumstantial evidence around certain things that I think is stronger than than other things. I think some aspects of public policy and all that. But yeah, some of the evidence on ed tech is is positive. I mean, the the real story is with all this stuff is variance. It's extra all this stuff's extremely high variance in the results we see. And then also, what's the counterfactual? Because people always like assume like you're replacing like a super high level of instruction with a super low level of ed tech. Often you're replacing a low level of instruction, and we we have a problem delivering powerful instruction in a lot of schools. You're replacing that with ed tech that may be marginally better. Now, is that optimal? You know, that's a good, that's a very legitimate question. Is it marginally better? That's a different question. Um, so I think a lot of this stuff gets it gets it gets all bound up in in an unproductive ways. And I might like I'm watching politicians to me, the real action with kids and screens and technology is like what happens when they leave school through the afternoon, evening till the next morning, especially overnight. You know, a lot of them are on it. And um, politicians can't do a ton about that in in in reality. What they can do a lot about is what happens in between like eight and three, you know, or seven and three when kids are in school. Um, and so schools are gonna then disproportionately bearing the load because a politician wants to show, hey, I'm doing something, I'm aware of this, that's where they're gonna go. And I'm not sure, as with some other things that are sort of more general society things, we decide to fight out in schools. I'm not sure how productive that's gonna be um for educators. And I wish, look, an interesting thing out there, you mentioned earlier some of these teachers, these younger teachers, it's younger, it's they're not all young. There's some mil, there's some let's let's let's give a shout out to the olds, like like me. Um, it's a it's a mix. Um, but one of the things when you talk to the ed tech, the companies, the AI companies, and you've probably seen this, they they're getting these teachers who are virally and sort of organically signing up for these things. So you'll talk to a company and they'll be like, Oh, yeah, we're in we're in 20 school districts and we're doing this, and then they'll tell you how many teachers they'll they have, and you'll be like, Well, that doesn't like add up. And they're like, Oh, yeah, we've got like another like 1,500 teachers, you know, we're adding, you know, 10 a day or whatever, just kind of find us. That's interesting. Those teachers are the ones I want to hear from because they are obviously like the power users, they're out there, they're seeking out these tools, they're trying to figure out how to use them and do stuff with them. Like, we should be like harnessing and capturing that. Those people have like learning to share and so forth. And we need to we need to find them, we need to hear from them, we need to support them. And and you hear this across like different vendors, just these sort of free, you know, I don't know what you want to call them, like go free radicals out there in the system. Um, uh, and we should, I think we should be doing, we should be doing more, uh, we should be more supporting and learning from them.

Alex Kotran 59:56

It's it's so interesting because you you started out with with like a pretty strong opinion that I that I share that with it, which is concern about the about social media, about you know, like frictionless AI relationships. Um but then you kind of ended on this point of like, and and there's going to be this like reflexive uh push to remove technology from schools to to the detriment of kids. Like how like what should what is the position that that you think like how how can we articulate this position? Because like I think that like sitting with the tension is I find really hard. And when people ask me, like, what is your opinion on screens? And it's just like, well, that's how do I even answer that? That's I don't have an answer either because it's too broad of a question.

SPEAKER_01 1:00:40

I'm excited. Let's look, I think what an interesting thing is happening is we used to think of states as a laboratory of democracy. Now we're gonna have nation states or to some extent. So Australia is banning it um, you know, from younger kids. That's like I think it's gonna be useful to watch, see how that goes in different places and figure out what we I don't think we I don't think we know yet. And I think if you start in a place that like all ed tech is bad or we can't get ed tech to kids fast enough because of the world they're going in, I think those are the wrong places to start from. There just needs to be a lot of learning. The evidence, I don't, you know, I don't think Jonathan Haidt is wrong. The evidence on sort of social media and kids is bad, and the evidence on kids being distracted is really bad. And I wrote a while ago I felt like these cell phone bans were sort of a white flag of surrender that like we can't compete with any of this stuff, so we're just gonna ban it. Um uh which isn't inappropriate. We ban, there's there's other stuff we ban for kids for various reasons, but I think there's just still a lot of learning to do, and I think it's important to just disentangle these things. AI is distinct from ed tech, is distinct from social media the kids are accessing on their phones, and we need to talk about each if we lump these together, I don't think we have a great conversation.

Alex Kotran 1:01:54

But to have a good conversation, you need folks who are like informed, at least curious, and like to your to something you said earlier, like have this posture and orientation towards learning. And like, how do you like maybe thinking about like whether it's clients or just schools that you're familiar with, districts you're familiar with, that have successfully created this like culture of innovation and a culture of learning, not just for like the students, but for the teachers and maybe even the parents. Um how how do we how can we guide districts to sort of bring their communities along for what is unfortunately a journey and not an answer, right? Like it's not like you can tell parents what the solution is, but but I think if parents aren't engaged in the question, you know, they're rightfully going to just want they're gonna want answers sooner or later, and the you know, they may not necessarily be pushing for answers at the right time.

SPEAKER_01 1:02:45

If you go to Bellwether's website, you look under emerging approaches, there's a whole AI section, and some of

Building A Learning Agenda That Sticks

SPEAKER_01 1:02:50

the more interesting stuff there. There's some stuff on um some case studies of how AI is being implemented, which I think you know, there's gonna be a half-life on that because there's so much going on. But right now, those are very, those are very valuable. I'd urge people to check them out. There's some stuff just on measuring and evaluating and making sure we're actually setting up ways to learn from this and do what you're talking about with innovation. Um, and there's not enough of that in and uh in education on obviously, you know, uh innovation and and fad travel pretty closely together. And so you've got to be uh evaluating and and and being rigorous about it. I think you see this in different places. The problem is it's not sustained. You get a superintendent, they come through, then they move on. Um, you get a school that does it, and then the leadership changes. Um, and so we sort of what we have in the sector in general is sort of random acts of of innovation um and then random acts of learning, but we don't have sort of a systemic uh approach. Uh if there's any, you know, if we ever get a good federal RD function back, that's a place where we need we need a lot more. But I can't point you to a bunch of places and say they have it right, because again, it is so episodic. Places try different things, they move on. And then look, the the the charter ventures, you know, I mean there's a lot of learning from what happened with with Summit. There's a you know, there's a lot, there's there's learning, there's learning good and bad with things that have succeeded and things haven't succeeded. Um, like we have a bigger problem, I think, with with that stuff not traveling and people learning than we do with sort of people trying sort of different things and trying to figure out. You and I, you know, we've been there's some interesting stuff happening in Ohio. They're trying to take like a statewide kind of approach on this and support their educators. And, you know, you and I are friends with Nick and we're watching kind of what's happening there. You know, stuff like that. I think is it should it all be statewide? Probably not. But what Ohio is trying to do, I think is really interesting. Let's watch and see what what comes out of that. So I think it's uh it's there's a bunch of different uh places, but I can't point you to one place and say if everybody did this, we'd be we'd be better off. I do think there's just a as I said, a lack of support for teachers and a lack of sort of clear rules of the road, and that is a big problem, particularly again. I think we're gonna see a real focus on policy here that may be kind of uh card ahead of the horse stuff before long.

Alex Kotran 1:05:07

I mean, but I think this is like I think you just articulated why you know when when funders ask me, well, you know, what is the like what is like the case study that you are trying to scale? And unfortunately, like I I agree with you. I think there are I think there are actually some charters that are um that are doing amazing work, and I think of like like Dream or Intrinsic or um you know design tech. But the challenge is like I was like, well, sure, it works in this like small school, but that's not necessarily going to work in a big district, which I think is probably right. But it but at the same time we're all sort of grasping in the dark right now.

SPEAKER_01 1:05:48

Yeah, so like I think that's what you need to do they need to shine some lights. We need more people shining lights.

How We Stay Current And Closing

Alex Kotran 1:05:54

That's where the action is, it seems to me. And just to bring us home, like you talk about this like learning agenda. I mean, you've been talking Talking a lot about just a posture towards learning. Um, where do you feel like you are learning the most as you kind of try to understand this landscape, whether it's AI or education or anything else, just you know, and any any content medium, uh, you know, this is dealership.

SPEAKER_01 1:06:18

It's it's three things. That's a great question. And I think everybody's doing it differently. I I want to turn this around and want to hear your answer. One is is there's a lot of good content out there, yours among them. Um, I read your stuff. I read critics. Like I think Ben Riley, you people, if if you're not reading Ben, you're missing out on a smart um critic. He he's strident on this, and some people don't seem don't like that, but like he's got a point of view and he's not afraid to say it, and he's raising uh criticisms. Obviously, Robin Lake, the work we're doing at Bellwether, and we have a specific newsletter on AI. If you go to our website, you can sign up for it, as well as the work we're doing. You know, Robin stuff is great, John Bailey. Um, there's you know, Dan Meyer on the math pieces, he's been talking about AI. So there's a lot of the content out there, and then there's more general interest AI stuff and on the sort of sector in general. I don't know, I've been to your take. I think the economists coverage is the best right now of sort of mainstream sort of legacy media, just their coverage on AI in in in general, um, is just is just excellent. Um, the second way is convenings. You just got to be with people. It there you you said it's like it, there's it's it's we're shining trying to shine light in the darkness. You got to just talk to people, hear what they are saying. And that's when I've gotten some of my biggest aha's when people say things and you're like, okay, wow, particularly people you like regard who have real subject matter domain expertise on certain things, and they're pushing your thinking. So, like any kind of new thing, you've got to get out there and and talk to people. And then third, educators, and that's at all levels of the system. Like, how are they experiencing this? And that includes, like, you know, we're talking about all the people who are power users or hyper adopters, but people who really aren't thinking about that much, too. That's valuable also, just to try to get a sense of uh what's happening around the country. Cause I think we can easily over-index on places that are sort of doing a lot and miss sort of the whole landscape. So those three things um combined. And I think again, given the velocity of this, I think those three things combined, they give you like a head start, but they I still don't think that gives you a complete field of view. I don't think there are enough right now, hours in the day, uh, even if you're gonna be entirely focused on this, and especially if you're not to sort of stay up, stay up because there's so much, but again, that those things I think they at least give you some kind of a head start. But what about you? Like, how would you respond to that question?

Alex Kotran 1:08:39

I mean, I think you can basically nail that. I think you you you closed with the most important piece, which is you know, you you have to be like uh steeped in the cutting edge perspectives, but if you're not also talking to you know teachers on Main Street, um it's very easy to get caught up in the this assumption that like everybody is sort of like having this conversation. Most teachers are not actually talking about AI more than uh once a week, once a month, even like for a lot of teachers, it's just not something that is that is top of mind. And and that's why when I saw that when I see the data, like the the latest Walden Gallup survey, and it's like you know, skepticism about AI is increasing. Completely unsurprising to me because like to your point, most people are not going to the ASUGS V Summit or South by Southwest. And so I think but you have to balance that with that's why there's even more value in folks understanding the cutting edge. And I don't just mean like technology itself, but like the research and like the learning sciences. But also the one thing I would add is um you have to use the tools, like you really have to use them. And it's like A because they are genuinely helpful, but also because what are we doing if like we're talking about this technology as if like and it's mostly through hearsay? Like it's just weird to me that like so many people that are supposed to be like literally shaping our strategy about this. And you ask them, like, oh, have you been, you know, trying, have you tried vibe coding yet? And they're like, Oh, I haven't gotten around to it. And it's just like, How are you? And it's reporters, even. I mean, it's like just my new thing is I'll like and uh when reporters are interviewing me, I'll be like, Well, have you have you used deep research? And they say, if they say no, I like we'll do run a research query on the story that they're writing and send it to them.

SPEAKER_01 1:10:17

Um you know who's great on that is John Bailey. Has been like, yes. So and and John, he writes, you can find him, and he's out there advising our folks. He uses all of these tools. He does all, and when they come out, he does different stuff with them. You know, it's not related to his professional work, and he's just like a constant learner, and it's kept him really at the sort of ragged edge of what they can do. I totally agree with you. And use it for stuff that's not your work, like use it around your house. Do have figure out tasks that you're doing that you could see if that that's I've learned a lot just using it for other stuff and then seeing how it performs or doesn't perform.

Alex Kotran 1:10:53

Yeah, and I almost feel like we who is the who is the equivalent of like you or or John Bailey for a school? Because I think the beauty, like like the the feature I envision is one where every school has a couple teachers that are like, whoa, have you guys tried the latest blank? And you know, they're like in the teacher's lounge, like sitting next to some folks at a laptop, just like sharing knowledge organically. And I think that's something that if you're if you're surrounded by um, you know, really smart people, that's something that we take for granted. But that feels solvable. Like that feels like, I don't know if we can get to every teacher in the next couple of years. I do think that we can get to every school and make sure that there's like there is like one person that is just like learning and figuring out how to translate that learning. Um, and so Andy, just to to end on that, I mean, I just want to encourage you, um, please keep writing. I'm really interested. You mentioned two different pieces, both of which sound really interesting. So I'll um we'll share the link to your uh to your sub stack in the in the description. And um if uh and then we shall by we should write something.

SPEAKER_01 1:11:54

I would like that. You and I have had a couple of ideas that I think it'd be great to revisit. I've been I haven't been publishing much lately. I need to get uh I need to get back on. I've been busy with a few other things, duct work, as it turns out, but um more more more stuff uh more stuff than that. Um but yeah, I need to get I've got a few things I so I I previewed two that I've been kind of noodling on. Um I really do appreciate your writing. I look forward to getting your newsletters that are thoughtful. Um uh and I appreciate just what you're doing out there. We talked about shining a light, like you are doing that, and that's uh that is super important right now. And and you know, people are stepping up to do that uh in schools, it's set the system of we talked about Nick earlier, like a super valuable. Yeah, thank you, Alex.