Teaching Math with AI with Amy Langville, Chloe Lewis, Lew Ludwig, and Kathryn Pedings-Behling

A few years ago, it was pretty easy for math educators to ignore generative AI. The chatbots of 2022 and 2023 were notoriously bad at math. But that’s no longer true! Today’s frontier AI models are very good at math—to the point of proving mathematical conjectures that have been open for decades. 

Today on the show, I have a roundtable of math faculty sharing what AI-aware mathematics teaching looks like here in 2026. Our roundtable guests are Lew Ludwig (Denison University), Chloe Lewis (University of Wisconsin-Eau Claire), Amy Langville, and Kathryn Pedings-Behling (both of the College of Charleston). 

Episode Resources

The Science of Learning Meets AI by Lew Ludwig and Todd Zakrajsek (2026)

Chloe Lewis’ website, https://www.chloelewis.net/ 

Teaching Mathematics with Generative AI Google Group

Episode 10. Deconstructing Calculus with Amy Langville and Kathryn Pedings-Behling (April 5, 2023)

Deconstruct Calculus Project

Episode 85. In-Class Writing with James Seitz (March 10, 2026)

“Types of Students’ Justifications” by Larry Sowder and Guershon Harel in The Mathematics Teacher (1998)

Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy by Cathy O’Neil (2017)

“The Agents Are Waking Up” by Marc Humphries on Generative History (April 17, 2026)

Episode 11. Generative AI in Computer Science with Brett Becker (April 18, 2023)

“Computer-Based Maths: The Change” by Conrad Wolfram 

Transcript

Kathryn Pedings-Behling 0:39

I had students being like, I literally had no idea that that AI could be used in this way. Like I had no they they said I will never study the same way again, like for a test, period. Like I had no clue that I could do this.

Derek Bruff 0:57

Welcome to Intentional Teaching, a podcast aimed at educators to help them develop foundational teaching skills and explore new ideas in teaching. I'm your host, Derek Bruff. A few years ago, it was pretty easy for math educators to ignore generative AI. The chatbots of 2022 and 2023 were notoriously bad at math. That, however, is no longer true. Today's Frontier AI models are very good at math to the point of proving mathematical conjectures that have been open for decades. Today on the show, I have a round table of math faculty sharing what AI aware mathematics teaching looks like here in 2026.

Derek Bruff 1:36

Lew Ludwig is professor of mathematics at Denison University, and co-author with Todd Zakrajsek of the new book, The Science of Learning Meets AI, which I believe is out to the very week this episode airs. Lew is one of the organizers of the Teaching Math with Generative AI Google Group, which is how I found out about our second guest, Chloe Lewis, assistant professor of mathematics at the University of Wisconsin at Eau Claire. That Google group meets twice monthly on Zoom, and Chloe recently shared her Critique the Chatbot assignment, in which students evaluate ChatGPT's attempts to prove mathematical statements that students had previously disproved.

When did you realize you wanted to be an educator?

Derek Bruff 2:16

And I'm glad to welcome back onto the podcast Amy Langville, professor of mathematics at the College of Charleston, and Kathryn Pedings-Behling, online education and e-learning coordinator, also at the College of Charleston. You may remember them from way back in episode 10 of the podcast in April of 2023 when they shared about their line of innovative mathematics textbooks called Deconstruct Calculus. These books are packed with creative activities that invite students into embodied learning, often requiring students to do something in the physical world that helps explain the mathematical concept.

Derek Bruff 2:50

Yes, it's another four-guest episode. And with all these smart people, the conversation runs a little long, so I'm going to wrap up this introduction right now. Thank you all for being on Intentional Teaching Today. I'm excited to have some fellow uh mathematicians on the podcast and to talk about how AI is changing the work of math education today. Thanks for being here. Very excited to have you all here. Um I'm gonna start with my usual opening question. Um, and it is this uh can you tell us about a time when you realized that you wanted to be an educator? And uh Kathryn, maybe I'll start with you.

Kathryn Pedings-Behling 3:30

Sure. Um, yeah, so I was the typical child. I always knew I wanted to be an educator, so I'll answer this by saying when I knew I wanted to be a math educator, and that's when I had class um with a high school math teacher whom I had for my junior and senior years named Dr. Nancy Long, who actually like completely changed my life. Um, and I knew at that point that I wanted to be just like her and be a math educator just like her.

Derek Bruff 3:54

That's awesome. That's awesome. I love that. I had I had a couple of key high school teachers too that kind of got me in that direction. What about you, Amy?

Amy Langville 4:03

Uh mine was later than Kathryn. I was in graduate school trying to decide what to do with my uh degree in operations research, and I was a TA in the math department, and they forced me to teach my own class, and I didn't want to do it and protested, but they they made me and then I found out I loved it. In hindsight, it made me made sense because I had been all throughout high school and um college. I was an instructor of physical education type things like basketball and volleyball.

Derek Bruff 4:31

So uh kind of a natural teacher, perhaps. Yeah. What about you, Lew?

Lew Ludwig 4:38

Um I would say so. I kind of in college, I kind of landed in what was called our CLC, the computer learning center. And uh I they had a very interesting thing set up where you would actually kind of coach or teach one-on-one different uh software packages, Lotus 123, Microsoft, or no, it wasn't even Microsoft back then. It was MultiMate, all these crazy things. But anyway, it was something that I kind of naturally took to and really enjoyed. And the fact that I look at my pedigree and both parents were teachers, and I have a grandparent that was a teacher, and I married it. It was just kind of like in the water there. So yeah, always been interested.

Derek Bruff 5:11

Yes, I've had many folks on the podcast who point to a family pedigree of teaching. Yeah. Um, and Chloe, what about you?

What impact has generative AI had on the teaching of mathematics?

Chloe Lewis 5:20

Um, I was an undergraduate at Northern Michigan University, which is in the upper peninsula of Michigan, pretty remote rural area. Um, and when I was there, actually, it was sort of undergoing a really big like economic shift. Like we were transitioning out of being like a coal mining area into, I mean, it was just like Obama-era energy sort of efficiency kind of thing. We're not going to be coal miners anymore, right? So there was um a lot of money to like re-education grants for um people who were like returning to school after a really long time. And I got the chance to work with the developmental mathematics course as an undergraduate while I was there. And I just really liked the like diversity of a college classroom in that setting. So that really made me want to be an educator and also to work at a public institution.

Derek Bruff 6:05

Oh, that's great. That's great. That's maybe a little closer to my own, is I knew I wanted to be a math major. Um, but when I started actually kind of doing some tutoring as a college student, that's when I realized I really love this education bit. Well, thank you all. I think that gives us a little, a little sense, a little glimpse of the kind of who you are. Um, but let's jump into this AI discussion. Um and Lew, I'm gonna start with you mainly because I know that you're one of the organizers of this math and AI group. Um, and I know that you've been very active in kind of writing and speaking about AI over the last few years. What how would you describe AI's impact on college math teaching since Chat GPT was launched in 2022? Have you seen any kind of trajectory there?

Lew Ludwig 6:51

Yeah. So okay, so the day that we'll live in infamy, right? It was November 30th, 2022, as you pointed out, when ChatGPT 3.5, which is now retired when that landed. Um, I think by this early spring of 2023, kind of the game for mathematicians, especially if you were teaching a proofs course, was hey, let's find the error that AI made in its proof. And we'll laugh at the chat bot. And wasn't that kind of a funny and cute little thing to do, right? Um, that lasted about a semester. Now you can only really reliably get errors if you specifically prompt the AI to produce mistakes that students typically make, right? So the game has really shifted. Um, I think also trajectory-wise, more significantly, AI has just decimated graded homework, uh, you know, with take-home stuff. Uh as a result, myself, I've actually moved this semester to a standards-based grading. You know, students demonstrate mastery of specific skills rather than just kind of accumulate points, that little game that we have. Uh, the interesting thing is, this whole structure, if you've ever never tried this kind of standards-based grading, a lot of overhead. I couldn't have done all this overhead without the very technology that broke my old system, right? So the thing that broke my course actually has helped me to redesign it. So, you know, the trajectory for me has been instead of trying to work around AI, is actually to kind of uh you know more lean into it and try to use it to my advantage. And and for this semester, I really like where I've ended up with that.

Derek Bruff 8:18

Okay. Okay. Um, Amy, Kathryn, I want to turn to you to next, in part because I think you both teach online math courses and online education has been particularly impacted by AI. Um, what's what's that trajectory been for you over the last few years uh as AI has has entered our world?

Amy Langville 8:38

Similar to Lew, like there was the slow, what is this tool gonna do, and then realizing it just keeps getting better and better and better. And so you can't uh ignore it, but on the same, on the same hand, it it's uh related to prior technology. So like Kathryn and I've had many discussions with colleagues about tech tool use prior to AI. And so we were already a little bit ahead of the curve leading towards the let's encourage tech tool use so that we can solve more interesting applied problems with students in an online space. And I guess maybe one of the first things we did, we used that typical you see a lot of folks do red, yellow, green on particular assignments as whether it's prohibited or allowed. And and Kathryn's got a little bit found a little bit more nuanced one where we have for every assignment it's clear to the student use is allowed or prohibited. Um, and then we also with our uh textbooks that we use, we created our own thing for deconstruct calculus uh for our students, and it has become a really useful physical component of the class. So much of an online class is on the screen, and to have this physical book has been very helpful. And in having that, I think, also eased the transition into AI in some ways because the book itself, we we already focused on process over product, so every every page had answers on it, and you know, and maybe I'll let Kathryn jump in here and some more about the use of the book, maybe and how it helps with AI.

Kathryn Pedings-Behling 10:16

Yeah, so uh we Amy and I always love a new technology. Um we we love to jump in, we love to learn about things, and we we realized pretty early on that this was gonna be a new norm for the world. Um and so uh instead of putting our head in the sand, we really decided to figure out like, what does this mean for us? What does this mean for our classes? What does this mean for assessment of students? Uh, what does this mean for the expected knowledge that students have in a class and what um skills they need to be learning, and and how that can then apply to their own lives when they leave college. Um, so Amy and I in particular love to teach the business calculus class. These are the non-STEM major students. A lot of them are business majors, but we have quite a few psychology majors and um you know, we have some education majors, people like that in there. And so we really started thinking about how we can teach them to use the tool in a way that it's going to be helpful when they leave the classroom, right? And they may never do insert math topic here again, right? This is what they want to tell you, right? But we use math as an impetus for like how do we use these tools to help us gain knowledge when we need to learn new things.

Kathryn Pedings-Behling 11:52

Um, and so our textbook, again, our students really enjoy this physicality of the textbook. Um, and something that we have always done that AI cannot replicate is we have quite a few three-dimensional things that students are required to do with their textbook. They're writing in their textbook, they're tearing out pages of their textbook and making paper airplanes and throwing them and graphing functions based on that, right? They're drawing on their hands to think about how three-dimensional maps work, right? So there are these sort of physical features that AI cannot replicate at this point. And so we're constantly thinking about that with our curriculum, right? What can we do? What can we do with our students that can't be replicated by a chatbot at this point in time? Um, and so that's just a little, a little glimpse into sort of how we're using it and thinking about it in our classes.

Derek Bruff 12:49

Yeah, and one of the reasons I wanted to have the two of you back on the podcast, y'all were on a couple years ago to talk about um your very unique textbook, um, is that um I I hear from some faculty, uh mostly like English faculty who are who are moving the writing process back into the classroom. I had a I had a colleague on the podcast recently, Jim Seitz, who's do who's has all of his students, they do all their writing in class. Um and and there's lots of interesting things to say about that. But I know for online courses, there's not an in-class option. Um, but I love the kind of embodied elements of learning that you've built in um to that book. Um Chloe, what about you? What is the last three plus years uh in AI? How does that look like in your teaching?

Chloe Lewis 13:36

I don't usually consider myself like an early tech adopter. I'm still hanging on to like my iPhone 7 or something, right? But like I um sort of that fall when it first was released to the public, I was interested in it from a pedagogical standpoint. Um, and so actually, I think the first assignment I gave, I was teaching uh uh advanced linear algebra class as a graduate student at Michigan State at this point. And like one of the first kinds of exercises we had proved in that class was like, oh, prove that these two sets are equal. And so they had like written down a proof of these two sets are equal on their homework, and then I gave them two other sets, and I was like, prove these two sets are not equal. And they this was like homework one, they like submitted it. I gave them feedback, they got to resubmit it if they wanted, like that was just sort of the normal process.

Chloe Lewis 14:22

Um, in their final assignment, I I used to give a proof portfolio, which I don't assign anymore, um, because I think it's too easy to do with AI, but uh it was sort of a metacognitive reflection on like how do you think about proof? And then it was also some concrete, like include some proofs from this class. Um, and so as one of the sort of metacognitive reflective questions, I I asked ChatGPT to prove that these two sets are equal when they were not in fact equal, uh, and then asked the students to like assess the validity of that proof.

How do different students trust AI output differently?

Chloe Lewis 14:53

Um, that question has actually been sort of my like benchmark for the last three years. Like uh still when falsely a lot of times now, if you command it to prove something falsely, it will say, This statement is not true. Are you sure you want me to prove that? Um this particular question of are these two sets equal, when I when I ask it to prove that the two sets are equal, it still returns a false proof every time. It it has even on um one of my research students had um joined like the temporary pro plan um when it was like available to college students, and even that model still like wrote me a false proof of that statement. So I guess that was the first thing I'd done with it, and then I've sort of continued that same um exercise in a lot of my classes, not just in proof-based mathematics classes. So I teach a lot of our uh math for the little arts type classes here, uh, where these are students who not just are not math majors, but they are not STEM majors usually, they are art majors and music majors and things like this. Um and so we cover, you know, voting theory and graph theory and topics like this. Um I have in the past like asked Chat GPT to like produce me something that is false like related to voting theory. Make me a voting theory table that does this. Um, and so we'll do an exercise in class where we talk about can this happen, and then we go through and think about it. And then I share a QR code with them. And I'm like, all right, here's what ChatGPT said when I asked it to prove that what do you think? Um, and one I think distinction that I've noticed when I give this type of exercise to my math for liberal art students, as opposed to when I give a similar type of exercise to my math students where I say, here's a proof that I made with AI, what do you think about it? Um, is that the math real art students are much more likely to doubt their own answer, right? Like we had just talked about it, we just come up to the conclusion before I even introduce anything about AI that was like, here's here's how this works, here's what we think. But once they see the generated solution, they're like, Well, the computer can't be wrong. I had a student say that to me one time. Well, it's from the computer, and the computer can't be wrong, so we must be wrong. What did we do wrong? Whereas my math major students are so much more likely to like read through a proof and be like, this makes no sense. Uh, this is stupid. Like, why did it do this? Right. And so I I think that's a really interesting distinction that I've noticed in the way that students respond to AI generated output.

Derek Bruff 17:24

Well, do you have a conjecture as to why that would be? I I have some, but I'm curious what you've what you've thought about.

Chloe Lewis 17:30

I think it's twofold. I think one, the culture of mathematics, or at least the culture of mathematics that I try to like build in my mathematics classrooms. And again, these are more advanced math majors, right? They've had math under their belt for a couple of years. And um, is that like uh right? We we are a we are a classroom that like asks a lot of questions, right? So like I frequently like write something up on the board, I'm like, I don't know, do we think this is true? Right. And then we'll like have a discussion about whether or not we think that's true. So that's sort of like questioning like, does this work or does this not work? I think that is sort of inherent to the kinds of math proof classrooms that I built. It's also just a broader part of the mathematics discipline, right? As a professional mathematician, one of one function of your job is to, you know, you are refereeing a paper, right? And so your whole job is to say, is this proof valid or not? Like that's that's built into the job. Um so I think that's one part of it is it's just this explicit training in argument validation that we get in mathematics, that is maybe not the same kind of training one would get in a different like kind of degree program.

Chloe Lewis 18:39

Um, but I think the second theory I have is like the mathematical uh confidence in these students is very different. Usually the kinds of students I see in a math for liberal arts course are the kind who write to me at the beginning of the semester and say, I hate math, I've never been good at math, right? Like the the whole litany of everything. But um I think one that makes them more doubtful of their own mathematical ability, right? So even though we just came up with an argument for why this is true, that like just like is completely shot by the time they see something that a computer made.

Chloe Lewis 19:12

I also wonder if like there's some, I think in the public, there's some kind of base knowledge about like how do large language models work, right? Like there's some some sense that like, oh, these are statistical prediction models. I think that's like a relatively well understood phenomenon by most of the population. I think that what does that mean to be like a statistically predictive model? I think that has a significantly different meaning to a student who is like an upper level math major, right? They understand statistics and probability at a different kind of level than a student who is, you know, a self-proclaimed, oh, I don't like math kind of person. Like I think that underlying understanding of how these models work is also a factor.

Derek Bruff 19:52

Yeah, yeah. Um I want to throw this open to to the rest of you. Do you see similar um similar situations where like some students are kind of over trusting the AI or or more skeptical? What does that look like in your in your courses, Lew?

Lew Ludwig 20:12

Um so I'm doing the standards-based grade thing, right? And uh I do a lot of uh actually true-false with this. It's it's kind of an interesting system, but there's a STEM question that has like maybe here's a linear system, and then I ask them a series of questions about that specific thing. And anyway, we were talking about uh differential equations, right? Linear differential equations and uh whether things would have the Wronskian or whatever. I had a kid roll in, this is a sophomore level class, and I you know, my true-false question, he goes, but isn't it in this context, isn't this true due to Abel's theorem? And I looked at him because my AI also pointed out in this very weird thing at the 600-level course, right, that you might invoke Abel's theorem, but this wouldn't fit in an undergraduate thing. So I impressed him on him, like, where'd you get that? He kind of like, you know, got it pulled back. So, you know, it it's it's interesting how the students, I think, sometimes blindly just accept what's going on. This was a computer science student taking a linear algebra differential equations course, but uh, you know, they were just uh, you know, bound to determine again. There's kind of that mindset, I want to get this right, I need this problem to get my meet my expectations and the whole nine yards. But it's just like, you know, he thought as long as I just throw out the word Abel's theorem, that that will be good enough, and that will kind of give me the passing grade that I need.

Derek Bruff 21:23

So Chloe, it sounds like part of your goal there is to help your students understand AI's strengths and limitations a little bit. Is that is that fair to say?

Chloe Lewis 21:35

I very much see it as like part of my job in in like working through these exercises in class is to uh sow doubt in the like complete and utter truth that is produced by a large language model, right? Like that I think that's really important, especially for the students, like I said, who are taking this math for liberal arts class and are maybe not as comfortable with mathematics.

Derek Bruff 21:57

Okay. Um, because I think you're also trying to give them a chance to practice their their proof skills, right? It's it's kind of serving both goals, right? A kind of math learning objective, but then also uh kind of an AI literacy, AI know-how piece. Um, Amy and Kathryn, do you do you also see AI literacy as kind of part of what you're trying to teach now in your courses?

Kathryn Pedings-Behling 22:20

So I I will say something that I think um I think it's not 100% related just to AI literacy. So what Chloe was talking about is a phenomenon I have seen with my non STEM major students, the whole time I've taught post-secondary. So I used to teach secondary, I used to teach high school level. The whole time I've taught post-secondary, it takes me almost until the second half of the semester for any non-STEM major student to correct me if I have made a mistake in class. And most of the time it's just because they tr they blindly trust me, right? They're like, well, I'm I'm not a math person. So like if she's saying this is positive five instead of negative five, then that's gotta be what it is, right? Even though inside they're like, wait a, wait a minute, is that right? I don't know if that's right. I spend almost half my class, the first half of the semester of my um, I also teach a contemporary math class, which is like math for the liberal arts, um, business calculus class. I spend almost the first half saying, you have you belong here, right? You have a place here. Your voice matters, what you think matters. You can think about mathematics, like you have that ability. And so I think I don't know that I don't personally know that this is any more blind trusting of a large language model as it was a uh blind trusting of Kathryn Pedings-Behling standing in front of them, right, as their math teacher, right? So I feel like I'm seeing the same stuff. It's the to me, I I I don't really see a difference between um them questioning something that's coming out of our, you know, AI agent versus them questioning something that's coming out of my mouth, right? It doesn't happen either way for those non-STEM major students. I want to stress.

Derek Bruff 24:19

They doubt themselves. They doubt themselves. And so if if someone appears to have some expertise or authority, then they're just gonna go with that. Yeah. Um, and that could be their instructor or a confident sounding chat bot, either one.

Kathryn Pedings-Behling 24:31

Yes.

Derek Bruff 24:31

Yeah. Yeah. Um, Chloe, I'm gonna circle back to you again. Uh, your your example is really interesting. Um uh so you show your students an output that ChatGPT has generated. You do not have the students go and kind of take that prompt and take it to Chat GPT themselves. And so I'm wondering how you think about um your students using AI and and like your role in encouraging them or discouraging them from using AI. I'm guessing that's an intentional choice on your part to kind of do it for them instead of have them use the tool.

Chloe Lewis 25:08

Uh yeah, I don't want anybody to be required to register with any particular uh AI, like to get an account on any particular AI website in order to participate in the learning activities that we're doing in class. So that has been the main motivation for generating uh all of the things myself and then sharing it with them. I mean, ChatGPT makes it really easy to like share a link out to a chat and things like that. We do, I mean, like all institutions or most institutions, I guess I should say now, we do have like uh institutional access to Microsoft Copilot. Um but again, I I've not had this conversation with the student, but I've had this conversation with several other just people who have ethical and environmental objections to the use of AI. Right. So I I don't feel like I want to put anybody in a particular position where they would have to make such a choice. Now, are most of them probably already using it? Do they already have their own Chat GPT account? Yeah, like I I I know that, right? But I I do not want anybody to be forced to use something that they do not want to if uh like in order to like participate in the activities we are doing in class.

Derek Bruff 26:15

Sure. And it sounds like for your goals, they don't need to either. So Right. Yeah.

How do you help students use AI chatbots for learning?

Chloe Lewis 26:19

Exactly. Um I actually I wanted to like come back around to something that Kathryn said and bring in uh a particular uh reference to the literature um that I think is supporting what her point is. Uh there is some work of Harel and Sowder in the 1990s about um student proof schemes. So they sort of diagram like uh the things that like when someone is sitting there reading or watching someone prove something, like uh how do you how do you validate that proof? Like, what is your sort of thinking process in order to determine do I think this is a valid proof or not? Um and they studied the appearance of different types of proof schemes in different contexts and different classes of students. Um, and one particular proof scheme that they found uh showed up in a lot of students was an authoritative proof scheme, right? Which is exactly what Kathryn was just talking about, the idea that like this thing has to be true because my teacher said it, because it is printed in a textbook. I think that is another, right? It is like authority broadly construed. And so uh Harel and Sowder sort of argue that these proof schemes kind of sit in a hierarchy of like, you know, uh what is I don't know, what is like the not good kind of mathematician do versus like what do you what is the robustly thinking mathematician do? And sure authoritative proof schemes sit relatively low on that totem pole.

Derek Bruff 27:41

Um but I think more novice notion of proof, yeah. Correct.

Chloe Lewis 27:46

Yeah, but I just I wanted to add that uh to support her point that I I that this is one of the things that that we continue to observe from Harel and Sowder's work.

Derek Bruff 27:54

Yeah, yeah. What about you, Lew? Do you well how do you feel about your students using AI? Do you encourage that? Do you discourage that?

Lew Ludwig 28:04

I assume that as soon as they walk out the door, they have access to AI, right? I mean, there's just there's just really no way of getting around it. I know uh Leon Furze is the other uh folks that we're referring to, the you know, the red, green light and all that kind of stuff. Uh Leon, he he's first developed that and he's kind of modified, he's got some like bubblegum-colored things now. But but he's uh thinking about that more kind of as an assessment way of how you might assess students. But again, he kind of believes also that once you step out, it's just naive to think that they wouldn't have access to that.

Lew Ludwig 28:33

So that's why I've really kind of directed things in a way to think with my students about how am I going to get to think about what they actually know and understand and can kind of demonstrate for me as opposed to just producing this product. Because it used to be you produce this product, that was your homework, that was your thinking. AI has kind of thrown all that out the window. So that's why I really moved towards this uh this standards-based grading to try to kind of eliminate that. Now, they know full on. I mean, I'm very heavily involved with AI. I talk about a lot. A lot of my things at the bottom will have a little AI disclosure about how I use it to modify or you know, make formatting changes or whatever the case may be. I talk to them about creating prompts for practice questions and stuff like that.

Lew Ludwig 29:14

But we also talk about, you know, at the end of the day, they need to do the hard part of learning. They need to do the you know, the friction that's involved. Because otherwise, I mean, I have a lot of athletes, right? You know, the old analogy of just, you know, let's just have the forklift lift the uh heavy weights for us. They know that there's not going to be any benefit to that. Uh the nice thing is, I think the uh the standard space grading, because they don't have to get it right the first time, gives them a little bit of a safety net. And they're for once in my I mean, I've been teaching for a long time, man, 30 years plus.

Derek Bruff 29:42

You said Lotus Notes. So that that aged you a little.

Lew Ludwig 29:45

Right. Yeah. So yeah, exactly. So they but they're actually learning from their mistakes, right? So they they have to get like eight out of 10 or better, and they get a seven, and they don't just say, oh, it's a seven out of ten. They look at that 30% and they focus on it for next time to do better, right? Uh I never would have done that without AI messing everything up. So it's kind of interesting. I'm I'm using AI to redevelop my whole my class, but I'm still kind of, I guess, teaching in an old school, not really analog, but you know, kind of in an old school way as far as uh getting students to learn and think.

Derek Bruff 30:17

So I'm gonna go back to Amy and Kathryn, um, because I I I know you do have your students use a particular AI tool that you two have developed. Um, can you tell us a little bit about that?

Amy Langville 30:28

Kathryn took a class, I think it might have even been loose class on how to uh make your own custom GPT, basically. And so we have um ChatGPT with uh custom instructions, and we fed it the knowledge base because we already had so many hundreds of pages of our deconstruct calculus book. It was easy to just feed that as a PDF into the knowledge base, say only use this and give it some instructions. And so we basically have a 24-hour tutor for our students that is customized to our book, to our language, to our notation. And um, then Kathryn could talk about some of the student uh reactions to that, and we've started to collect data on custom versus general GPT, interacting with these as a tutor. Uh, we've gotten some really great feedback.

Derek Bruff 31:21

Kathryn?

Kathryn Pedings-Behling 31:22

We at the College of Charleston, so we do not provide a license for open AI, ChatGPT. They have a license for Microsoft Copilot. Um, I'll leave my thoughts on that to myself. And um but like Lew said, the assumption is they basically leave our classroom and they're using some sort of AI. And um I don't want to get into this whole ethical thing, but this is what if I ever have students talk to me about it, my whole thing is that uh the college provides Microsoft Office 365, which sits in a data center that we require our students to use, right? And we don't have a problem with that. There's nobody talking about ethical implications of our students all having a Microsoft Office 365 account. But then there's all this conversation about ethical implications of them using Chat GPT. And so that anyway, my whole point is I can have a conversation all day long about um reducing our cloud usage as a whole. I can have a conversation about that all day, but I have a lot of trouble pointing the finger at AI, which is why we decided to dive in on this hole.

Kathryn Pedings-Behling 32:40

So um I started doing a lot of thinking after this MAA workshop about AI agents that have a reduced um knowledge base, right? So we don't let our AI agent access the web for answers. Okay, so it has the knowledge base. We have our book, we have our syllabus uploaded there, assessments, et cetera. Um is that enough for our students to be able to ask it any questions that it has versus them just going to chatgbt.com, open, you know, not an agent, right? Um, and asking it questions, right? So how do they interact with those two things? So I pulled a couple of tools together, um, and we have some survey and some open-ended questions. Um, and so at the midterm in the final exam, we invite them uh to study with the AIs, right? So the the AI agent that's specific to their class and then general chat GPT. So general chat GPT. And we give them a few, sorry, go ahead. Please go ahead.

Derek Bruff 33:51

Yeah, let me jump in just to clarify. So I think I'm hearing two attributes of your agent that are potentially relevant to what you're about to say. One is that it is uh essentially trained, it has access to your course materials through your textbook. And I think it's also been prompted to behave more like what Anna Mills calls an ethical tutor, someone who's not just giving away all the answers to students, right? Which off-the-shelf chat GPT does not have that quality.

Kathryn Pedings-Behling 34:18

For sure. A hundred percent. Yes, absolutely. Yes, we um we had played around, Amy and I had both played around with Khanmigo some before we did this, right? And this idea of the Socratic type um model. And I did do some playing around with bullying our agent AI and figuring out how long it would take it before it would give them the answer and all this kind of stuff. And I I tried to help train it in ways to make that last longer, right? To make it so you had to bully it longer before it gave you the answer. So we give them some prompts to start off with, right? Pick a topic that's gonna be on the midterm and ask the AI agent to make you flashcards for it, whatever, right? Ask it to make you a multiple choice quiz, 10 questions. Ask it about one of those questions on the quiz that you didn't understand and ask it to go through, right? So we give them some prompts. And the first time we did this, which was fall midterm, this past fall 2025, I had students being like, I literally had no idea that that AI could be used in this way. Like I had no, they they said, I will never study the same way again, like for a test, period. Like I had no clue that I could do this.

Amy Langville 35:31

Um which I think anyway, yeah, go ahead, Amy. I just think that's particularly important in our online class, right? And face-to-face or students have access to the math lab. These are distance ed students, and to be able to have this 24-hour access to a tutor that speaks the same language and and they understand it's a conversation. It's not just a way to go grab an answer and cheat, right? It's to have a conversation and to help me prepare for this midterm and final exam. So they were using it because of the coaching that we've provided. We, you know, there's it's hard to check on some of this, but they were using it in a constructive way, right? So we coached them on how to use it.

Kathryn Pedings-Behling 36:09

Yeah. Um, yeah, I don't know that I had uh any more to say about that, but we were collecting data again this semester. We're excited this summer to analyze that um to figure out sort of uh what they liked about each one. They have to answer things that they liked or didn't like about each one that they've studied with. Um and I I mean, I personally think it's should it's not really a surprise, but I personally think agent tech AI is where we're going, right? Like everybody's got an AI chatbot somewhere.

Amy Langville 36:40

And I think I think the neat thing about that is for me to witness, you know, Kathryn went to Lew's class, but then to see um how easy it is to make one for every single class that you have. Like it's it Lew, Lew had this set up, and it's like, oh, you make the custom instructions. We're luckily had our own you know textbook material, but not every movie has that. But there's it there's certain facets of your class that you can make. And Kathryn did some really cool ones for an independent study student. It was a student class of once, just one student in a class that you know it was able to sort of mimic and virtual classmates and have um conversations as if you were it was a um math ed student that was going to be teaching and mimic conversations that you might have with high school parents. And so you can do these things rather easily with not much startup, not much know-how about coding. Like that's I mean, we was talking about Lotus 123 . Imagine if you had to do something like that 30 years ago now as an instructor, it's so easy to make these tools.

Derek Bruff 37:42

Yeah. Chloe, I'm curious what you make of this idea of providing students with a kind of AI-powered tutoring resource. What what what did you just hear that stood out in your mind as as interesting or useful or maybe concerning?

Chloe Lewis 38:02

Um I haven't like uh set up any sort of particular tools for my students use. Like I said, I mostly sort of bring them something already generated. Um but I do have a lot of conversations with my students, especially I'm teaching abstract algebra this semester, um, with actually like a returning cohort of students that I'd already taught in geometry. So it's like small class, nice size. I know these students pretty well. Um and uh all of my proofs classes are very like you try something once, I give you a lot of feedback on it, and then you try it again. Uh not quite a standards-based grading, but like a specs grading kind of situation, I would say more. Um and in a lot of those conversations, I guess my I'm just always sort of curious. I'm like, Do we use AI? And my students themselves are actually quite negative um on how well it does explaining mathematical ideas. Again, these are abstract algebra students, so I might get a different response if I asked my calculus student something about this, right? Um, but I I was talking to the student recently, uh, they're like working on some exam revisions right now. And uh like before I'd even give them their test back, she was like, she's like, I couldn't figure this one out on the test, and I got here, here, and here. And she's like, and I went and asked, you know, ChatGPT or Claude which everyone, she was like, the answer it gave me was like really bad. I was like, she's like, it didn't, it didn't make sense. And we sort of looked at it together and like flicked it into Claude, and it was like it wasn't all that great. Um I I think there's like more promise in this sort of self-contained ecosystem. Uh I I have a colleague at uh UW Stevens Point, uh, Grant Kopitzke, who uh also teaches a lot of distance learning, hybrid sort of things. Um, and he like one of his assignments for his calculus class is like uh they like have to have a conversation with his particular uh calculus tutor, and like they have to like send the log of that conversation to him. This is one of his assignments, and he really enjoys that. Um so I I think those use cases are intriguing.

How good is generative AI at doing mathematics?

Chloe Lewis 40:08

Um I also do like encounter quite often, I think, especially in my math for liberal arts class, I have a lot of English majors, not all of them, but I have a lot of English majors who take those sort of who feel very negatively about the future of generative AI as a whole, right? And so uh when I like because I have them uh one one assignment in that class is we read uh Cathy O'Neil's Weapons of Math Destruction, uh, and then they do a journal assignment in response to that, uh, which is a really nice book, and it's got a great interchapter that is really accessible to a lot of people. Um, but that is I sort of ask them to reflect on the role of, you know, did they see a weapon of math destruction in their own life? And I get a lot of papers from English students that are like, what what is my future gonna be if if all of this exists? And it feels uh insulting to or just incredibly negative, like sort of perspective on that. So I like I do think there are students out there who who don't it like aren't using AI tools in this way. Um whether that's like good for their employment prospects or whatnot later on, I'm not so sure. But I but I do think this is an attitude that does exist among the population of our students.

Lew Ludwig 41:24

Yeah, I just want to, as a uh innocent bystander, I just want to point out there's uh two different things going on. So our group down in the Carolinas, they're using ChatGPT, but what's it called? The corporate account, what's it called? Um what is it?

Kathryn Pedings-Behling 41:37

Are you talking about teams?

Lew Ludwig 41:39

Teams, yeah. So what they're using, they're using it's a pro model. So this is a deep thinking model. This thing crushes stuff. Gazam and I, when we ran the workshop, even last semester, she's like, yes, it will do all of undergraduate mathematics. So you have to be careful when you say AI can't do things. Free model, I always, when I give my talks, I usually put up like a three-speed Schwinn bicycle from the 1960s. That's the free model. The paid-for model is usually Doc Brown's DeLorean from Back to the Future. Very fast, we'll take you, but dangerous, right? But but again, the thing is these these thinking models, um Abel's theorem, again, there's hardly anything that the thinking models that the paid for $20 models cannot do. So you're hearing slightly two different conversations going on. Sometimes it's making stuff up, sometimes just keep in mind that we're talking about totally two different models of what's happening. So just kind of point that out.

Derek Bruff 42:30

Hey, this is Future Derek jumping in with a recommended reading on Lew's Point. Just last week, historian Mark Humphries wrote a post called The Agents Are Waking Up that made a similar point. If you're basing your estimation of generative AI's capabilities on free chatbots, you're missing out on a lot of the power of these tools. It's a good read that gives a sense of where AI is going in the near future, and it's a little scary. See the show notes for a link.

Derek Bruff 42:57

Amy?

Amy Langville 42:58

I'd like to jump in on that too. Um, Lew, I agree that even further, uh besides just the free models, there are those explore custom GPTs that users contribute or companies contribute. So like the Wolfram one, I think, is amazing. The Wolfram GPT and the MATLAB one now, like, you know, the things that they can do. And so I just want to come back to Lew's comment before how it just decimated he had to totally revamp his class. I'm at that position. I haven't taught a class, a graduate class that I haven't done in a couple years. And now with these tools, especially with the MATLAB plus GPT, I feel like I need to totally redo my graduate class and start totally over. And but Lew's right, there's opportunity, right? That's that's something new. I've been teaching a long time. This is now fun and interesting. What am I gonna do? How and I come back to some of Chloe's things. What is it we're trying to get students to do? And I keep hearing over and over in hers is like verification and validation. And that was prior to AI, right? Because we talked about there are other tech tools, or there is the authoritative expert that, and so those skills become all the more heightened. Like we really have to get students good at doing those, verifying and validating. Because the tools aren't going anywhere. I do think they're just going to become more and more ingrained, and it's hard to not use them. Like you try you can't even do a regular search without a Google search anymore without getting AI response.

Derek Bruff 44:19

A couple of years ago on the podcast I had on Brett Becker, who's a computer science educator. Um, and the the AI tools were pretty good at coding before they got pretty good at math. Um, and so CS has had to deal with this for a while. And and he argued that hey, it may be that. Our first year computer science curriculum is more about um verifying that programs do what we want them to do than actually coding them ourselves. And and he said, Look, this is all this is a useful skill already, right? Like if you're a if you're a software engineer, you've got to read other people's code and make sure it does what you want it to do. And so we just may get to that kind of uh earlier in the process um uh of CS education.

How do you use AI in your course prep?

Kathryn Pedings-Behling 44:59

Um Yeah, and I want to say, if I could, can I just piggyback on that really quickly? The College of Charleston has a newer major called Computing in the Arts, where they're focused on like the front-end design of things. And these students used to stumble quite a bit over having to design the back end before they could do all the front-end art stuff that they were doing. And I had a student in my business class who she and I were talking, and she said, AI has absolutely changed the way I'm able to do what I want to do with my major. She's like, I'm no longer up all night trying to figure out how to code the back end of these things. And I get to focus on the artistic piece, right? On the front end of all of this. That's what I want to be doing with my major, right? And I just think that's a great example of um of what it can do, right? What it can, what cognitive demand it can free up for our students who are in these artistic um type majors.

Derek Bruff 45:58

So Lew, I want to go back to you. Um, you mentioned using, speaking of getting a little AI-powered assist in your professional work, you mentioned using AI to help you implement standards-based grading. Um, I know you talk a lot about ways we've we focus mostly on student use of AI, and I think that's a great, a great place to be. But but I know you talk a lot about how instructors can use AI in their work. Are there are there a few kind of strategies or ideas that you find particularly useful to share with math educators who are maybe interested in using some AI in their their course prep?

Lew Ludwig 46:35

Oh my God. So the course prep. So, you know, I'm looking at Amy when she's going to do that class. She's going to have an AI graduate student, right, Amy, over your shoulder, like talking back and forth with this and pushing back and forth. The thing that our math colleagues have to realize is that they've done a lot of hard work over the years and they become experts in their field, right? They're experts about their content, their discipline. They know their students very well and they know the context as well. They can then take that and leverage that while using AI, right?

Lew Ludwig 47:06

So I want to redesign my course. So, okay, um, I'm teaching Calc 2, right? And um okay, I know that students are gonna mess up some of the sequence tests and they're gonna, you know, what are they gonna do? Uh they're gonna or sorry, the convergence test, right? So they're gonna like um they're gonna confuse two different tests going on. So I can ask AI, say, hey, give me a series where students will be tempted, maybe you like to use the ratio test, but the comparison test would actually be the better move, right? So as an expert, you know your students will struggle with that and you can ask that question. Then you can follow up with and also give me two follow-up uh questions that will actually build distinction for my students. So now you're kind of channeling the AI. I'm not having my students use it. I mean that you could, right? But I'm having them, you know, AI is giving me a better question that I can ask for my students because I'm tapping into that expertise that I've seen them struggle with for over the years, right?

Lew Ludwig 47:59

Um my son is teaching AP stats as a 25-year-old. It's like, holy cow, how's he doing that? Because he leans into the AI and he says, hey, explain this to me. You know, I know they're gonna, there's linear regression, they're not gonna understand this. Give me three different ways to explain that, right? And then he can go through and filter out which one works best for him and then help to explain that to the to his students. So I think, you know, math colleagues, we have to understand that we are a bit in the driver's seat with AI. If we ask it to do something, if it gives us the wrong answer, I mean, if a student gave you a wrong answer, you wouldn't say, Oh, stupid student. You say, Oh, let me ask that in a slightly different way. So use your power as a mathematician, use your power as an educator to kind of rechannel that energy. And when you're working with the AI, treat it like a student. If it didn't give you the right answer, oh, you know what? Let me ask that in a slightly different way. Let me kind of move that in a different direction. Um, and man, you can just, as an expert, you can just make this AI stuff sing for you, right? It does some really, really strong stuff.

Lew Ludwig 48:54

It like I said, it's it's re it's revamped my whole course, or I should say, let me rephrase that. I've revamped my whole course with it helping me, you know, kind of provide the structure of what's going on. But it's just been a blast doing that. Uh now again, I I, you know, I like I said, I don't know if it's chickened out as the right word, but I I have pulled things back in the classroom. I'm not doing as much um deliberate homework, but I have these notes, right? I create videos and I do a little stuff with things. I have 61 students, my video notes, I'm getting like 80 hits. Okay. So either I have some psychotic student who's watching it 50 times, which I don't think is the case, but they're actually learning, they're looking at the notes because they know they will be tested on this material. And again, getting back to that standards-based grading, they're trying to focus on the stuff that they missed. So if they didn't get it the first time, that's why I'm getting more hits on a view than you know, just the minimal of 61 or something like that. It's really amazing. And I'm not even, I'm not giving any grades for this. I'm not assigning it, right? They're actually doing the work for once. I'm I'm stunned.

What do you wish your math colleagues knew about AI?

Derek Bruff 49:52

So I um I keep referencing to old podcast episodes, but it's one reason I do this is to see these connections. I had uh Sharona Krinsky and Robert Bosley on a couple of episodes ago who who run the grading conference and the Center for Grading Reform. Um, and they said uh alternative grading practices just keep coming up, right? We didn't come here today to talk about alternative grading practices, but like like it's one very sensible response to the advent of AI. And so I hear it a lot in the guests that I have on my podcast, even if that's not the ostensible topic. And so a little more proof of their their theory today.

Derek Bruff 50:28

Um, we are almost out of time, but I want to do um, let's do a round robin with one last question. What do you wish more of your math colleagues knew about generative AI's impact on teaching and learning?

Chloe Lewis 50:42

I will start. Um I uh part of what I've been working on has been uh a research project with two of my undergraduate students, Ellah Olson and Madison Schwartz. They're both uh mathematics education majors. Um and we have been um testing how uh students respond differently to proofs when we tell them this has been generated by AAI versus when there's no mention of the author. Like, are you more or less likely to believe that? Um, so we've been working on this research project for the last couple of years, and um, as part of that, we just ended up prompting the AI to write and prove a lot of different mathematical theorems. So um in doing so, we just were like, well, we kind of know a lot about what AI does with math. And so we've we're talking at our um department meeting, the three of us. Um, and I think the revelation of like you know, like it can do your entire take-home exam, like just full stop. I I do think that was surprising to some people. And so um for people who have like maybe really never tried those things, like I think they'd be shocked by like I said, it can it can do it all. Well, it could give you an answer for everything, right?

Derek Bruff 51:52

So, and especially if you if you last checked in in 2023 or 2024, when when the tools were were often really bad at math, um, that has changed. Yeah. Kathryn?

Kathryn Pedings-Behling 52:05

Um, so I would like to stop having people ask me the question, how do I keep my students from cheating with AI? That is the constant question people are asking. How do I keep my students from cheating? How do I keep them from cheating? I wish we had so many more people in our department who thought like Lew thought, where it's like, this is an opportunity for me to completely revamp the way that I'm teaching mathematics, right? And I have that opportunity, right? Um so that's I don't I don't know. That's that's what I'll say. And I don't know if Amy's gonna bring up from Solve Khan's book, but we we like, we have really linked on to this idea of um now that AI is here, you can expect more of your students. You can ask them to do more harder things because they have these tools. And we're really taking advantage of that in our classes.

Derek Bruff 52:59

Amy, what would you add to that?

Amy Langville 53:01

Um I've heard some people say, well, AI only knows what's already known. And so what the human can, but I I don't know that that's necessarily I think it has elements of incremental creativity. I don't think it'll ever do paradigm shifting creativity, but but so then that leaves like what is left for the human. And uh if you have this encyclopedic resource, and Kathryn was saying, yeah, we can now ask grander assignments, they can expect more because you have leveraged this amazing assistant. Um but I also come back to okay, well then we need to foster creativity. And what exactly? That's such a vague word, and what does that really mean? And what does that mean in mathematics? And so like these broader discussions about what other things could we focus on? Um and I come back to Conrad Wolfram's four steps of computer-based math, and at one point he was like, oh, in step three, we're focusing on the computation step, and that's the thing that computers are good at. Well, now it's like computers are actually getting better at even the first step, and sometimes the second, and certainly the fourth. So so what is, you know, again, what is left then? What what we need to rethink what we're doing.

Derek Bruff 54:04

I don't know if that's exciting or scary. Um of both. Lew, what do you wish more of our math colleagues knew about AI and teaching and learning?

Lew Ludwig 54:13

Uh I'm gonna piggyback a little bit on what Kat or what Amy said about, you know, kind of getting involved with it. Um, I I had a thing earlier today, and I was kind of thinking about this. Uh for those of us who've had young children, right, and you try to get them to uh eat something new like broccoli, right? You can't, you know, I'm not gonna have my kid decide whether they're gonna eat broccoli by listening to some other kid. They have to try it themselves, right? But moreover, it's one and done is not the deal, right? In order to try something new and kind of get used to the flavors and all that kind of stuff, unless you have that weird gene for broccoli, uh, you know, it takes like five or six times. So, you know, I think for for the math colleagues who have tried broccoli a long time ago, or maybe just listen to somebody else complain about broccoli, yeah, it's probably time to pony up and try the broccoli again. Uh again, I would encourage, you know, if you do have the means to do it, uh, to poke around with one of the paid models or the thinking models because it will really surprise you.

Outro

Lew Ludwig 55:07

But again, kind of spinning it from, oh, what are our students, you know, with Kathryn saying about our students cheating about how could I leverage this? How could I make myself into, you know, there we've had a lot of issues with with education. It wasn't like, you know, things were running along fine and all of a sudden AI hit and then things were bad, right? There was a lot of stuff going on. And I think AI just has kind of amplified that. But I do think we can kind of use, we can kind of channel this negative thing that it does do on the on for ourselves and kind of actually turn it into something positive to help ourselves and help our students better with their learning if we you know use it appropriately and and kind of you know do all the stuff that we already knew was good. We now actually have to pay attention to those, uh, you know, how students learn and cognitive science and all that stuff.

Derek Bruff 55:49

Yeah. Well, um, that's a good place to leave it. I know there's lots more in this topic we could explore together, but um our time is nigh. Thank you so much for being here and being part of this panel. I really appreciate it. Um yeah, thanks so much.

Lew Ludwig 56:02

Thank you.

Amy Langville 56:04

Thank you all. Nice to meet you guys.

Chloe Lewis 56:05

Thank you.

Derek Bruff 56:08

Thank you to Amy Langville, Chloe Lewis, Lew Ludwig, and Kathryn Pedings Behling for coming on the show and sharing their current AI-aware teaching practices. In the show notes, you'll find links to Lew's new book, The Science of Learning Meets AI, and to Amy and Kathryn's Deconstruct Calculus Project, and to Chloe's website, and to all the books, articles, and podcast episodes we mentioned during our conversation. And if you're a math educator, I highly recommend joining the Teaching Mathematics with Generative AI Google Group. That's where some of the best discussion of AI in math education is happening right now. See the show notes for a link to that group too.

Derek Bruff 56:49

Intentional Teaching is sponsored by UPCEA, the Online and Professional Education Association. In the show notes, you'll find a link to the UPCEA website where you can find out about their research, networking opportunities, and professional development offerings. This episode of Intentional Teaching was produced and edited by me, Derek Bruff. See the show notes for links to my website and socials and to the Intentional Teaching newsletter, which goes out most weeks on Thursday or Friday. If you found this or any episode of Intentional Teaching useful, would you consider sharing it with a colleague? That would mean a lot. As always, thanks for listening.