Podcast Season 2, episode 10: How Dr. Richard Charles, DPS' top technologist is thinking about AI in schools

Transcript

Alexis Menocal Harrigan 0:08

Hi everyone and welcome back to the Board Hawk podcast. We recently had our friend Antonio Vi Hill on from Aurora Public Schools to discuss what his thoughts are on AI and education. A very hot topic right now. We are likely going to be doing a few more interviews with folks in the field who are leaders in this space, and I'm so excited. Welcome on Dr. Richard Charles from Denver Public Schools to the podcast today. Dr. Charles is a national thought leader in the effective adoption of ai of technology in schools, excuse me, including ai and has been thinking about machine learning and generative ai well before chatbots have revolutionized how we're engaging with technology. I'm sure I could read a much, much longer bio, but instead we just wanna welcome Dr. Charles on and then have him say a little bit more about himself and how he, kind of came, came into this world around becoming the chief information Officer. Welcome Dr. Charles.

Dr. Richard Charles 1:04

Thank you so much, Alexis and Alan. Nice to meet you again. So a, a bit of my background I was a math nerd in college and came through that pathway and so one of my experiences prior to graduating with a degree in, in pure mathematics was an experience during summer. At the University of Rochester where I got a chance to really explore things that were passions of mine. So things like astrophysics. And while I was doing that. I asked my research advisor what are some of the other topics that I can pursue maybe in the area of computer science. And he mentioned, well, the leading computer vision group in the world is actually here at the University of Rochester. So that was Dana Ballad and his team at the time. And so I got a chance to sit in with that team and learn all about computer vision. I asked the question, so what do I need to do in order to have computers be able to view things artificially as if they were a human being in ai? And they said, you have to know a lot of math. And because of that, I pursued a master's degree in applied mathematics from cu Boulder and a PhD in Applied Mathematics from CU Boulder as well. And so when I came to Colorado. I was hired at the time with a company called US West Advanced Technologies in Boulder. It's supposed, it was supposed to be the Bell Labs of the region. And so did some research topics there, did some really exciting work there and throughout my career, have always. Focused on AI types of technologies, and so much of it is anchored in mathematics. I felt that math gave me a real opportunity to, to really capture the sense of what's going on right now. And although there are limitations in interpreting some of these large language models, it, it really does give you a solid foundation. So I'm excited to be here. Thank you for the invite.

Alan Gottlieb 3:23

And Dr. Charles, I'm just curious what brought you from that world into the world of public K 12 education?

Dr. Richard Charles 3:30

That would be my wife. She, I married into an education family. Parents were longtime educators, assistant superintendents, brothers are also PhDs and, and in the central office. And doing quite well. And then of course, the running joke in the families that the pharmacist is the one who prescribes all the medication for the, for the educators in the family. And she has a sister who is a pharm d as well. So, yeah.

Alexis Menocal Harrigan 4:02

Okay. I love that. You, you mentioned at the beginning of sort of your academic journey astrophysics. So did you ever have intentions of going down that route? And and I'm really curious to see if you've, if you've watched, here if you brought Practic Hail Mary yet so good.

Dr. Richard Charles 4:19

Yes. I, I, I can't get enough of, of what's happening right now with the space program. I did have a passion for it to the point where I, I did pursue it and was actually in the process in 2012 and ended up being a highly qualified astronaut candidate. Wow. So very exciting work happening now in NASA and. Just hoping that we can continue to pursue some of the exciting next phases of the work that, that they have just given us a glimpse into. And very exciting. Those photos are just amazing. I'm sure you all agree

Alan Gottlieb 4:56

that's, yeah. As long as we get the anti-science morons out of our federal government, we'll be in good shape. And that's my editorializing. I'm not asking you to respond. Got it.

Alexis Menocal Harrigan 5:08

Alright, well maybe I'll start with the first question here. So, you know, we know students are using ai, but not all teachers are necessarily teaching AI literacy or showing students how to use AI for learning. What do you see, I'll start with, with a broader question. What do you see as our responsibility in this space, whether you as an administrator, as an educator or classroom educators to, really show up with students on how to teach and use AI responsibly in the classroom.

Dr. Richard Charles 5:42

Yeah, so Alexis, I apologize for you probably have heard this over and over again every time you've been in one of my talks. But what I start off with is that the kindergartners today are the graduate in class of 2037, and we as educators have a responsibility to prepare them for a world that we know absolutely nothing about. And we have no idea of what the tools will look like by the time they become adults. So our best guess is to anticipate and to dream what is impossible with this current technology that we have, so that we can prepare students for that world. It is a huge responsibility, but the good news is that we've done this before. The challenge is, and Thomas Friedman, we were talking about him before this mentions this, the pace at which change is occurring has never been. Faster than what we're experiencing today. So when we look at technology from the time a student enters into kindergarten and 13 years later when they graduate from college or from high school, that shift in technology is now. Drastic. Very, very drastic. And so our responsibility is to give kids some of these tools. Make sure that their guardrails ar around them so that they can operate safely and know how to use it. Because ultimately they will be in professions that use many of these tools. Many of them will be obsolete by the time. They get to their careers. So we have to be thoughtful about that. I also think it's important to always keep the human element in all of this work, right? So one of the, the concepts that I learned early on when I made the shift to the field of education was this whole notion of who's doing the thinking, is doing the learning. And right now. There's been a lot of buzz in the news around folks students not thinking because they're using AI and they're submitting it to teachers, and teachers are using ai, AI then to evaluate the work. And so I. If we use that model of who's doing the thinking is doing the learning, then AI is getting really smart, right? Because we are circumventing some of the, the hard work that needs to be done in the learning process. And so cognitive psychologists are looking at this and it's really important for us to understand that many of these tools that we are now experiencing and, and have access to. Have biases in them because of the data that they're trained on. So how do we manage all of those pieces? And so we're very thoughtful at DPS about making sure that we're not giving students tools that are that can violate their privacy in some way, or give them advice and things of that nature. So we stay out of that role. We think that these are tools that should be used for their studying study plans, study schedules things like really engaging with career pathway development and things along those lines and supporting teachers in, in getting them there.

Alan Gottlieb 9:12

Does it, does it require a complete and total rethinking about what it means to teach and learn in a time when the possibility of doing all those things you just talked about, about basically farming out all the thinking to, to a machine is possible?

Dr. Richard Charles 9:28

Absolutely. I think we education it education has not changed. In, in all of its existence. Still a sage on the stage model where the teacher's in the front of the classroom and pouring the knowledge into the students. You've seen that over and over again. Even if you ask ai, what is the future gonna look like? It's funny because. That's exactly the same model that it's giving, even though the technology changes on the desk and everything else. So we need transformation. I thought we would've seen some transformation after the pandemic because that really forced us to rethink access and opportunity and all of those really important concepts around the field of education. And for a while there we did, but then when the pandemic was over, we went right back to the norm. So yes, I do think we need innovation and education. I think there's some great groups that are looking at that. There's a group in in Stanford that I work with occasionally, and, and they invite me out to, to speak and to observe some of the research that's occurring there. That's very, very exciting work and other organizations are very much engaged with trying to redefine what education could look like in the future. And so I think those efforts need to, to really be further explored and supported as they try to do something that has never been done before.

Alan Gottlieb 11:05

I'm sure it's too complicated to, to go into in depth, but if you could just touch on what are some of the highlights of that work that the people in Stanford are doing?

Dr. Richard Charles 11:14

Yeah. There's some exciting research. There's a research repository that they created that looks at how AI is being leveraged across the field of education. And it's high quality research. It's engaging and it's now becoming a repository for other educators to be able to see how effective it is whether or not it's a viable option given where students are. And it's, it's the start of being able to look at interventions in education as a way to really have AI support. Our understanding of what, what those interventions should be and how effective they can be.

Alexis Menocal Harrigan 12:04

Is that the accelerator for learning at Stanford? It is Stanford

Dr. Richard Charles 12:07

Accelerator for learning. Yeah.

Alexis Menocal Harrigan 12:08

Yeah. We work in my day job, we work with Isabelle over there, who's the lead. That's right. And it's incredible work that they're doing. So if we have any folks who're interested in really understanding what's happening at the forefront of this research with, transforming the future of education with technology. They're, they're doing some really cool stuff out of Stanford.

Dr. Richard Charles 12:29

Yes. Yeah. That's Dr. Isabel Howe, for those of you HAU thank

Alexis Menocal Harrigan 12:33

you,

Dr. Richard Charles 12:33

who's been phenomenal in that research.

Alexis Menocal Harrigan 12:35

Absolutely. I wanna go back for a second. Speaking of transforming learning, I mean teachers, depending on when they graduated from their programs or came into the profession from maybe an alternative program, aI as a technology wasn't as readily available to them in their training and in their practice. So one of the things I see a lot is, you know, this focus on professional development, upskilling teachers and then helping them think about how to transform their practice in the classroom. At the same time. The, the concern I have, and I think a lot of folks have is educators already have so much on their plate and how are we. Thinking about upskilling and training teachers and help and, and asking them to maybe change what they've done for 10, 15, 20 years in the classroom in some cases and beyond without it feeling burdensome and just another thing to pile on on top of everything else that they're trying to manage in this, in this crazy world.

Dr. Richard Charles 13:37

It's a great question and I would, I would share that when I started in the field of education I think. We were still using attendance books with pen and pa, pencil and paper and, and things of that nature. It was before a lot of the larger student information systems caught hold in the industry. And even then that argument was the same Alexis, that is, we have so much to do. You want us to learn this new tool? Right. And in learning such a new tool, it really allowed teachers, it freed them up to be able to have a little bit more time. Right. So attendance was much more manageable. Data analytics became much more an manageable because of some of the analysis and really sharp thinking that's built into some of these tools. So I think that argument is always gonna be there because it, it always seems as if we're adding more and more and we're not taking anything off of the plate, right? We're still responsible to to race. Student achievement across the board. And so our challenge as a leadership team then is to make sure that we are strategic in talking about ways in which a, a tool such as AI can support teachers. For example lesson planning is one way in which teachers can use a, an AI as a thought partner. How can I engage students more deeply? How can I extend learning? And it will give them a tremendous resource with several options on how to do that effectively. So lesson planning is one right now. An example, a, a challenge associated with that is the tendency to lump everything together and throw student data in there, right? So if you start throwing in information that is identifiable to students, so in other words, you can identify a student based on the information that you're putting in there, then that becomes problematic. And so it's challenging in that there's a tendency to say, let's just go ahead and throw everything in, and then it will be able to do all these great things for us. So things like IEPs not recommended. And so we are giving guidelines on our website. On the academics website. We have an artificial intelligence. A webpage that provides all of the details on, on how AI can be used and some of the recommendations that we have for teachers, for students, and for parents. That I think addresses exactly what you asked about Alexis, right? Finding those, those really key points and key functions and processes within the school day that would make the the teacher's life easier.

Alan Gottlieb 16:42

Yeah, I have a quick follow up to that, which is just I would imagine that, you know, talking about you always, teachers continually have stuff added to their plate and nothing ever taken off. I mean, the temptation, I'm sure at times, especially for teachers in, in the secondary grades to, to have AI help them with. Reading papers and you know, if you've got 30 papers to grade and they're not short papers, that's just a hu That's, you know, teachers work late into the night doing that stuff. What are, what are the guidelines you have and it, are there circumstances under which it's okay for AI to say, do a, a first read through and give you some feedback that you then go into? Or, or is it just No, you, you have to continue creating. That's

Dr. Richard Charles 17:19

exactly right. Yeah. So, so human in the loop, as I said before. You always want to be doing some of the thinking, right? Let's not give up the opportunity to learn about what a student's is, is challenged in. So as a first pass I've seen teachers and we've heard stories of teachers that would use AI in order to, to give guidance to a student. On first level feedback, second level feedback, and then maybe that third version goes to the teacher as a first submission. So iterating is really an important concept. This whole notion also flies in contrast to the whole idea around plagiarism, right? We have, when it first came out, everyone was so worried about students using AI for plagiarism. It is the least of the worries that I have in the role that I'm in right now. And in fact, some research has shown from Stanford University that these plagiarism tools are not effective nor accurate. And so the legal cases are mounting. Where parents are now challenging some of these systems and, and they're winning. So having more of a balance in saying, yes, you can use it. Please disclose all of your sources as you do so, so that we're fully aware that you using AI to build your first, second version before submitting. I've also seen students do things like submit all of their writings to an AI tool. And then say to the AI tool in my voice, give me the first draft of this assignment. Right? So where's the plagiarism happening there? If the student is in fact being sourced, their work is being sourced. In order to do that. So plagiarism is a slippery slope. I rather parents and teachers think of this as, as a tool. No different than, you know, when we were younger, we were going to the Encyclopedia Britannica, or we were going to the library and looking through those index cards that were just. You, you know, how painful that was, right. And, and finding sources that way. It just really sets us up well with the tools that AI provides to be able to inspire students more specifically with the details of what a career pathway may look like and what a real problem in the field of that particular pursuit can look like.

Alexis Menocal Harrigan 19:57

I, I wanna take this in so many different directions because I, like, there's like four different things I wanna talk to you about based just on that comment alone. The, the first one I'll, I'll say is as we think about, the teacher's grading, for example, the, the essays, you know, I am in the very joyful period of the year where my children, my child is in his CMAs testing. So mornings are going really well around this household. Let me tell you right now no anger and frustration at all. I feel for you. Yeah. My, my third grader is not, is not thrilled about CMAs testing. But we're, we're doing testing right in a way that we've probably been doing the same way for a long time. One of the things I get excited about is thinking about the fu and I don't know how many people get excited about this, but I think you will, especially given what you were mentioning about Stanford.'cause Stanford's spending a lot of time thinking about this and Dr. How spending a lot of time thinking about this is the future of assessments and how are we thinking about how we're evaluating the student's learning journey. Instead of it being like this point in time, standardized test, what could, you know, if, if we were to really, take out the boundaries of what we know about testing students. What could a, a, a more effective way of evaluating student progress against learning and content against, against a specific content area, what could we do? So I guess my question is if Dr. Charles were, you know, in charge of, of, in charge of the universe for a day, and you could wave a magic wand and do any, any changes to, to the future of, of assessments and, and how students are being evaluated? What, what would, what would the perfect scenario look like for you?

Dr. Richard Charles 21:43

So, that's a, a great question and have lots of thoughts about it and can go in different directions. But I'm gonna, I'm gonna give you one way in which I've seen researchers really address this. Student portfolios from the time that students enter school in kindergarten all the way until you know they graduate, you have a a, 13 years of information that you can tap into in order to get a, a sense of, of what a student can and can't do. And you get to see their various phases of development as we, as they go through that journey. And so I know Dr. Tony Wagner at Harvard University has spoken about this years ago where student portfolio should be. The key to to, to how we understand what a student can and can't do. We just talked about an assessment, a single assessment in time, and the issue of plagiarism. If it's so easy for students to use AI to to plagiarize in, in some way, then there's. Fundamentally something wrong with your assessment. So you should be changing that assessment that excites educators because. You think about what the possibilities can be. One of my hobbies is going onto a website it's called Kaggle, and it it provides major research challenges in a wide variety of fields to data scientists and computer scientists to be able to provide solutions to. And what's powerful about that is any one of those problems. Students can now attack in a real world setting and provide insights that perhaps the adults would not be able to do. So that's just one platform. But the World Economic Forum has a list of, of challenging problems and things of that nature. So you're now giving students the tools to be able to solve major, major problems and. Project-based learning should be at the heart of much of the work that they're doing as they go through their, their learning journeys. And so between project-based learning and portfolios in K through 12, I think that would provide a lot of sense of what a student can and can't do.

Alan Gottlieb 24:30

Question that I have is, I'm, I, I'm just curious about how a system as large as DPS or any major public education, you know, school system, how, how do you keep pace both policy wise and practice wise given you talked about the pace of change. We have no idea today's kindergartners what their world's gonna be like, but I think the pace of what's happened with AI in the last two years, even arguably even the last year. It's been so dramatic that it's just, you know, we don't even know. And, and is it exponential and gonna keep happening? What, how does a district or a system become nimble enough to deal with that pace of change?

Dr. Richard Charles 25:06

It is a challenge, and I think districts are doing the very best that they can do in light of the challenge that they're faced with. So with regard to policies. DPS was at the forefront of some of the work around policy guardrails. For artificial intelligence. We were one of four school districts that worked closely with the council of grade city schools and with consortium with school networks. To establish something called the K through 12 generative AI readiness checklist and an associated rubric. And as we were developing a lot of these things, we realized, you know, this, this is gonna move pretty quickly. And so folks are gonna start rewriting many of their policies if they don't get this right. So what advice can we give them? And the advice that we shared then was. Stay high level with all of the things, right? Talk about responsible effective, secure and unbiased use of ai. And that's what we are striving for as a district. So you're not gonna find the names of any of the tools that you see. I know that a, a gentech types of models are posing a particular problem that we can talk about in detail. And we'll get back to that, but. Barring any sort of small change. For the most part, our policies have stayed in place. We do respond when we see from the state legislature that there's conversations happening around what's best for students and what's best for the, the broader business community around artificial intelligence. And we've pushed back in some cases, and in those cases it, it's kind of tough for folks to hear, and I typically get, typically get a phone call and I explain to them, and then they, they pretty much understand, yeah, this, this is not a good thing for us and we, we should really rethink it. So things will continue to evolve, continue to, to, to move quickly. The slowest pace that it will ever be is the pace that we have today. And so we're anticipating that if we stay high level. Really train students and adults how to use these systems responsibly, keeping in mind data, security, privacy and so on, and, and knowing that it's it's bias in some way, then I think we can, it becomes a manageable task. And, and that's what we're talking about.

Alan Gottlieb 27:44

You were just talking about challenges presented by AgTech models and challenges or problems. Could you elaborate on what those are, because I'd just be really interested in hearing.

Dr. Richard Charles 27:55

Yeah, yeah. So I'll start by describing what an, an agent is, right? So you're, you're all familiar with a large language model, philanthropic, open ai Gemini, and so on and so forth. Typically you can interact in the interfaces that you're all familiar with. However, you now have the ability to create what are called agents, and agents can go off and do multiple tasks. On their own that you tell them to do in some cases, and in some cases you just tell them what the objective is and you tell them also to use an age agent approach to solving the problem and they will spot off other agents to go off and then solve the problem. One of the biggest challenges there is cost structure. So the cost model for many of the large language models that we are experiencing now is anchored on the tokens that they use. And these are, just, think of them as words, right? The number of words that it submits and the number of words that it generates back, and so on and so forth. Well, if you get some of these agents that are really smart and really elusive, which some of the research is saying is now happening, then they can go off and be stuck in a loop or they can go off and try to solve world hunger as one of the side, you know, tasks that God generated as a result of the prompt that you put in. So prompt engineering is now very critical as a skill. To be able to have, so that a, you get the response with the fewest cost, right? So can I write a prompt that would allow me to have to minimize the cost of my interaction with the system? And also a prompt that will. Halt in, in effect, if you're familiar with the governor in a car cars engine where after a while it reps so high that it stops going any higher because there's this mechanism called a governor that that stops at, at a certain point. And that's the sort of notion that developers are realizing that they need for these age agent systems now, because otherwise the cost structure becomes problematic.

Alan Gottlieb 30:21

I can't help, but I'm really dating myself here. Think mm-hmm. The old Disney cartoon from like the 1950s, and I'm not that old, but the, so Fantasia where there's like Mickey Mouse and they're playing the music, the sorcerer or the Apprentice, and he's trying to save time and not have to clean up the laboratory, so he. Like, does this spell And it turns into like 50 brooms and then a thousand brooms, and then a million brooms, and then a billion brooms. And it's like,

Dr. Richard Charles 30:43

that is the, that's a great metaphor for egen systems.

Alexis Menocal Harrigan 30:47

Yeah. Or for a more modern take. Harry Potter, when they're in the chamber and they touch the goal, it's like the Midas touch. Everything gets duplicated.

Dr. Richard Charles 30:55

That's

Alexis Menocal Harrigan 30:55

right. So

Dr. Richard Charles 30:57

that's right. So being able to spawn off these additional AI. Bots to do work for you is a big deal. And right now it's still very early in, in the growth of these systems and and so companies are, are not necessarily profitable just yet. Because they're taking on much of the cost, particularly in the field of education. But at some point, that's gonna gonna be an issue for, and if you're not mindful for, for school districts, if you're not mindful of it, then it can, you'll find a, a bill for$250,000 because of a, that you did. Right? So I know that many organizations are faced with that right now.

Alexis Menocal Harrigan 31:41

So let's talk, let's talk about that a little bit. Dr. Charles. I think the last, one of the last times you and I saw each other in person was actually in the Silicon Valley at an event held by Google and you had mentioned to me. That you had helped bring in, I forget how many, hundreds of thousands of dollars of, of cost savings or department did to the district because you were consolidating so many of the tools that, that the district was using. One, I, I would love to hear about that. So you could share, share with our listeners, but then two, you know, where you sit within the organization as the CIO. I mean, you have such a big responsibility to make sure that all of the contracts that the district is, is going into and all of the agreements with all of these companies. And, and let's just talk about the ed tech companies for a second. One that, that, you know, they're safe for, for teachers and students to use. Two that, you know, your. You're stewarding taxpayer dollars. Well obviously you, you don't wanna be spending millions of dollars on tools that aren't necessarily gonna produce meaningful student outcomes. So how do you balance that responsibility? When I, I know that there are so many sales reps from all of these companies out there trying to, you know, promise, promise the world that their product is going to be the revolutionary thing that's going to bring every student to, to reading at grade level, for example.

Dr. Richard Charles 33:03

Yeah. And they're slapping AI on every one of their products. Yes. Whether they, it's just a call to an AI tool which is an an API or whether it's something that they're building in house. It's a huge problem for school districts. And there's a lot of money in that, in that arena. That. That school districts are paying for some of the services we are. Over the past couple of years, we've shifted our thinking pretty drastically, and on the same time we were developing some of the tools on artificial intelligence. When I took the helm, I, I recognized that we had over a thousand applications running on our network, and the majority of those applications had less than 10 users. And so we figured, you know, we needed to take a look at, at, at the problem. And so between the Department of Technology Services, which I lead the dots team, and some really talented individuals on that, on our team here and some talented individuals and the educational technology side of the house, from the academic side of the house. We came together and involved the procurement team, the legal team, and did an entire revamp of that entire process. The process looked like a spaghetti chart. One of our members of technical staff here at DPS Maria Marba, she, showed this diagram. And it was the funniest thing because it, it looked as if it was a a spaghetti a pot of spaghetti. And then by the time we were finished, it, it was a very clear seven step process. Now, that being said we do recognize that we always have to iterate on this and refine it. So that's what we've been doing over the past few years. We did get it down to about two, 250 or so applications. Ultimately, it's now rising back up to about 450 applications and you could see these applications being submitted as well. It's public knowledge is a Colorado state law that requires that it's. The atm, it's called academic Technology Menu atm. Do dps k12.org and you will see all of the applications that we are using in dps. So we are constantly revamping. I know we had the accessibility law as well that we now have to make sure that every. Application is accessible. And so for the past two, two and a half years now, we've been really working hard to address that. And every so often we go back and we clean things up. And so as things continue to evolve and features are added to software tools, we're making sure that we're revisiting each of those software applications to address some of the needs that we have. And. One other really important point is, you know, there's, we are. Firmly aware that there's a, a national dialogue right now around, let's go back to paper. Paper and pencil. I don't know if you all are familiar with that.

Alexis Menocal Harrigan 36:28

That's one of the questions I have for you, but let's jump into it. The, the tech lash. But, but have a follow take test on.

Alan Gottlieb 36:35

No, I wanted to go in a different direction, so, okay. But I, yeah, so what, just talking about what you're talking about, like the move to go back to Blue Books for tests so that students can't use anything except their own brains and a pencil, you know?

Dr. Richard Charles 36:46

Yeah,

Alexis Menocal Harrigan 36:47

yeah. But, but there's a huge tech backlash right now and you know, some folks are, are really digging their heels and saying like, we just need to go back. Others are like, well, we need to be more discerning on this. So that, that was one of, one of the questions I had. So you're, you're on it. Let's, let's hear it.

Dr. Richard Charles 37:00

Okay. So, so my thoughts on this, mathematics is, has been one of those subjects that, that's similar in it to education in that we've not seen a whole lot of change in how mathematics is taught over centuries. Right? It's, it's pretty much the same. You gotta learn the timestables at some point and then continue to do this. What's unique about that is most other subjects. Are impacted pretty significantly and update their sequence of learning. So for example, in biology, we now know more information about the the human cell because of. Tons of research over hundreds of years. And also some of the insights that's, that is resulting from some of the incredible work that de Saba at Google and other folks are doing with protein folding and things of that nature. So we now know much more information. Textbooks will be changing as a result of that because our models were, were incorrect or incomplete at the time because of what we knew. That's not, that does not seem to be the case for mathematics. We keep going back to the same thing, right? So when you talk about innovation in mathematics, the person who comes to mind for me and I know it's not just mathematics, it's much broader, is a gentleman by the name of professor Terence Tao. He's at UCLA, genius, mathematician. He is leveraging technology in like six or seven different fields of mathematics. And we, mathematicians, lawed all of his work, he's just an amazing individual, was child, child genius, but also leveraging technology in order to get better bounds on some of the open problems and mathematics. And he wrote a a, an article in the Atlantic the other day. They talked about the leap that he's seen in artificial intelligence and its ability to solve real world math problems. A mathematician by the name of Paul Erdos great mathematician, posed a number of problems that have been longstanding and now AI is able to solve these problems. Pretty significant. So the reason why I bring that up is even folks who've been, have been traditionally taught mathematics are now pushing the envelope with discovery and developing new mathematics as a result of some of the tools that we have and the insights that they're getting from that is tremendous. If we go back to paper and pencil the way that it has always been. I'm sure it will satisfy some folks who find value in that. But I do think that it becomes severely limiting, particularly when it comes to the curriculum. Just about every curriculum company that I'm aware of is now delivering things electronically. So when there's an update in new knowledge or new research that has revealed something publishing companies. Are able to send an update to their software and every single student now has access to that same information. So the diffusion of information for the learner is pretty significant. And that is something that we find value in. Aside from the fact that these are tools to be able to present gather your, your ideas in different ways I think that that is a really big idea. The, the next one that is, I think a, a bit of a challenge is we talked about the pace at which things are changing, so the gap between what students are learning. Even in the case where they're getting information from publishing companies, as the research gets updated to what they learn from in, in kindergarten until what, where they what they learn in, in 12th grade and graduate, that gap is going to widen tremendously because the, with AI in, in our world today. How do we, how do we describe this diffusion for AI systems? It's very clear if an AI knows how to do something, that version of the software isn't, is updated, and every version across the globe knows the information instantaneously. So. How do we keep up in a world like that if you are now going back and limiting the diffusion of information? So the diffusion of of information is critically important in, in this setting.

Alexis Menocal Harrigan 42:23

Thank you.

Alan Gottlieb 42:24

That's a great point. And I think maybe too often overlooked in the sort of fear, fear moments we have. So. Last question'cause we're a little short of time and this is really going in a different direction, but it also, I think is in your bailiwick. So it's why I want to ask about it. And that is about we're, we're, we're going back sort of into the, the modern day instead of the future a little bit here, but, just curious about where DPS is currently in terms of the use of personal devices, like cell phones in classrooms, and I know there's, there's work being done in the community to sort of come up with a policy, but, but where, I mean, you know, I, I've read Jonathan Hates the Anxious Generation and all of that work, which is I think is fascinating and just how it affects learning. I'd be interested in your thoughts about where that is going and needs to go and maybe to bring it into the. Present slash future a little bit. You know, there's all this talk, or there was a few months ago about the former Apple designer Johnny or Joni Ivy going over to OpenAI and he's working on some magical device that he's gonna develop for OpenAI that's gonna be, not have a screen necessarily, but gonna be this really cool AI thing that's gonna be the next iPhone. And like how do you. Plan ahead for something like that, that could be the next thing that causes huge distractions and disruptions in the education world.

Dr. Richard Charles 43:37

I think as long as they don't embed it in your skull, I think we're fine. No. So I'll, I'll address the computer device. We have a computer device. Advisory committee is a result of, of the, the state law that we are trying to address. And that committee was pulled together by board members and it's led by board members. And we have principals, teachers, community members, all a part of that committee. And we've met now for about, I think seven or eight meetings and really looked at this issue of a computing device during the school day. And they're in the process.

Alexis Menocal Harrigan 44:21

Sorry, Dr. Charles, I'm sorry to interrupt. When you say computing device, just so folks are, are aware,

Dr. Richard Charles 44:26

oh,

Alexis Menocal Harrigan 44:26

you don't mean necessarily personal or computers. It's it's com. It's. Personal devices like watches, smart watches, phones, things like,

Dr. Richard Charles 44:34

correct. Yes.

Alexis Menocal Harrigan 44:35

Okay, cool. Just wanted to clarify for folks.

Dr. Richard Charles 44:37

So and, and the thinking. Thank you. Alexis, the the thinking behind it is these technologies are a distraction to students. Right. Students already have access to Chromebooks in our environment and in other school districts it will be something equivalent to a, a Chromebook. And so during the school day, they're pulling out their phones and interacting, doing social media and doing other things. So how do we not disrupt the learning environment when they're bringing their personal devices in? So that excludes. Our devices, our Chromebooks from the school district. And so there's discussion that's occurring right now, and several committees went. Several com staff members went ahead of this committee in front of the committee to share with them all of the knowledge that's needed in order to, to make a, a decision and a recommendation to the board. So I know that they're in the, in that stage right now. But it's a, it's a huge challenge, right? So some parents are very concerned because that's their only lifeline to some of the students, and it, it poses a challenge for some parents. In terms of student learning, it is a challenge because it's a distraction for most, most students. During the school day, there's bullying that's occurring and, and things of that nature. However, flip side of that safe to tell is often done on a personal device. Right. Where if a student sees something that's concerning around the safety of another student they can, they can access it on their personal device, but they can also access it on our Chromebooks as well. It, it's, it is a huge problem and know that we're working on it and we see different sides of it, and I think the community members have done a fine job and will be making the best recommendation to the Board of education.

Alan Gottlieb 46:42

Thanks. We had, we had Alex Medler from the Boulder Valley School District Board on either late last year or early this year. And they Boulder, I think you'd probably know better than I do, but they just went ahead and all outright banned cell phones from classrooms maybe effective the beginning of this school year. So they moved pretty boldly on it.

Dr. Richard Charles 46:59

I think some districts are gonna do that and some are gonna be softer regardless of what the situation is. I think the state is right. We need a, a uniform policy so that we're not setting educators up where a teacher says, you know, put that away and then the student says, you know, they allow it in my, in my science class.

Alexis Menocal Harrigan 47:19

Exactly. So it's really unfair. Co

Dr. Richard Charles 47:21

consistency is the key. Yeah.

Alexis Menocal Harrigan 47:22

Yep. It's really unfair to the teachers when it's classroom by classroom. Rules and they're different. And yeah, I think, I think it's important to have a minimum schoolwide policy, ideally district-wide policies.

Dr. Richard Charles 47:34

And then what was the other part of your question, Alan? I'm sorry.

Alan Gottlieb 47:36

It was just the whole thing about like then how do you get ahead of the, the next personal device coming along is gonna be this open ai, Joni, Ivy, whatever it is, you know?

Dr. Richard Charles 47:44

Yeah. That, you know, the personal devices, these things. I was an early, I'm always an early adopter, right? So I have the, the, the smartphones as well. And, they too will fade and then there'll be something else. Right? So we just need to be, to be flexible. And so in a world where. You know, I think about how I use some of these devices as both in my role as CIO and as a mathematician, and the use cases are very, very different and very, very justifiable. And so another example of when you just take technology, not in the case of the, the, the school day, but when you just take technology for what it pro presents as an opportunity for you to explore the world in different ways. You should really embrace that. I mean, there, there's value in, in doing that. So the mathematics I can do with a device like this is eons beyond what I could have done with pencil and paper.

Alan Gottlieb 48:53

Mm-hmm. Right.

Dr. Richard Charles 48:53

In terms of analysis, in terms of writing, in terms of all of the pieces, modeling, so on.

Alan Gottlieb 49:00

Probably even beyond like what you could have done back in grad school or whatever. Absolutely. The most sophisticated Texas instrument calculator that costs hundreds and hundreds of dollars back then.

Dr. Richard Charles 49:09

Yeah. And I, I actually wrote a, a, a medium article about that.

Alan Gottlieb 49:13

Interesting.

Alexis Menocal Harrigan 49:14

Do they still use those in schools? The, the TI calculators?

Dr. Richard Charles 49:17

They do. Okay. They do. It's, yeah. Yeah, and I'll reserve my comments on that, but yeah, it's another example of how mathematics. Just does not want to shift. However, you know, there are some really good online tools, like Desmos is a great online tool for math teachers to be able to do a little bit more exploratory types of learning with students. So it's changing slowly.

Alexis Menocal Harrigan 49:42

That's great.

Alan Gottlieb 49:42

I think we're about out of time. Dr. Charles, I know Alexis has another appointment coming right up. So this was, again, you know, we've gone really long because we did this with Antonio too. We could talk about this for another couple hours, so I'm sure we'll, we'll have you on again as things continue to accelerate and change, and we really appreciate your time and your insights.

Dr. Richard Charles 50:01

My pleasure. Thank you so much for having me again.

Alan Gottlieb 50:04

Okay. Thanks again.