Harnessing Artificial Intelligence in Higher Education

Following up on the previous episode about AI’s potential applications to higher education, this episode focuses on the work that the Computational Approaches to Human Learning research lab at UC Berkeley is doing under the direction of Dr. Zach Pardos. We talk about ways AI can assist with transfer or credit mobility, ways AI might be able to streamline the creation of degree pathways for students, and ways AI is being leveraged for adaptive tutoring, all with an overarching goal of increasing the student’s social and economic mobility.  

Key Takeaways:

• AI can be considered as anything that takes what a computer is good at and pushes the boundaries closer to doing things that only a human is good at.
• It is vital that we use AI in service of our humanistic pursuits, that it’s used in a way that retains the humanistic character of our education enterprise.  
• AI can help add desirable regularity to systems that are sometimes unnecessarily idiosyncratic.  

Host:

Doug McKenna
University Registrar, George Mason University
cmckenn@gmu.edu   

Guests:

Dr. Zachary Pardos
Associate Professor of Education, UC Berkeley
https://bse.berkeley.edu/zachary-pardos 

References and Additional Information:

Welcome to OATutor - An Open Source Adaptive Tutoring System

https://oatutor.io/ 

AI Transfer and Articulation Infrastructure Network (ATAIN)

https://atain.org/ 

A University Map of Course Knowledge

Transcript

Copy

0:00

You're listening to For the Record, a registrar podcast sponsored by Acro. I'm Doctor Zachary Pardos, associate professor of education at UC Berkeley, and this is harnessing artificial intelligence and Higher Education. Hello. Welcome to For the Record. I'm your host, Doug McKenna, University registrar at George Mason University in Fairfax, Virginia. In March, I put out an episode about artificial intelligence, and I expressed some skepticism about potential applications of AI in the registrar's office and in the work that we do. I received some really interesting feedback about that episode, especially from people who made the point that individuals can leverage AI to improve their own performance or to assist with a task that would have taken a lot longer without using AI. And while those examples are certainly compelling, and I didn't mean to imply that AI wasn't useful. I haven't been sold on the adoption overall of AI by registrars' offices, not completely anyway. But then at the annual meeting in Seattle this April, our guest today, who runs the Computational approaches to Human Learning Research Lab, co-presented a plenary session about ways higher education was harnessing artificial intelligence. He showed examples of ways AI was helping student mobility by improving the transfer credit process, but also by enabling kinds of degree exploration that previously would have been incredibly labor intensive. And I thought to myself, self, maybe AI can and should be applied to the work registrars do. So let's talk more about that with Doctor Zach Pardos from UC Berkeley. Doctor Pratos, welcome to the podcast. Thanks for being here. Thank you. And I can call you Zachary for the or Zach for this conversation. Yes, please. Perfect. Thanks, Zach. So to kick us off, would you tell us a little bit about yourself, where you're from, what you do, maybe a little bit how you wound up working on AI at UC Berkeley? Yeah, so I'm an associate professor at Berkeley. I'm in the School of Education, but I also teach a large data science course in our data science major and have an affiliate position in cognitive science. My background is is all computer science, bachelor's, masters, postdoc at MIT, however, through that whole education, all those phases, I've been focused on applying. AI techniques to education, including my PhD was on computational models of of student learning. Um, I think it all came from a curiosity of how the mind works, what's the architecture of the mind, what is it analogous to? Is it analogous to a hard drive and memory in a computer, um, that kind of wasn't a sufficient analogy, but, you know, AI was a very intriguing way to study. And how that reflects human cognition, and then to use AI to study. Uh, learners and pathways to higher education became um intriguing pursuit. Right on. So right here at the top, let's start by defining some terms because there are a bunch of misconceptions about AI being brand new and in general, when people talk about AI these days, it's almost always about generative AI or more specifically, it seems like there's been a reductive understanding of AI that AI equals chat GPT and we know that it's a lot more than that. But would you tell us how do you define AI? And when you talk to people about AI, what do you mean? Yeah, I can give a, I can give a sort of a popular definition of it and and the the way I've seen AI. So when when I was a Undergrad and then a a PhD student, I was using a kind of machine learning model called Bayes Network to estimate if students had learned Pythagorean theorem, for example, by observing their various actions within a tutoring system. Beya Networks back then was considered the AI, you know, model to be studying. And so I think AI is anything that's pushing the boundary between the tasks that computers are good at, and the tasks that only humans are good at. So right now it's neural networks which drive chat GBT those, that is the primary sort of modeling paradigm that's trying to push that boundary of of what only humans are good at and trying to make computers good at some of that too, and you see that in Intelligent gameplay, like games like Go or, you know, um chess has been sort of beaten for a long time, and now I suppose driving, right? Autonomous driving and and what what are the kind of computational models that are trying to make computers good at driving too, and it's it's neural networks and really the same kind of neural networks that that power chat GPT. I wish there was an application that would make humans good at driving. Yeah, we haven't mastered that. It's a bad example. Make computers good at something humans were never all that good. Let's talk a little bit about your lab. Um, you run what's called the computational approaches to Human Learning Research Lab. Tell us a little bit about that. How many researchers are there? What's the focus of the lab? Um, what kind of work does your lab do? Sure, so, lab was founded when I joined uh UC Berkeley as an assistant professor in 2013, and we study AI applications for credit mobility, for advising, and for adaptive tutoring, all in the context of higher education. In 2015 we uh submitted. Uh, NSF grants to apply what have today become the, the go to AI of of neural networks, to apply nascent neural networks to higher ed administrative data, enrollment sequences, course descriptions, um, and I think we were the first lab to receive grants to apply that kind of methodology to higher ed data, and it led to the research agenda of of pathways research, articulation and transfer today. The lab is growing. Uh, we have 5 PhD students, 3 research uh data analyst students are sort of kind of like postdocs, but postmasters. They have a research position and, um, uh, a healthy team of about 15 undergraduates. That's awesome. And have you been affected by, did any of your grants get doged? Thankfully, this phase that the lab is in in terms of funding is completely foundation funding funded. So, you know, thank you to the Gates Foundation, your fund college features, etc. for um supporting the work and uh I think they're an important um pillar in uh educational research right now given the conditions of things. Yeah. At the annual meeting you talked a little bit and showed us some pretty impressive images about using data to sort of map relationships either between individual classes or subjects overall. And I think that that was in reference to some credit mobility issues and then also some advising issues. But would you tell us a little bit about how that was done, what data sets were used, how you gathered that data, and then why did you put those things together in that way? Yeah, so one of the examples was finding equivalent courses or exploring how similar the course material was among courses just at one institution, and that was Berkeley, and we called it a a university course map of knowledge, and that's that's published in a journal called Plus One. And it's a visualization of the mind of a neural network, and what it was given. We thought, interestingly, was just enrollment histories of students. So, we, we didn't tell it what department the course was associated with, we didn't give it the catalog description at that point. We only told it who took what course and at what time. And that creates enough of a kind of network where the neural network tries to understand, well, given this sequence of course taking and that sequence of course taking, which courses were sort of synonymous with one another. So if I told you uh a sentence of, I really blank that movie, I'm gonna try to buy it on Blu-ray. I liked it so much. Blu-ray is a kind of video uh disc that um some people might not be wear or maybe the listeners here will know that, but um my students may not. And uh but I could say I, you know, was thrilled by the movie I'm gonna buy it on Blu. So I can replace different words and even though you don't know what blank meant, you can infer from context what that word is about. And so similarly, the neural network kind of infers what courses are about from looking at tens of thousands, even hundreds of thousands of such sequences, but instead of sequences of words, it's sequences of course. So you OK, everything was taken here, but this was replaced with that. Maybe that is an elective or maybe that's satisfying the same requirement category. So in any case, enrollment histories and then later catalog descriptions were fed to the neural network to look at similarity, and we validated that against uh faculty definitions of courses that are so similar, you can't get credit for taking both restrictions. And then we started applying that cross institutionally. Right? And that cross institutional similarity is essentially a um addresses a big problem with transfer, which is credit mobility and articulation. Right. When you pulled those data initially, one of the concerns about using AI in higher education is sort of a sense of security and as a registrar and registrar listeners, and this is a registrar focused podcast, one of our concerns is FERPA. And so how does your lab go about protecting the data. And what sort of safeguards are in place for the privacy of data, not just in this specific example, but in AI in the usage of AI overall in higher education applications. Yeah, it's a great question. So what we've found since running that study is that using public sources of data can get us pretty close to what that more private source of data, which is enrollments, could get us in terms of estimating course similarity. So using catalog description. titles, and existing database of equivalencies, like a receiving institutions, um, articulations that they've approved, we can get very close to what we could get with enrollment data. Now, it is still better to have enrollment data. Uh, to make the model more robust, particularly if the catalog description has experience concept drift, right, or if it never was all that accurate to begin with, some catalog descriptions I see are, you know, intro class, you know, is the one sentence description. So it can compensate for that. But in in the case that we do use enrollments for that purpose, it could be completely de-identified enrollments, and we don't make them public. And that covers FERPA, as far as I've studied it and various ingestion of of student data I've used. Also, if it, if it needs to be identifiable, let's say it's being used as part of a AI course recommender system that needs to kind of communicate directly to the student and know their identity. Then as long as it is part of the institutional operations, which can be extended through partnerships and vendors and so forth, it's still covered under FERPA. Of course you need protections, data security and so forth. I think the important thing with AI is that you're not feeding a prompt that includes identifiable student information to a company. That gives you no assurances that they won't retrain, you know, on the prompt data. So that is something to look at and we've been trying to find such a license. It's a little bit complicated to secure that, so, you know, we are,-- are very cautious to not uh include that in-- prompts. Right on, right on. What's the overall goal of your lab, and you're focused, you've mentioned credit mobility. What do you hope to establish or accomplish? What's the end result of the work that you and the people who work in your lab are doing? Uh, the end goal is to facilitate greater social and economic mobility. That all of these sort of logistical hurdles that aren't intentional, that maybe we've rationalized that as higher ed, that this is by design somehow is meant to be this complicated, that we can't. Yeah, it was not. It's not a feature. It's definitely a bug, and it's, and it's hurting students disproportionately, um based on background. So Removing, reducing the friction of administrative processes and student processes to find your way through higher education, be it that kind of mezzo level, finding your way of choosing your major, choosing your institution, choosing your course, and then when you're in a course, Providing some supports that teachers are in control of to help scaffold the the learning process. Is that where the adaptive tutoring comes in, because that's one question that I have. Explain what adaptive tutoring is. Yeah, so adaptive tutoring is a bit of a broad category, but it the the the canonical example of it is a system that breaks down what's being learned into learning objectives and is constantly assessing where students at in that learning objective, and if they've mastered it. It doesn't have them practice it anymore, and if they haven't mastered it, it they practice more. So it's about giving a custom amount of work to students, and it's based on kind of Bloom's notion of of mastery learning. Right. And have you had any successes in that area? We have, we've had some successes in terms of measured um effect on learning. We do find that it does produce significant learning. It was an effort that is ongoing to basically replicate the positive learning, gain results of commercial systems, but in a completely open source Creative Commons form. So everything we produce that we get funding for, we're collaborating with. Cal Bright College online college in California and and other schools. Whatever we produce for this system can be reused without licensing fee. It's also open source, so a PhD student, you know, or undergrad, anyone around the world who wants to tinker, the look and feel of the tutor can do so, and we hope that that catalyzes more innovation in this um adaptive technology space. That's pretty amazing, and I applaud the openness of this because I find that in higher education there is uh generally just a spirit of collaboration amongst administrators, amongst registrars for sure. And so it's lovely to see and to hear that this work is both being done, but then also shareable and reusable across because it's important work. You mentioned that sort of this work is ongoing, and I was wondering if you could talk a little bit about what does that mean? What's that look like? What are the phases that you're in right now? What's upcoming, what are the big sort of milestones that you see for the development of this ability to harness AI in particular ways. Right, so the phase zero, which we're passed is showing that the, that technologically, algorithmically, this uh satisfies this kind of task of equivalency checking and that um the algorithms can also plan out a student's journey across institutions accurately to satisfy, you know, receiving school degree requirements. Now it's kind of a combination of practical, logistical matters and policy. And so we're we're finished with the stage of recruiting for an initial cohort of 60 to 100 institutions, some of them at the system level, some of them individual institutions. To then see, you know, how do you implement this technology such that it gets taken up, such that it saves time, but also that it moves the needle towards this goal of like, greater social and economic mobility, that it really is leading to more pathways being created, students getting more credit, students having greater visibility into future paths to take, and fewer of these instances where they find out, oops, that rule was, you know, 2 years old, you actually have to retake this. Oh, and that of course is. being offered anymore and so forth. There's a lot of this kind of cross referencing of data that we force students to do where they have 7 tabs open looking at the PDFs of this school and that school. I'm like, isn't that a technology thing to help? Yeah. So um the second phase will be the student facing phase right now it's very administrative facing, right? It's creating the pathways. And then phase two, which will start in the summer, is helping students traverse those pathways. So being aware of the pathways, figuring out what the usability issues are there. And then, and then try to broaden the kinds of equivalencies we can create past course to course to course to requirement category, course to gen ed course to common course number, and so we can kind of help institutions create and students create greater regularity, right, of processes and equivalencies and emerging topics and in higher ed and not require a kind of active state Congress and $50 million to um Do program articulation. Yeah, yeah. And do you see this work circling around then to come back to sort of statewide systems or to interstate compacts of transfer and transferability. And just as a side note, there is a separate Acro podcast called the Transferee, which everyone should listen to because it's great. And so sorry Loya that we're talking about transfer ish, but with the data that we'll be able to glean from these kinds of advancements, do you see this later than informing Policy discussions, like I said at the statewide level or interstate compacts to say, look, here's the, the environment or the network of transfer articulations and and ways that students are navigating through here, we could do that there or we could make it easier across Like, how do you see this data being used sort of at a macro level later? Or have you envisioned ways that it will be used? Uh, we have, and we've kind of had to, um, because we've been blessed with having partnerships with the state for different projects, including helping aid in common course numbering. Um, and so how are these things connected where you have a common course number, in some cases, you have a course to course, in some cases you have a new gen ed taxonomy. How can AI sort of help smooth the edges between these things? And I think the the first killer application will be kind of automating the creation of crosswalks between all these types of educational objects. Fascinating. Yeah. Say more about that. Yeah, so, um, first of all, state to state, there's different. Geneds, right? There's and some-- states-- academic freedom. That's the freedom. Everybody gets to decide what's theirs and how it runs-- exactly-- creates all these problems, creates all these problems and some states like Texas are lucky enough to have many text. and many systems. And on one hand, you want to allow academic freedom. On the other hand, it creates a lot of logistical nightmares when it comes to credit evaluation and understanding how students can move between institutions and, and I guess have student credit freedom, right? And and so what AI can do is help us. Maintain both, you know, you can have your academic freedom, we're gonna use AI to estimate the crosswalks, the equivalencies between different degree requirements, different gen ed major requirements, etc. I think part of that compromise needs to be, you know, that that faculty have an open mind uh to incorporating technology in this way. It, in my view, it's taking It it's something they have to be involved in in in terms of governance, but it's also not a task that most faculty would put in the top 10 of things they pride themselves in, you know, as part of their position. I don't think they necessarily want to be making. You know, dozens or hundreds of decisions about credit equivalency as a faculty member teaching a course. And if they could be assured that students aren't going to come to their course or the post requisite of their course unprepared, I think they're happy to delegate that to another process. Yeah, I think that's an important point is that faculty members care deeply about student learning, in my experience. And they want students to do well, they want them to succeed. And when they design a series of courses with a subsequent course having a prerequisite, they want to make sure that the students know the material from that prerequisite so that they can be successful in the post requisite course, right. But as you said, like it's not. It's not high on the list of tasks for faculty members to review course syllabi from other institutions to see if this the course over there matches the course over here. Right. And they're they're usually, they have less information than they need to really make a strong prediction of how that transfer student will perform and transfer students are very gritty, so they're probably going to perform fine. So, you know, I, I think we can offload that cognitive load of. Asking faculty to create that predictive model and and use modern technology and analytics with them in the loop, maybe letting them know, here's the students who got credit for your course and how well they did, you know, do you want to initiate a reconsideration of their articulation? Are you fine to keep it the way it is by default? So I think there's a happy medium that is out there for sure. And that's what we'll be exploring in this current phase. That's cool. I appreciate that. Let's go a little higher level, and I'm interested in your thoughts on what are the ways that AI has the potential to change higher education and, you know, are those foundational changes, meaning like this AI is a true industry shifter, or do you think that AI will play more of a streamlining sort of softening the edges, reducing administrative burden, kinds of a role. What in your opinion? What potential does AI have? I think a little of both. So, you know, the, the first kinds of products or innovations will be around efficiency, right? And, and softening those edges. Also, with, with most introduction of, of technology, it's usually education that's pitched as it's gonna revolutionize education and being around for enough decades of of those catalysts, you know, it usually has a mild effect. In the case of AI where I, where this does feel like it could be, you know, bigger than other technologies that have been introduced, like it's certainly gonna be bigger than MOOCs, right? I was just gonna say MOOCs, like, you remember when MOOCs were going to destroy education. I remember I was at MIT at the debut of the yeah. And now we're, yeah, um, and Sam Altman, I think is saying AI can be bigger than the internet. So it's going to be like somewhere between there, bigger than the internet, bigger than MOOCs, between the internet and MOOCs maybe I think is reasonable. Um, I think in terms of like the structure of higher ed or at least higher ed administration, I hope that AI has the role of making us less siloed. You know, in, in what we do, I know some, some faculty work, particularly at the upper division is going to necessarily be kind of bespoke, right? And it's going to be their own, as are all courses they teach, but things that we do, whether it's advising, articulation, admissions, I think AI will have will help add desirable regularity. to systems that are sometimes just unnecessarily idiosyncratic, right? There's a difference between putting your signature on something and just having a A kind of structure that has gone out of control in in terms of being idiosyncratic.-- So they're adding just from-- the accrual of random things over time and you're like, but this is how it is. Right, you know, if the course comes in and has a 0 at the start, we convert that to an A, you know, all of these little rules. I think it can help add sanity to that, especially the information systems, which will in in um in the end help students as well. Right on. So that's sort of for me, that's sort of the streamlining, softening the edges kind of a thing. OK. You said a little of both, and so I'm gonna, I'm gonna poke you on the like, what are the ways that you think AI has the potential to really change higher education? Um, Yeah, I, I, I guess the more broad than that. So my my standards for higher ed are kind of low in terms of like what is a big change.-- So I saw what I just said is like-- a big change, um. But maybe that's not, I think that's realistic, right? Like, higher education is this big ship. It's a big boat. It takes a lot to turn it. And even incremental shifts over time really changed the trajectory of where that big boat goes, right? And So I think you're not wrong to think that streamlining some things, having faculty offload some of the cognitive loaded, but not necessarily rewarding practices, that that does move the needle in ways that maybe then that frees up some time for faculty to engage differently with their students or pursue additional research or some other thing that makes the world a better place. When they're not encumbered by sort of these other administrative tasks. Yeah, I, I can go an inch further than that. Yeah. Um, so a couple years ago, we were asked to co-author a chapter for something called California 100, which is the the future of X in 100 years in California and try to write that chapter, of course, you know, impossible for technology, but of course, you know, you have to try. Um, so, drawing from some of that, I think we, and, and again in that it, it was not ultra radical things, but You know, for example, I couldn't even think in 100 years, you know, a higher being too different, but um, universal sort of admissions, which is already kind of the case with some systems where you can cross register very freely, transfer sometimes a little bit more difficult, but there are some systems that are multi, you know, intersegmental, where there is a little bit more um ease of movement. But I think with AI and these kinds of equivalencies, you could have, you know, much more higher ed and entry point into the higher ed could be much more universal than it is and and still have a kind of sense of ownership. Oh, I finished from this school. So I think there's a lot of models, including with online, right, how you could expand the concept of cross registration. In reasonable ways, and of course accreditors have to be on board and institutions to do as well, but I think it opens up a lot. And I also think that with both the ability to have technology for help faculty and leadership create pathways. And the ability of technology to do the generative AI to refresh course materials and assessments and so forth. I think the combination of those with with maybe a little less rigid degree complexity requirements. So like imagine a a degree. Um, requirement standard that does not look akin to a, you know, early 70s programming language. Right? Like can you imagine that? So like a little bit simpler rules. I think the combination of those could really make um higher ed more nimble in responding to what, to what Society needs, whether it's employers, right? Whether what it means to be a human, so we actually need more in the humanities right now. If you want to create a data science program, you know, you're talking about 5 years or it's a long time to develop such a program and have it articulate and all of this. I think technology can make higher ed more nimble and addressing the needs of society quicker. I like it.-- Let's get-- there. Doing my best, appreciate it. A couple of more questions before we close out. And you've sort of hinted at it, but I, I'm gonna pose the question in this way. What's one thing that you hope for education in an age of AI? Well, Certainly that the the AI, the use of AI is serving us in our humanistic pursuits well, and that it's used in a way that retains the humanistic character of the educational enterprise, that is. Something that distinguishes education, particularly public education, right, from corporate training or, you know, other variants of of education. So I, I think making sure that an institution has a a values system, and they make sure that either Whoever is providing the technology is aligned with the value system, or in the cases where that might be less tractable, that at least their use of AI is consistent with their values. So every time a new technology comes out, particularly with AI we ask ourselves, OK, well, what's unique about us that still technology can't do? I, I think the moment here is, well, remind yourself what your human values are, right? So that when AI that somewhat resembles us comes, we don't. Uh, outsource to it, those, those values, but I think there is a way to do this that actually enhances um the mission and, and the value, executing the values of, of a higher ed institution. Yeah, that's a great answer. I really appreciate you retaining that focus of the humanity, uh, as part of the educational process. I think that's critical for us as administrators to remember the humanity of our students and of our faculty and The people we serve, it's very easy to get caught up in sort of here's another form, here's another form, here some registrations, but like keeping the idea that I'm a human being and I'm providing service to another human being is really important in the way that we approach our jobs and, and I think is unique. I've I've worked for other industries and and it's not always that way. Uh, and so I really appreciate that answer about sort of retaining the humanity in this. Yeah, it's a human development enterprise. Yep. Zach, if you are ever looking for a phase 4 for your lab, let's talk about classroom assignments and how to do the allocation of academic schedule development into appropriately sized classrooms. At some point in the future. That's I'm, I'm putting a plug for classroom scheduling and some future AI development work for you. That's, that is a tangential part of a planning grant we have. So if we get the planning grant, we might touch on that that dream topic. Excellent. Come on back because I've got a lot of ideas. Looking forward to that. Awesome, Zach, thank you so much for taking some time to chat. This is fascinating work. I'm skeptical of sort of AI and its applications to registrar work, particularly, um, partially because of my concerns about privacy, about data controls, about all of those things. It is very reassuring to hear someone who is as engaged in higher education as you have been and continue to be. And making all of those considerations and still moving the needle forward with applying technology to some of the difficult challenges that higher education faces. So I appreciate you sharing the work that your lab is doing, and I wish you and your lab all the best. Thank you so much, Doug. Thanks again to Doctor Zach Pardos for taking the time to talk about the work he and his research team are doing with artificial intelligence in higher education. It's important work and I hope that we're able to realize the enhancements to administrative processes, to reduce obstacles for students and to support the overall goal of economic and social mobility that is the promise of higher education. Thanks very much for listening. I hope that you have an opportunity to get away for a bit this summer either on an actual vacation or maybe a staycation or even just a long weekend. It's important to work hard and I know that you do, but it's also important to take a break now and then to recharge. Read a book, join a choir, audition for community theater, go for a hike, do whatever works for you. Until next time, drink some more water, stretch your legs. I'm Doug McKenna, and this is for the record.