Episode 6: The Next Generation Classroom: AI-Powered Clinical Learning

Hosted by Matt Calendrillo, Chair of ACAPT’s Clinical Education Commission, the episode features Benjamin Stern, MS, DPT, of Tufts University School of Medicine, who shares how AI-powered patient simulation is expanding opportunities for student practice and skill development.

Listeners will gain insight into how programs can create customizable patient scenarios, provide real-time feedback, and integrate AI into existing curricula.

Transcript

SPEAKER_00 0:00

Welcome back to FlexCast, where ACAPT explores the ideas and innovations shaping the future of academic physical therapy. I'm Emily Weber, Director of Marketing and Communications at ACAPT. Today's episode dives into one of the most talked-about and sometimes misunderstood topics in education right now: artificial intelligence. Today, we welcome Dr. Ben Stern from Tufts University School of Medicine, who has been working at the intersection of clinical education and AI-powered patient simulation. His work focuses on helping DPT programs expand clinical practice opportunities for students, all without dramatically increasing faculty workload. And our host, ACAPT Clinical Education Commission Chair and co-founder of Live Every Day, Matt Calandrillo. Welcome to Flexcast.

SPEAKER_01 0:55

Thank you, Emily, both for the opportunity to host and have this great conversation with uh with Dr. Stern Ahead. I feel like I have a new career in front of me. I think uh I feel like Amy Polar today. So perhaps this is a new career move and switch for myself. I will host anytime, Emily. I'm glad to have the conversation. And this couldn't be more fitting. I think from a from a clinical education perspective, we're constantly trying to balance access to both meaningful patient experiences with quality clinical education in real time, with faculty burden, bandwidth challenges, cost placement concerns. It's it's a pressing topic, and I'm eager to see where this conversation goes today. We recently had a listening session that we really delved into some of these same concerns, and it's I find it's just coming at a great head. Some of the conversation we recently had was around inpatient placement capacity and the challenges around that, and some of the solutions that that group was coming up with, I think we might touch on in this conversation itself around how does AI and simulation perhaps meet to decrease again those burdens of cost, travel, availability, and on all those sorts of concerns. So, again, welcome, Dr. Sherno. I'm eager to have this conversation. We've broken it up into a few different segments. And I think the first one is let's hear about you. Like what drew you to AI within physical therapy education, and what kind of gaps do you think you're trying to address within your work for ClinEd?

SPEAKER_02 2:15

Yeah, well, thanks uh both of you for having me. I'm super excited to be here. Um so uh for me, um, I sort of came into AI sideways. So I was uh around 2015, uh I was um doing some research using machine learning to explore things like nonlinear dynamics and causality and tipping points. And somehow I ended up teaching 70 plus DPT students. Um, and I just I felt like I couldn't give them enough quality practice or enough individualized feedback. Um, and so I was a new faculty and feeling pretty self-conscious about teaching. And I was thinking about these questions like um, are my assessment items aligned with what I actually want the students to learn? Am I giving the students enough chances to practice what we can what we talk about in class? Um, can I give my students maybe individualized feedback at a scale that's realistic? Um, so for me, it was really about taking skills that I had been using and trying to build tools that would help me become a better faculty member.

SPEAKER_01 3:22

Fascinating. I love the word sideways there because I think like we're all being pushed into it or being touched by it, um, whether we're we're we're seeking it or not. So I like your intentionality around that. Um perhaps give us some some of those specifics around the call them gaps, but just the implications that it has within the clinical education learning environment.

SPEAKER_02 3:46

Yeah, I I think the the thing that really clicked for me with um well when when chat GPT came out, when the first real public-facing version came out, was uh it seemed like large language models were kind of like Legos that the chat bot, the the vanilla chat GPT or co-pilot or Claude, it's sort of the the picture on the box. And I think the real value is when you start shaping the pieces around meeting your own goal, right? Like like solving a specific teaching problem. Um, and and I should say one of the things that's been helpful is I started taking courses in the computer science department at Tufts a couple of years ago. So that's impacted how I used the tools. Um, and so the the tools I was trying to build were were to address these gaps, um, like more repetitions for students, um, better feedback, um, a safe place to to practice really hard conversations with patients. Those were sort of the main things I was thinking about.

SPEAKER_01 4:47

Yeah, fantastic. And I think that that paints a great picture with the Lego blocks for sure, and using them as tools to supplement what we're already trying to um to complete within that learning curriculum. I love it. Um, one of the challenges that I think we're all facing in ClinEd is giving students reps, quality reps, and then how to expand upon that that depth and breadth kind of component of it while still balancing all the needs, right? How do they psychomotorly learn? How are we increasing their cognitive um capacity within it? How are we affecting that that effective domain? So what is this AI kind of model in in clined or in that educational realm? What does it actually like look like in in practice?

SPEAKER_02 5:28

Well, I think the the easiest way to picture it is maybe um in our lab. So we have several small groups who are working at different stations uh set up around the room, and each station is staffed with an adjunct. And oh, I don't know, you can imagine anywhere from four to six or seven students at each station. And um each group of these students is interviewing a simulated patient. Um, and for example, that that patient could be a college athlete, and that athlete's afraid that if she tells the truth or if she shares too much information, she could be pulled from her sport. And so the students see and they hear the patient. And at the same time, uh, I or the adjunct can actually adjust the patient's emotional tone in real time.

SPEAKER_01 6:14

I love it. Tell me a little bit more. What is that what does that look like in terms of that the alterations or the tweaking that can happen in real time?

SPEAKER_02 6:21

Um, so we can take a patient, uh, for example, who is initially uh really anxious. And over time, we can take that patient's personality and uh we can make them a little bit more frustrated, right? So the students trying to pull information uh in a way that they've read about in their textbook or that we've lectured about in class, and now they're being confronted with this additional data, right? This additional stuff that's coming from the patient in terms of the these emotional cues. And so now they're grappling with these two different pieces of the patient. Uh, and so it's been a really, really exciting experience for us, sort of watching how that changes, how the student begins to collect information.

SPEAKER_01 7:08

I love it. And then um what kind of time frame were we talking around? Is that those are those subtle changes happening over the course of a one use? Are they engaging with it multiple times and that's where they're seeing that patient progression, that patient challenge, or perhaps a bit of both? How do you guys implement that?

SPEAKER_02 7:24

Um, yeah, actually over a single use. So the the thing that really told me that that this kind of crossed a threshold was was watching how nervous some of the students got. And it wasn't because stress was like our goal that we were trying to set up some kind of gotcha moment. Um it was really because I think the that interpersonal pressure that can sort of manifest in the clinic was showing up inside this safe environment, right? Our lab. And so um we've seen students apologize to these simulated patients after a difficult exchange. Um and and the key teaching feature is really that the LLM, the language model, provided instructions so that the patient gives us uh, well, we gave the the language model instructions so that the patient gives us more useful information when the student is empathetic, when the student's organized and when the student's patient-centered. And the uh the the simulated patient gives less useful information when the student is not empathetic or organized or patient-centered. Um, and so that's the part that I hope really mirrors that real clinical communication much better than a paper case or something that they might find in a PDF.

SPEAKER_01 8:34

Yeah, very, very interesting. And then talk about around scale. How are you guys currently incorporating it within your program? I'm hearing groups of you know, four to six students. That sounds like some different integrated clinical experience kind of models. So, how do you guys incorporate it?

SPEAKER_02 8:48

Yeah, so um, Matt, we we uh use this in a few different ways. Um, so in synchronous sessions, because we're a hybrid program, we we work through a case together. Um, so in my primary care class, I will put together a patient case and and in class, I'll let the students decide what we're gonna ask the patients next. Um, like I said, in lab, we have small groups who interview patients, uh, and those patients have different personalities and different circumstances, and that makes information gathering harder in realistic ways. And then asynchronously, um, students can keep practicing on their own time, right? So some students end up completing more than 20 hours of simulated patient interviewing over a semester, and that's way more than we could ever provide one-on-one with faculty or or standardized patients. So, you know, I think in a nutshell, there there are a few different approaches that we use, right? We use them during synchronous sessions, uh, we use them during labs, and then uh asynchronously.

SPEAKER_01 9:51

Yeah, that's a that's a tangible impact. That's that's notable across uh across that many students. I love it. Just to just to pivot a little bit, because I think um you used some some big fancy words around technology that I think might feel daunting or scary to individuals. So um perhaps break that down a little bit. I'm assuming there's not a heavy need from a technical understanding of the coding or the backside of it. What's that, what's the interface or use look like from your vantage point?

SPEAKER_02 10:17

Yeah. So I think from from a um at the simplest level, you can imagine just a text-to-text kind of interaction, right? So a student maybe opens up a chat window and types something like, What brings you in today? And then the patient on the other side has a diagnosis and a backstory and a personality. Um, and maybe that digital patient has another person in the room, right? A caregiver or a parent or a child. And then from there, the conversation just unfolds, much like uh an interview. Um, so you know, again, much different than a worksheet.

SPEAKER_01 10:54

Fantastic. Fantastic. And um, maybe I'm going a little off topic here, but you're leading a webinar upcoming. Um and some of the content I'm assuming is going to be interactive and the ability for some of the participants to kind of see this in action.

SPEAKER_02 11:10

Yep, absolutely. Yeah. So uh that webinar is really going to be based around explaining to um the attendees how they can put this together themselves. Um, but we're also going to show them how to integrate features like voice. Uh so one of the things that that really struck me, and this actually came from the PA program at Tufts. I had been uh speaking with folks there about creating these sort of digitally simulated patient interaction experiences. And they said, you know, the feedback from the students was that uh not so much that the students wanted to be able to speak to the patient, but they wanted to be able to hear the patient speak. So one of the first things we integrated was this speech component. And now it's gotten to the point that even with these sort of um over-the-counter chat bots like Chat GPT or Copilot, um, it's really not much of a leap to uh go from typing to your patient to actually speaking to your digitally simulated patient. And so we'll go over how to create those and and see how um how it sort of works underneath the hood.

SPEAKER_01 12:17

Exactly. That's what I was hoping was going to be part of that content, which is great and I think should get some people um both excited and perhaps off the bench to take those first steps to get started. So, how did your program make that decision and make those kind of first step leaps?

SPEAKER_02 12:32

Um well, I think uh, like I said, the the program sort of made the decision because I I needed I needed tools to to help myself. And so I just uh sort of took the position that okay, I need this, I need to build it, and I'm gonna build the things that I feel um are going to be most useful to the areas where I feel like I have the biggest weaknesses. Um and then from there was really having conversations with program directors and faculty and sort of seeing what their needs were and and um then sort of tailoring each tool to faculty needs, right? So these digital patients um aren't only being used in my courses and primary care. So um they're being used in uh the professionalism classes, they're being used in neuroscience. I think we're using them in musculoskeletal practice. Um the the underlying blueprint is pretty similar across all of them. And then it's really just sitting down with faculty and saying, hey, you know, this is sort of how it works at the base level, and this is where you can can sort of tweak these things uh to meet your own needs. And I think a lot of it too is being willing to sit down with faculty and listen to to what their needs are and what they're struggling with, and figuring out, you know, number one, whether these tools can help with that. And if they can, um how to help them figure out what are the the specific pieces of the tools that they can tweak to make them most useful for their particular needs.

SPEAKER_01 14:08

Yeah, great explanation on that. That's very helpful. Pizza, a good picture for it, especially when you were talking um previously around just like the scale and the impact upwards of 20 hours that can happen, where the the request on faculty load to be able to do that would be uh insurmountable, right? So you're talking around meeting the the needs of the faculty, the students, meeting objectives. I think the next kind of natural progression would be how's it work on the back end in terms of feedback, both for how it's all doing as well as feedback to the students? Because that also is often a kind of a timely um necessity and also a time crunch that happens within clinical education. So, how do you guys manage um just overall feedback within a model like this?

SPEAKER_02 14:52

Yeah, so I I think um the interesting piece with this uh that I built in is sort of this um, and and you can do this a number of different ways. So this is just an example. Um, but I built in a set of instructions um that I'll show in the webinar too, um, that are composed primarily of that echoes model. And those instructions run underneath uh so the student isn't really aware of anything going on during their interview. Um, but what's actually happening is that that language model is keeping track of everything that the student says during the interview. Um, and so uh at the very end, when the student types in N simulation, uh the model then can spit out um exact quotes from students and exact things from the interaction. Um and so the student has a really a really good idea of of the things that they did well, maybe the areas that they struggled, and those are actually tied to specific quotes from the interaction with the patient.

SPEAKER_01 15:54

And I'm sure the the students appreciate that in terms of how timely that is. Have you seen an impact with um with that?

SPEAKER_02 16:00

Yeah, absolutely. Um, so uh we we uh we we've had a few interesting things happen, but but I think one of the things that that kind of struck me was that um uh a student said that uh the scoring really made it feel like like a game without the pressure of having to perform in front of the whole class, that it made them want to do it again and get a better score. And so I think that's sort of what immediate feedback unlocks, that the students are willing to practice more when the feedback is fast and when it's specific and when it's tied to what they actually said in a session. So in our setup, each conversation is scored in the background against this structured um interviewing rubric. So when the student types and simulation, like I said, they get this breakdown with these direct quotes from the conversation, what they did well, what they missed, and and suggestions for for next time.

SPEAKER_01 16:53

Wonderful. Again, yeah, that feels like that's checking a lot of meaningful boxes all at the same time. I think we we talked of, or I talked of of hesitancy before, um, which I think is it happens with any new technology that's out there. Um, but there's there's also just global concerns around AI and and integration like this. So, what concerns are you currently hearing from faculty within your staff or those others that you're educating?

SPEAKER_02 17:16

Yeah, the the biggest concerns I hear from faculty are um reliability, uh privacy, bias, um, the fear that maybe students are going to use this to bypass or get around the need to focus and and to think really hard. Um, so those are serious concerns. And my view is that these tools really work best when they're focused, um, when they're supervised, and when they're positioned as educational supports rather than authorities, right? That that they're not sort of the last word. Um, I think the the overused line right now, keep the humans in the loop, is actually really appropriate here. And when we ground the simulation in a specific case uh with a specific purpose and a specific rubric, it becomes much more useful, but it still needs that oversight. And I think that that some faculty skepticism is really healthy. I don't think the answer is to engage in abstract arguments. Um instead, you know, I really uh I've had really good success showing what a well-run session actually looks like and having faculty observe and and sort of getting their feedback and their response.

SPEAKER_01 18:32

Yeah, that sounds like a wonderful balance um between both sides of that scale. Are you noticing any common misperceptions when you're introducing this, perhaps to a faculty member that's a little more hesitant around the process?

SPEAKER_02 18:44

Yeah, I think the the biggest misconception is that this is about replacing faculty or replacing clinical education. And and I definitely don't think that that's the case. I think of it as more of like a faculty multiplier and uh a practice multiplier. Um I don't I don't think anything here replaces the clinical instructor watching a student uh work with a real patient and seeing how that student brings their humanity to the encounter. Um I think instead what it can do is sort of let students walk into a clinic having already handled um dozens of structured interviews, including really, really difficult ones, in a safe environment. Um, I think the other misconception is that students are automatically going to learn well just because AI is involved and they won't. I think the design matters. And if the tool becomes a shortcut to the answer, I think it can absolutely make learning worse. Um if instead um it's a training partner with a rubric and with boundaries and with feedback, then I think it can increase practice and reflection and and self-correction.

SPEAKER_01 19:52

Yeah, you kind of stole my next line, which is around like where should programs begin to be thoughtful if they're if they're investigating, kind of starting this within their curriculum. Can you kind of expand on that a little bit?

SPEAKER_02 20:02

Yeah. Um, yeah, I think I think that's actually it's pretty clear that that first, um, you know, you want to use appropriate tools and keep in mind things like HIPAA, um, right? That we don't want to put real patient identifiers into these systems and and we don't want to rely on these sort of model generated scores as the sole basis for like high stakes kind of grading. Um, I think we also want to think about things like access inequity, um, that if some students have better tools than others, then maybe what's happened here is you've built a two-tiered experience, which is not great. Um, and I think we should really invite students to critique the model when it falls short, and that we we shouldn't try to sort of brush it under the carpet. We should really talk about that stuff and keep asking for feedback from students and from faculty. Um, I think the the the tools that that we use um to help our students have changed a lot, especially over the last couple of years. But I don't think our responsibility as educators has.

SPEAKER_01 21:06

And well said, and and I'm I'm not asking you to be a wizard here, but where do you see the future of it, both with, I guess, within your program and then within some of these evolving AI technologies? Because to your point, it's different than it was just a short time ago.

SPEAKER_02 21:20

Yeah, I think some of the things that have been really striking to me are sort of the the pace at which these things have changed, um, not only in terms of the uh the ability for the models to produce reliable and accurate answers and more human-like answers, but also the interfaces. So the latest interface that we've developed uh for our programs at Tufts, um, in addition to being able uh to speak with the patients, the students can actually see a patient. Um, and so we can choose whether this is a male patient or a female. Female, a young or old, uh, from different backgrounds, and that patient's physical response actually reflects their emotional tone. So as we sort of modify or nudge these emotional sort of levers uh during the the interview, the physical response actually changes. So I can imagine over time that these interactions will become more and more realistic. Um, so it's it's it's really an exciting time to be sort of developing this stuff and watching it change so quickly.

SPEAKER_01 22:33

Yeah, and we were kind of talking this before the the cast started. I think um, in my own experience, students start full-time clinical experiences, let's say that, and come at it thinking that, because we asked them directly, like what are those skills you're looking to hone in on? And it always lives in that psychomotor zone. That's where they think the the art of physical therapy exists, which is the how much pressure, where do I put my hands, that piece of it. And I think they're also coming off of a heavy academic program where, although it's not solely what we're focusing on, there's a lot of cognitive load there. So there's a lot of cognitive thinking. But we all know it's all about being human and how you can connect with an individual in that affective domain. So what you were just touching on there feels like the future has deeper and deeper inroads to potentially tapping into that in a meaningful way. I guess my question is do you find um do you find holes in the work as it currently is around managing the affective domain with the levers that you have to pull? And then how do you how do you find yourself, I don't know, working in that environment, trying to make sure that the student is appreciating the whole situation?

SPEAKER_02 23:39

That's that's been a really um unanticipated sort of push and pull for me. Um, because as I mentioned earlier, you know, you can actually see students becoming nervous, um, especially once we engage the the voice interaction, but even more when they could actually see the patient. Um, their whole demeanor changes. And for example, in lab, you know, none of the students want to be the first one to speak, and they'll all look at each other for a while before they say anything because they don't want to say the wrong thing. And and so on one hand, you know, I feel like, man, I really don't want to make these students any more anxious. But at the same time, you know, that really does reflect the clinical reality. And and one of the things that had sort of spurred this development for me a couple of years ago was was speaking with faculty, and we were talking about when students go on their clinicals, that one of the things they struggle with um are sort of the real things that occur in the clinic. That, yeah, this patient has shoulder pain, but they have a bunch of other stuff going on in their lives. And, you know, sometimes they behave appropriately, and other times they they behave incredibly inappropriately. And that's just sort of the nature of being a clinician. Um, so yeah, it's it's a real push-pull to watch.

SPEAKER_01 24:56

Yeah, and you you hit the nail on the head there. There's there's differences, the mix is always going to be unique. There's unique um patient personalities, there's unique uh student personalities, and how those two kind of coexist or or meld. And it sounds like you're touching on it a lot within this work as well. I'm sure we could talk for forever around this and this content. I'm excited for the upcoming webinar. Um, be sure to kind of push out within our clinical education community that that's happening because I think that's a loud voice that should um should engage in that work for sure. Um, anything else on the top of your mind that you'd like to share that we didn't touch on yet?

SPEAKER_02 25:29

I think one of the things I would I would say is that um, you know, for folks who are listening, what I'd ask you to try to do between maybe now and the end of the week is is pick a case that you already teach, um, open up your your institution's approved AI tool, right? That's important. You don't want to, you don't want to just throw um your your information into just some some random uh over-the-counter chat bot because you don't know how they're gonna use it. So, you know, usually if if you're at a university, they'll have a tool that's approved, maybe it's co-pilot or they'll have an enterprise license for something else. But anyway, um pick this case and then run it live with your students in a single class session, and and that's it. And and if you do it, email me. I want to know how it went. And I think um, you know, I think this is it's a really interesting um phase that we're in right now. And uh you just sort of have to jump in and trust that it's gonna be okay. You're not gonna break anything. I think that the realistic starting point here is that you don't have to build the whole system on day one, that one case taught using this approach is really enough to start. And what we're aiming at, I think eventually the real game changer isn't just making the patient talk, it's manipulating the patient's personality and the context, right? The the patient who doesn't trust doctors or healthcare professionals, or the athlete who's worried about maybe losing a season, or the patient whose child is running around the room distracting everybody, or the caregiver who's answering all your questions on the patient's behalf. I think that's where students stop uh reciting questions, and that's where they actually start learning how to actually manage the encounter.

SPEAKER_01 27:10

That's wonderful, uh Dr. Stern. That's wonderful. I think that feels low stakes and an easy entry point for it. It's been an absolute pleasure speaking with you, and I again look forward to the webinar.

SPEAKER_00 27:20

Clinical education has always evolved, and what we're seeing now is another step forward in how we create meaningful, scalable learning experiences for students. Dr. Stern, thank you for sharing your work and giving us a practical look at how AI can support both students and faculty. And thank you to our host, Matt Calandrillo, for guiding such an impactful discussion. If you're interested in exploring this further, be sure to check out ACAP's webinar on AI Powered Patient Simulation, the faculty multiplier scaling clinical practice with AI-powered patient simulation. And thank you for listening to Flexcast. Until next time.