Would you trust a robot doctor?

Would you let an AI doctor diagnose you and write your prescription? Sit with that for a second, then hear the other side.

In this episode of Catholic Futurist, Benjamin Crockett sits down with Michael Pencina, director of Duke AI Health, and Louis Kim, who moved from tech into healthcare and now works on spiritual care in hospice and palliative medicine. The same tech that feels wrong in your own clinic could put 50 hospitals in regions that have none. They wrestle with what we lose when no person is in the room, and the lines Catholic healthcare won't cross.

Timestamps:

• 0:00 Intro
• 2:27 What makes Catholic healthcare different
• 4:56 The hardest debates
• 7:59 What AI does in hospitals today
• 10:37 Lines AI should never cross
• 13:00 The future of care
• 14:55 AI in hospice and spiritual care
• 16:27 Changing the economics of care
• 19:18 What the Church should do now

Transcript

SPEAKER_01 0:00

Some of the debates got really vigorous when it came to applications in what I would call limit cases or kind of extreme cases. So for example, um, let's imagine a hospital of the future that is so automated that a patient could walk in, get diagnosed, prescribe medication, and discharge without ever encountering a person. Technically possible. Okay. Now it sounds like kind of weird and spooky, and we wouldn't want that. But what if that level of automation and uh efficiency allows you to place 50 of these hospitals in the developing region in a you know an indigenous, right? Um, versus, oh, uh, we need to staff it with at least two people. Well, okay, well, you know, you then you can only do 10. And uh we we had you know divergence, and it really then forced us to think uh for everybody, regardless of the position, but why does that bother you? What what is missing?

SPEAKER_00 1:07

If I can just maybe start off and just to both of you, um, what were the conversations like these last two days?

SPEAKER_02 1:13

I think very productive conversation. So our group, about 15 people with very diverse expertise, and really everyone authentically engaged in trying to think where will the AI take us, and how we can make sure that it takes us in the direction that helps preserve or maybe even amplify human dignity and the common good. So very much aligned with the uh exhortation that Pope Leo included in his letter.

SPEAKER_01 1:53

I'll just add uh Michael referred to the group and the quality of the group. It was very diverse. So we had um an executive from a hospital system, um, executives from insurance companies, uh medical school professor, theologians, an ethicist for a Catholic hospital uh trade association. So it was quite rich, it's about 13 people. What was really striking uh is with that experience, taking uh Catholic social teaching uh principles, principles from the intellectual tradition like human dignity, and how does that apply practically within these separate contexts for these individuals to make it real? And that was really powerful.

SPEAKER_00 2:32

Can you can you both share maybe briefly like what separates Catholic healthcare from like general health healthcare?

SPEAKER_01 2:38

Maybe I'll just quickly respond and then Michael, you can add um uh clearly um uh a Catholic sort of sensibility versus uh secular sensibility does have this primacy on human dignity and sanctity of life. Um, also acknowledges suffering and um as a formative experience. So the way that might translate then into the application of AI is as things get more mechanized, as AI gets more powerful and supplants human capabilities, how do you preserve um the human connection in that uh healthcare sort of event or uh or activity, um, especially not just now, but like decades from now. And so we wrestled with how do you define that sort of bright line boundary where a human needs to be involved when computers and AI and robots will be, you know, arguably more effective.

SPEAKER_02 3:34

Right. Um I'll quote one of our session participants who said that the purpose of Catholic healthcare is to extend the healing ministry of Jesus Christ, right? And I think that that's a really important context. And I think another participant emphasized that in the Catholic social teaching, and as it pertains to healthcare, the spiritual and the physical are part, that's who we are as humans, and you cannot separate it, right? So that has important implications for the use of AI. We cannot say just put the spiritual aside and look what AI will do for the physical. We have to keep the holistic context in mind, right? And that kind of underpins human dignity. So, but that that's that's the mission and the difference of Catholic healthcare.

SPEAKER_00 4:32

What did you guys find the most spirited debate?

SPEAKER_02 4:35

I think an interesting question we were trying to address, and I let Lewis comment because he really uh was going deep into it. The the question was the role of humans, people in the context of the healthcare enterprise, right? So the the very interesting discussion was how much can we delegate to artificial intelligence and what are the tasks that remain intrinsically human that you need, you need a person, right? Full recognition, AI is not a person. So that's important to set aside very, very clearly, right? So you need human beings interacting with human beings in the context of healthcare. But I think where some differing views came is how far we can allow artificial intelligence to go versus where the human person, uh, human being comes in.

SPEAKER_01 5:32

Yeah, just uh add to what Michael was saying, the some of the debates got really vigorous when it came to applications in what I would call limit cases or kind of extreme cases. So, for example, um, let's imagine a hospital of the future that is so automated that a patient could walk in, get diagnosed, prescribe medication, and discharge without every encounter person. Technically possible. Okay. Uh now it sounds like kind of weird and spooky, and we wouldn't want that, but what if um that level of automation and uh efficiency allows you to place 50 of these hospitals in a developing region in a you know an indigenous, right? Um versus, oh, uh, we need to staff it with at least two people. Well, okay, well, you know, you then you can only do 10. And uh we we had you know divergence, and it really then forced us to think uh for everybody, regardless of the position, but why does that bother you? What what is missing? And that led to then a list um that was generated. Well, you know, some people said, well, what's missing is um uh human accompaniment, uh a divine mediation, um, relationality. Some people said, you know, we can't really trust uh AI to be f uh without any human verification or consent. So there's a long list, but that was where we had uh some fierce debates. And and it's sort of not surprising because uh, you know, you can have a list of principles that's called principalism, but when you get to the actual application, that's where things get quite difficult.

SPEAKER_00 7:16

When you when you both look at healthcare and you know applications for AI, where is it actively you know helping right?

SPEAKER_02 7:23

So we spend a little bit of time looking at the kind of three goals that many health systems have. One is to increase access to care. Access is a major issue. Developed world, developing world, there is an issue of access, affordability, healthcare in many countries like the US is too expensive, people cannot afford it. And then reliability, right? Receiving high quality care that's good and safe in this in this context, right? So those are those are the goals. And then we kind of juxtapose the Catholic social teaching principles and looked at examples of applications of AI that would satisfy one or more of these goals. So to give an example, ambient scribes are getting very popular, right? So when you come to the doctor, when I take my child to the pediatrician, it used to be that the poor doctor had to type in and talk to me or my child, right? Multitasking. Now there is the ambient scribe that listens to the conversation and creates the clinical note that the doctor will review afterwards, but it frees the clinician to have a face-to-face direct conversation. I like this example because it shows how AI is rehumanizing the interaction in the healthcare setting. Now, we also have to be careful that the clinician reads the note afterwards, right? That there are there are not errors, but that's just one example of positive application. And in general, there was consensus in our group that back office tasks, the tasks that do not that either enhance or are not affecting the patient-clinician interaction, having them delegated to AI, that's a positive, right? Filling out forms, all the things that we have nurses do or other healthcare professionals that have to do with the operation of healthcare, but not the encounter, I think AI can be very helpful, and that's viewed as a positive.

SPEAKER_00 9:39

What lines do you guys see clearly the church should take to make sure that we don't cross that line in terms of AI, you know, replacing that human care?

SPEAKER_01 9:48

Uh, an example uh was what we termed non-impersonation. So even if AI, an AI entity could feel like a person, um, it should be very clear to the patient uh that it is not a person. It's like a red light on a video recording. Um that was one example of kind of a bright line. Um another um, you know, let's go back to this personless hospital example. People acknowledge, you know, you may want to, it may be the expedient thing to do, proper thing to do, but at a minimum, as an example, uh, we should provide everybody an option to talk to and meet a real person in all of these settings. Uh so those are two kind of examples of sort of edge cases and and where you know Catholic uh sensibilities, you know, uh come into play. Can I just add one thing about the question about um um just the role of AI? I think um, you know, for people not in healthcare systems, they should uh uh be aware, this really natural think of AI and its concerns in a patient uh clinician interaction. But of course, AI is suffused everywhere. Uh administrative tasks, um uh data, uh storage. Uh it's just not only the patient uh uh doctor interaction. Uh obviously diagnosis, development of drugs, it's everywhere. Uh we uh handed out a sheet of probably 50 areas where AI is suffused uh in healthcare. And for most people, it's not visible to them.

SPEAKER_00 11:31

What are you both most excited about in terms of the the future?

SPEAKER_02 11:35

So I think there is a lot of inefficiency in healthcare, right? The costs are rising, the quality as measured as life expectancy and health is not going up. So, right, there are issues of national debt related to the cost of healthcare. So I think AI can be very helpful. I mentioned the back office um interactions, making it more efficient, again, aligned with the good of patients, right? So I think there is, we can believe that we might be able to improve care, increase access, and reduce cost with the use of technology. I will say I think the next frontier is more direct clinical applications. And interestingly, um, pre-pandemic, pre-GPT, you saw a lot of development focused on the clinical, better diagnosis, better prediction, right? Going into the uh preventing disease or curing disease. Post-pandemic, with the strain on healthcare and emergence of GPT-generative AI, which lends itself to the more operational tasks, it shifted more to the back office operations. And that's where the focus is. But I hope that down the line we'll move into the diagnosis, right? Being able to catch breast cancer that's not visible on mammography because AI is able to use it, right? Those are the examples that will be through prevention, will be saving lives, identifying new treatments and medicines, new cures that will come in as a result of having AI. So I think it's a staged process, but with tremendous positive potential.

SPEAKER_01 13:30

Um instead of uh answering what am I excited about, uh, because I'm not a formal practitioner, I'll I'll um address uh what I'm curious about uh with AI and healthcare. And uh my personal focus is, and now that I'm transitioning from the tech industry into the healthcare field, it's in the area of hospice and palliative care, end of care, into life care. And then and specifically, I'm curious about um spiritual health. Um uh I'm one of several uh advisors to a physician, uh George Washington University, who uh Dr. Krina Christina Pukowski, um, who leads an initiative to inculcate more spiritual care uh in clinics and hospitals, essentially raising the chaplain at a peer level with doctors. And she has a lot of research and and methods to measure spiritual distress, to inculcate more um spiritual health. Uh, what is the role in AI then? Could AI, through this uh ambient listening, uh listen to a conversation, detect or in advance uh spiritual distress? Um a controversial area is uh in facilities where there aren't enough nurses or chaplains, uh, whether we lack it or not, there are already bots uh that um in even little robots that that provide some um facsimile of a companion. Uh in and what are the issues with that? So those are things that I'm I'm I'd say curious about.

SPEAKER_00 15:02

I have one final question here. If you had one thing you wanted the institutional church or the or the Vatican to to take seriously with regards to healthcare and AI, what would you want to see happen?

SPEAKER_02 15:13

So recently I have studied rerum novarum, right? The encyclical written at the time of a technological revolution from over 100 years ago. And it was very, very influential, right? And we can see the history mean there to different places, but the spirit of rerum novarum helped shape it and and and helped kind of avoid the extremes of communism on the one hand and uh unencumbered capitalism on the other. And we're not in a perfect society, but there is some kind of middle ground here, right? So I'm thinking that there is an opportunity, and with Pope Leo choosing the name Leo, I think pointing out in that direction. I think uh the society at large, as one of the participants commented, this is the opportunity for the Catholic Church to lead. There is a void in the ethical space, right? So having a well-thought-out encyclical on AI, and again, it's developing, it's it's changing, right? It's not an easy document to be writing, I think would be would be very, very valuable for Catholics, but I think for the world at large.

SPEAKER_01 16:31

I would add um technical literacy that's quite deep and acknowledges uh I think um a lot of concerns around AI, even just positive statements about AI, uh, is really um addressing chatbots. Uh when you really kind of dig down, like what what what you know, what's your vision of AI, your concerns, and what what what are you envisioning? They're really talking about chatbots. Um an AI is much broader than that, and it in five, ten years from now, it'll be you know even broader than that. So without that kind of deep level of technical literacy that understands how AI works and different domains, um, things stay at kind of a platitudinal level, and that would be unfortunate. Um, you know, we have Vatican scientists and bioethics and even astronomy. Uh, we need to demonstrate uh similar level of literacy for AI. The second thing I would add is um is go below the principles and uh have case studies and um investigations when these principles are actually applied in really tough limit conditions. As you go through the different uh Catholic social teaching principles, some of them will force decisions that are at odds. It's each other, at odds at each other. And having case studies and helping people discern that uh is really important.

SPEAKER_00 17:54

Well, Lewis, Michael, this has been wonderful. Uh, thank you so much for the time, for the conversations and the incredible work that you guys are doing. Um, I hope we can do this again at next year's Builders AI Forum.

SPEAKER_01 18:05

Thanks for the great questions. Thank you for having us.