The Signal Room | AI in Healthcare: Strategy, Governance & Ethical Leadership
The Signal Room is the podcast for healthcare leaders implementing AI in healthcare with strategy, governance, and ethical leadership. Hosted by Chris Hutchins, founder of Hutchins Data Strategy Consultants, the show goes deep on AI strategy for healthcare, AI governance in healthcare, healthcare governance, ethical AI leadership, and responsible AI development — with CMIOs, chief AI officers, and operators driving trustworthy AI systems, clinical AI implementation, and AI compliance in healthcare across real-world health systems.
Each conversation unpacks healthcare AI ethics, healthcare AI risks, AI bias in healthcare, algorithm bias healthcare, health tech governance, AI implementation for healthcare leaders, ethical leadership in AI, and the practical realities of responsible innovation in healthcare.
If you are an AI strategist, healthcare executive, CMIO, chief AI officer, or AI governance leader committed to ethical leadership in AI, The Signal Room equips you to lead AI transformation effectively and responsibly. Join us for AI risk management in healthcare, healthcare data governance, AI strategy for executives, executive decision making in AI, and the trustworthy AI systems shaping clinical decision support and the future of healthcare AI.
The Signal Room | AI in Healthcare: Strategy, Governance & Ethical Leadership
From AI Strategy to Execution: Ethical Leadership, Trust and the Operational Reality of Healthcare AI | Brian Sutherland
Use Left/Right to seek, Home/End to jump to start or end. Hold shift to jump forward or backward.
AI strategy for healthcare stalls when it collides with operational reality — Brian Sutherland on ethical leadership and moving healthcare AI from plan to execution.
Healthcare innovation leadership rarely stalls at the strategy layer. It stalls when AI strategy for healthcare collides with operational reality, leadership alignment, and the workflow assumptions the plan never questioned. Brian Sutherland, an AI product manager who built Humana's first member-facing intelligent virtual assistant, joins Chris Hutchins to examine why AI pilots do not scale and what AI leadership strategies look like when they survive first contact with the bedside.
What We Cover
- Why AI pilots stall at the 80/20 trust problem, and what that reveals about leadership alignment in healthcare organizations
- The adoption gap where technology outpaces human change, and why friction is the most underestimated variable in every AI deployment plan
- How to treat an AI system like a junior employee, with structured onboarding, retraining as policies change, and designed failure response, instead of a finished product
- Why AI governance is most useful as an enabler, not a bureaucracy, and what that looks like in practice
- How diverse perspectives expose AI blind spots, and why resistance appears exactly when those blind spots force a course correction
Key Takeaways
- AI strategic leadership means designing for failure, not just for success. Breaches and errors are inevitable. Organizations that plan for them absorb the hit; the others do not.
- Trust in healthcare is declining even as reliance on AI increases. Any AI leadership strategy that ignores the trust paradox inherits it by default.
- AI strategy consultant work stops being consulting the moment rollout begins. Execution is the real strategy, and the gap between executive approval and frontline use is where most transformation ends.
Frameworks & Tools Mentioned
- 30/60/90 AI onboarding model (treat AI systems like junior employees)
- Humana member-facing intelligent virtual assistant ($7M annual savings, 31% lift in task completion)
- Governance-as-enabler vs. governance-as-bureaucracy
- Human-in-the-loop clinical decision points
- Structured breach and error response for AI systems
Timestamps
- 00:00 Brian Sutherland on AI in high-stakes, customer-facing healthcare
- 01:48 Why AI fails: leadership, workflow, and operating model gaps
- 03:08 The adoption gap: technology outpaces human change
- 03:51 The trust paradox: declining human trust, rising AI reliance
- 05:22 Why pilots do not scale: the 80/20 trust problem
- 07:13 Learn before building: operating the workflow first
- 09:56 The gap between executive approval and frontline use
- 11:54 Governance done right: enablement vs. bureaucracy
- 14:30 Diverse perspectives: reducing AI blind spots
- 17:44 Governance as multi-perspective decision-making
- 21:35 Clinician trust: AI support without damaging relationships
- 24:29 Limits of AI empathy in patient interactions
- 26:48 Designing pause points for human judgment
- 31:16 Treating AI like a junior employee: training and oversight
- 33:35 Expiring approvals: governance must evolve with AI
- 34:54 Designing for failure: breach and error response
- 36:26 Continuous oversight: watching for new blind spots
About Brian Sutherland
Brian Sutherland is an AI product manager and advisor focused on customer-facing AI in high-consequence healthcare environments. He built Humana's first member-facing intelligent virtual assistant, a platform now genera
About The Signal Room: The Signal Room is a podcast and communications platform exploring leadership, ethics, and innovation in healthcare and artificial intelligence. Hosted by Christopher Hutchins, Founder and CEO of Hutchins Data Strategy Consultants. Leadership, ethics, and innovation, amplified.
Website: https://www.hutchinsdatastrategy.com
LinkedIn: https://www.linkedin.com/in/chutchins-healthcare/
YouTube: https://www.youtube.com/@ChrisHutchinsAi
Book Chris to speak: https://www.chrisjhutchins.com
Today in the Signal Room, I'm joined by Brian Sutherland, elite AI product manager and advisor focused on customer-facing AI and high-consequence healthcare environments. Brian built Humana's first member-facing intelligent virtual assistant, a platform now generating more than $7 million in annual savings while improving patient experience, including a 31% lift in task completion and measurable gains in satisfaction. That combination, financial impact and human impact, is rare. His perspective on AI is not abstract. It's shaped by decades of watching his mother navigate a health system fragmented across payers, providers, and pharmacies, where automation often added friction instead of removing it. That lived experience shows up in how he designs systems, understands the manual processes before automating them, pressure tests decisions before scale makes them irreversible. And he builds AI that meets patients where they are, not where the dashboard says they are. Brian advises leaders who are building out or buying AI and need clarity on what will break first, where trust erodes, and how to make high-stakes decisions they can defend later. Brian, welcome to the Signal Room. Been very excited and looking forward to having you on. And I just want to jump right in because there's so many cool things that we're going to talk about today. You know, in our first conversation, it was clear to me that you're the right guy to come talk about some areas that I think are needing a lot more attention than they're getting these days. So let's start with one that's probably kind of a top of mind thing for folks, and I seem to be seeing a lot about this, but where do you see AI initiatives most commonly failing? And is it like leadership alignment, workflow design, operating model, or is it a combination of those things?
Brian Sutherland:Oh, it's definitely a combination. I don't even think I could just say like point to one thing. A lot of that I see really happening is leadership will come in, they'll have a really great and strong idea about what it is that they want to do. Right. And it's the almost the classic story. We have an initiative, it becomes a whisper down the alley game, long-winded translation. But then I also believe that we are also up against a relatively young technology still. There's so much that we still don't really know as far as what disciplines we should be really putting forward into this, how we should be positioning this. And so then, yeah, workflow is yet another portion of this. So it really does come down to all three of these things.
Christopher Hutchins:You point to something that I think needs to be repeated. This is still relatively new. We've gotten so used to things moving at the clip that they're going, six months into something that's still relatively new, we feel like it's legacy to us, and we forget that we still have a lot more growing and learning to do to really get these things to operate and do the things that they're capable of.
Brian Sutherland:It's so true. And the technology may be moving really quickly, but as human beings, it's in our nature. It's just part of us. We're wired for comfort, we're wired to not want to go through difficult growth, difficult challenges. So the technology is moving faster than people are willing to change. I mean, it's a significant friction point, and it's something that whether we like it or not, when you're looking at a sequence of events, you're looking at a daisy chain of processes, you're only as fast as the slowest item in your chain.
Christopher Hutchins:That's true. It's really kind of an interesting dilemma. We've got these two things that are almost diametrically opposite each other right now. So the trust factor in human relationships has eroded significantly, but we're still way too quick to trust technology. And we need to find some places in the middle there, I think.
Brian Sutherland:Yeah, for real. And I think even talking about trust in general, there's also, because we have plenty of sci-fi out there around AI in general, that has really kind of informed us of this could turn into something really terrible, really catastrophic. Not to go into doom and gloom, but everybody's first instinct, I'm hearing my father going, you know, this is Skynet that's coming around. And that in and of itself has sparked a lot of distrust right out the gates. And that's not even the real distrust that we should be having, in all truth. There's so many other layers of distrust that we have to actually cut through.
Christopher Hutchins:That's an excellent point. When you're talking about adoption and whether we do it quickly or not, the interesting dilemma that I see out there is that there's pilots everywhere. Everyone gets excited about it, but then when it gets deployed and we start to get into it a little bit, it seems to stop really being proliferated and getting the traction that it should. So why is that from your perspective? Why do they start to hit some barriers when they get to an enterprise level?
Brian Sutherland:I mean, it's all over the place. It's pervasive, right? The endless supply of pilots. The common pattern that I really start to see does come back around to starting with that trust factor. Usually the big thing I tend to spot out almost immediately is the ambition is larger than what was actually designed in the system. It's not designed to build trust, it's technically designed to give maybe a quick answer. And it maybe covers, I will say, like 20% of the big use cases that are out there. But in trust building, you can't just say 20% and you're done with it, right? The old 80-20 rule. You do have to still care for that other 80 in some fashion. That doesn't mean we have to build everything out, but it does mean that we have to be considering what do we do for that other 80.
Christopher Hutchins:That's a really interesting challenge to try to overcome. I think expectations are something that we don't do well with sometimes in terms of setting them at the right levels. I've been talking to some folks recently about the criticality of some of the roles that are not necessarily put close to the front and center of the process that need to be. And those are the ones that are really going to be helping to drive the execution and help to really make things stick. Make sure people are understanding what the value really is and why it's important that we're trying to tackle these things. Because it's not a shortage of administrative functions that really could be done much more efficiently. We just need to listen to the folks who know exactly where some of the pain points are so that we can address the right things.
Brian Sutherland:And one of the ways, I actually wrote about this recently on LinkedIn, one of the ways that we can actually go about that to really get as close as possible is to become one of the operators ourselves. I actually helped Humana with building out one of their first AI customer-facing AIs that they had out there. It was in their chat platform. And one of the things that I recall from it was I actually took calls for two weeks prior to even working on the project. And the exposure that you get from that, like hearing what the customers are actually going through, in the case of Humana, we'll say members, because it's member experience, patient experience, it actually created a couple of different dimensions of thought. I didn't just see where the pain was, I also saw where it maybe was more appropriate to put that technology and where it was less appropriate to put it.
Christopher Hutchins:That's interesting. I'm thinking back to when I was working in New York. On one occasion I got to go have a tour of one of the office suites where they were taking care of cancer patients. And what was really remarkable to me is that things had changed dramatically over the course of a number of years. And not only did it help me to understand where some of the pain points were, it gave me a much better appreciation for what people are doing day in and day out that are really right on the tip of the spear when it comes to taking care of people. We were just at a very different place. So as frustrating as it can be for people like us sometimes doing implementation because we're not getting where we want to go fast enough, sometimes we need to pause and take a look back and just see how far we've come. And it really makes it worthwhile when you take the time to do that. And you undoubtedly will come across things you had no idea how big the impact was, and it makes the work feel much more rewarding.
Brian Sutherland:For sure. And even funnier too, and I haven't done this in a while, but returning back to that workflow, like you just said, how far we've come. We forget as implementers that we moved the needle and our understandings are all built and predicated off of an outdated workflow.
Christopher Hutchins:Yes.
Brian Sutherland:And we're seeing, coming back to the original question, we're seeing a lot of that, that we have new workflows that we really need to be assessing and evaluating, and then figuring out where it's best to place the technology, where are the friction points within that existing workflow, not the workflows of yesteryear.
Christopher Hutchins:We're talking about the expectation component there for a second here. What do you think gets most underestimated between the time of the executive approval and the frontline deployment? Because there's usually some level of expectation that we've missed somehow.
Brian Sutherland:Oh yeah. And that timeline is going to vary. In my opinion, it's going to vary from organization to organization. Your smaller shops, like startups, it's probably pretty quick. The company is so small, the nervous system isn't really completely blown out yet. Contrast that to a Fortune 500 or higher. All of the things that we take for granted in maybe your standard rollout, you usually have a PMO of some sort, or maybe a change management office. There's a running tally of these are the key essential items of anything that has to be gone through, that has to be done prior to saying we feel confident and comfortable with a production release. Now it seems like that's all been forgotten. That the technology, because it introduces this new degree of speed, the thought appears to be that really the wrinkle was all around how do we get the technology built faster? That was never really the issue. The issue is actually centered more on how do we coordinate all of these different competing audiences together and how do we get them on the same page? What do we need to do to instill confidence in all of these individuals that we're working with so that we can approach a production environment free and clear.
Christopher Hutchins:Yeah, I think some of that kind of leans into governance conversations to some extent. Not from the way that people traditionally think about it. I hate to even say traditionally because it's been such an academic exercise for the most part over the last decade or so that I've been more heavily involved in technology development or even data analytics. How do you see from a functional standpoint where governance really can help be an enabler versus the external factor that feels like somebody's trying to keep us away from things or trying to limit what we can do?
Brian Sutherland:Yeah, right. And unfortunately, I know how you feel about not wanting to even say the word traditionally. Governance in and of itself now has this perception attached to it, that governance slows us down from the speed that we're trying to gain. What is unfortunate is governance really protects it, it has the interests of what you're doing in mind in the sense of enabling for going forward. Governance has to have a very significant play, at least at some point in the process. I'm trying to help navigate a partner right now through trying to get something into production at this current stage, and I won't name names for the time being. But the short story for that particular group is that there's a lot of pioneering going on. And in that pioneering, we have to at some point pull governance in. We have to make sure that when we pull them in, that we are not just allowing governance to turn it into a death by committee. We want governance to inform. We want governance to feel confident that we have been thinking about these different areas that we have to be really mindful of and we have to be careful of. But that we can't do it without them too. That their input is important, it's valuable, and that it helps to then shape and crystallize what are the controls necessary to be able to freely scale.
Christopher Hutchins:You're talking about something that really needs to be understood much, much better. When you're thinking about it as an enabling function, you need to think about the different perspectives you want to make sure you have in the room to make sure that you're asking the right questions, make sure that you're solving the right problems. Oftentimes there's the initial what I would call the survey, or listening session. You take, you identify a few different takeaways. Then people go and will build something in a vacuum and come back, and there's a surprise waiting for them that it actually is not meeting the need that they thought they were trying to address. It's really an involvement in a relationship that has to actually be engaged. And when you're dealing with AI, there's an evolution. Your models are always going to be training. So there's got to be a really functional cycle that accompanies development and deployment, an ongoing operation of the solutions we put in place because it's going to continue to learn, at least theoretically, and it's going to solve a whole set of problems. And then at some point in time, we should see enough of a transformation that we can actually start adjusting some deeper things or go further. But we've got to have the rigor and the procedural aspects of making sure that we are governing and we are monitoring and we are constantly having the different perspectives looking at the things that we're working on.
Brian Sutherland:I mean, it's the yes and at this point, because it's everything that you just highlighted, Chris. And with those additional perspectives, the failure of what happens when you have somebody take these specs, go off into a silo, start building, the blind spots that no one can really predict or anticipate. You're never going to see those blind spots on your own. And then this introduces yet another perspective that wasn't even considered. When blind spots start to get surfaced and we start to pressure test against those blind spots, there is a natural order of resistance. And from at least my personal experience, usually that resistance is based on I didn't really think about that when we first started. We've moved down the journey so far already, and I've made some pretty big commitments right now. So now what do I do? Do I go back and now have to renegotiate on those commitments? And what kind of fallout am I going to feel from that? So that's a little bit of my own recognition of and appreciation of what certain individuals who are getting that pressure, what they're feeling, and how do you really navigate through that challenge? What expectations do you set? And what's your get well plan? How do you get to launch and then how do you do your fast follow?
Christopher Hutchins:That's an important point. When oftentimes we are working on developing and deploying a system, which happens every day, all day long in most organizations, the challenge is the things that you're not anticipating that do come up, no matter what it is that you're developing. If it's a system, an application, a platform, there are always these things that you come across that you didn't anticipate. And the governance in the context of seeing all the angles, to me, that's one of the most valuable things you can have because it's much harder to miss potential gaps if you have as many perspectives as you could have looking at it from the different angles. You mentioned blind spots. The hard part of them is we don't even know we have them. And so we need to have people that are seeing all these different perspectives.
Brian Sutherland:Some of those blind spots are, in all truth, sometimes they're benign. Oftentimes they are actually very detrimental, especially if we start getting into some of the more legal aspects of things, fines, penalties. And that's only just scratching the surface. That's not even exploring branding impacts, that's not exploring your lifetime value impacts, anything that basically could have negative effects on your business.
Christopher Hutchins:These things that, as hard as they are to implement and get moving on, they do finally get you to a much better place in terms of what I think is important, which is trust. So in your experience and going through different implementations and figuring some of the things out that you've been talking about already, what does that trust start to look like inside of an organization as you're maturing and going through these processes?
Brian Sutherland:It's not easy. It looks like a number of different things. I think most people who probably will be listening in are going to be no stranger to walking the tightrope. You obviously have some objective that's been set, probably at the most senior of levels in executive leadership. And you're now trying to also balance out against what you know could have some very adverse effects while also balancing against that. One of the things that I look at is are there alternative ways to still get to, say you had a savings goal for the year. Are you, do you have alternative ways to obtain that savings that perhaps was not what you originally thought? Maybe your original thought was let's automate something in the foreground. Let's say maybe we do a generative readout to your customers, but then that has a lot of risks attached to it. Is there a way to be able to build trust in your experience first while also building the muscle for generative experiences maybe in the future? And how do you also gain some efficiencies leveraging those generative experiences to streamline or reduce effort when you get into, and we'll use a contact center connection, when you finally get to that upfront IVR 800 number and you get it to the agent or the associate or the person at the front desk, whoever's taking that call, what are ways that we're offloading the effort on their part so that the costs associated with that start to become less?
Christopher Hutchins:I think an interesting piece of it is there are a lot of different angles that we look at this, the trust piece of it. But there's the trust in people, which we touched on a little bit. But now when you're talking about the recommendations, how do you see really overcoming some of the barriers to get to helping clinicians get comfortable, to trust recommendations or to at least be able to assess whether they're really solid or not? And then similarly, if you talk a little bit about what it's like on the other side of it, it's really the patient and the provider relationships or the patient and the nurse relationships. Those are really what we're trying to make sure that we protect and preserve. But it seems like if we're not doing things purposefully, we could really mess up in the trust arena that could cause those things to go completely opposite from where we want to go.
Brian Sutherland:Yeah, that's precisely right. And part of that trust building is also knowing when it's personal. The more personal something gets, the less likely you're going to get trust out of a person to want to go to a machine. Whereas if it's something like, say I went on Amazon to buy a pair of sneakers, there's some personal aspect to that because maybe I just need the new pair of shoes. But if I don't get that new pair of shoes, I'll still be alive. When we're talking about healthcare, which is extremely personal, and we talk about, we'll even look at it through the lens of provider and patient, and not only provider to patient, but also we'll add in the insurance company, the payer. It's very personal at that point, especially for the patient in the middle. Patients don't go to the hospital just for fun, and I never want to stomp on anybody who actually suffers from conditions, but patients are going to providers because there's something that they need to have addressed. There's something with their health that's at stake, and it's survival. It's not something that you want to put in front of a machine. So trust building in a personal experience, you really need a human in the middle in order to be able to build that trust. A machine is never going to be able to, quote unquote, understand what it is I might be going through. It can mimic language to suggest that, but a human on the other side is going to snap and react very quickly to that.
Christopher Hutchins:And that's a huge point of emphasis that I think we should really make sure that we touch on more, probably even more so in the coming months ahead, because the human relationship piece of it is absolutely essential. When you think about where somebody's state of mind might be when they're going to the doctor, there's already a deficit that we need to address in that there's something that's not going just swimmingly for them. And so it's really important that we make sure that we're not trying to get the technology to somehow act empathetically. That's really not going to be helpful because it cannot read body language. Well, I mean there could be some technology on this theoretically, but I don't even think we care about that. We don't want that. People don't, if when something breaks, I don't know about you, but I don't want the automated line when I call.
Brian Sutherland:Right, right. I want to talk to a person. It's funny because you mentioned body language. There's also a hidden language because most of the applications right now, to your point, are not able to read body language. And a lot of that is because of the limitations of what medium is being used. I'm picking up a phone, well, that's purely audio. I'm typing in a chat window, well, that's just words on a screen. But there's a hidden language when we're in an audio conversation, for example, awkward silences. Maybe you can hear a tonal change in the person's frustration or distress. And these are things that the machine is not able to respond and react to effectively. Plenty of people that are trying to make that a reality, but when you have a very personal situation going on, you don't want to guess that they're having a bad situation. You want to be absolutely pinpoint right that they're having a bad situation.
Christopher Hutchins:Right. I think you're hitting on something that is another place where we need to build some things into our workflows that purposefully give us a moment to pause and really consider where things are at exactly this moment, and are there things that I should be thinking about differently based on what I just heard or what I just saw? AI is basically, for lack of a better phrase, almost a decision tree type of thing that you're dealing with. And it's looking for one of three answers, and you give it a fourth one, what's it going to do with that? We have to have some mechanisms, at least from a procedural standpoint, when we're working in deploying the technology, that there are pauses at strategic moments where you're enabling the human judgment to step in and do what it needs to do.
Brian Sutherland:Exactly. And now with generative capabilities, I really do call it mimicking a person at this stage, where you have your branches, you supply this fourth option, it's an unpredictable item. People in their very nature, in the core of how we engage in dialogue, you can predict to a certain extent, but there's always room for unpredictability. So now we are also in the realm of, maybe you have the guardrail to stop a random inject and contain it. Without that guardrail, though, you could run into a situation where, depending on how creative you set it up, the bot will start to, and I hate this term with all my soul, hallucinate. I more personally say it's just making stuff up. If you were to treat it like a person, what would you say? Well, the person's thinking on the fly, and they're making up whatever, they're thinking in real time, and they're making up an answer based off of what knowledge they have and what intelligence they have exposure to.
Christopher Hutchins:This is a crazy time that we're living in. I remember back in the early 2000s when the internet was still relatively new. I mean, you could hear me firing up the modem, my tiny little Macintosh computer that my phone can outperform now by, I don't know, a million percent.
Brian Sutherland:Yeah, right. It's insane. Back in the, well, so let's see. When modems were first coming out, like 14.4K, I think, and then I had a 9600 or whatever. Oh boy. So that's even slower still.
Christopher Hutchins:Yeah, it's crazy. I don't know if I would have believed we would deal with the technology that we are back then. I thought what we were doing then was pretty amazing. The fact that I could actually write BASIC and create a formula that would just count infinitely. I thought that was great. I created the nice little loop. Not so earth-shattering anymore. So I want to kind of pivot a little bit because we're getting to the end. I don't want to miss the opportunity just to talk a little bit about where you see some opportunities where leadership needs to start to think and do things a little bit differently. Because when you're talking about governance, again, we're not talking about controlling and stopping things. We're talking about enabling them. But there needs to be some structure and some framework and guidelines that really help to keep us focused in the right areas, make sure that we're moving in the right directions. We understand the organizational tolerance for risk. We also understand what the legal ramifications are for the what-if scenarios that maybe we're thinking about, maybe we're not. But what are you thinking about and what are you telling folks as you're talking to organizational leadership, potentially board members, how should they be thinking about it? And what do we need to be considering differently than we do today in terms of how we're overseeing things?
Brian Sutherland:Yeah, first and foremost, with all of this, a lot of what my messaging tends to be is centered on getting your framework in order. Treat what you have as an employee. As maybe untasteful as that might be, because I know that there's some folks, like McKinsey has some thousand employees that are all badged as AI employees, but there's some soundness behind that. If you treat what you have as an employee, so remove AI from your vernacular and just think of it as a junior grade employee, what would you be doing to support that employee? You would have onboarding instructions for making sure that that employee can become effective within their first 30, 60, 90 days. With an AI tool that you're introducing, if you treat it like an employee, you might have a longer runway, but you'll more than likely develop something that will actually benefit you for far longer than your average employee that you hire. And then we have to also look at it as it's never going to mature past the junior employee stage. So what does the support staff look like around supporting that employee? What kind of remedial training do we have to administer on a routine basis? When a new policy is being determined and is being implemented, just as you would with a product and another employee within your organization, you need to consider the training timeline for getting that person up to speed. That was definitely the epicenter of the messaging that I have.
Christopher Hutchins:The really important distinction, and I love the way you frame it, think of it as an employee, because it makes a lot of sense to do that. And why it's so different is we're used to putting technology in place that's really automating rule-based tasks. And you can turn it on, you have to feed it some electricity, you might have to tune things up once in a while when we're talking about equipment, computers, or whatever. This is different in that what we're implementing will continue to evolve, it'll continue to learn. It actually is interesting because if we don't miss the opportunity, we might be able to make some significant improvements in how we even understand medicine, because medicine is an evolving science, always has been. But yet when we try to measure things from a quality standpoint, we're treating it like it's static and it never has been. So this governance as a process in how we think about it really needs to be constructed differently so that it has that understanding and that it has to continue to be adapted. Because when we approve something today, six months from now, the approval is based on something that's no longer true, usually, because this capability is going to continue to advance.
Brian Sutherland:And with everything that you were saying there too, Chris, you actually jogged my memory. The train came back into the station. The second point is if you are treating it like an employee, assume that it will make mistakes, not that it might make a mistake. And as you think about that, design your support struts, your framework for when those mistakes happen. What is your policy and process and procedure for those mistakes? In a world like healthcare, the most common mistake that I see come up is usually not intentional in any way or malicious, but it would be an unintentional breach of PHI. And now I've accidentally disclosed information I should not have. And anyone, all of us in the healthcare industry, as we all know, you have a policy in place in your organization for when a breach happened. Now we go and disclose it, we self-report, and that protects ourselves, and it also strengthens ourselves so that we protect our patients and our members in the future. That we don't have one leak become a million leaks.
Christopher Hutchins:Just a little bit of a difference between those points. I think that's a very well put framing for that. So as we're kind of wrapping up here, we've talked about moving fast and the areas where we need to make sure that we're taking the right steps. We don't want to slow progress, we do want to make sure that we're doing things at a logical and a responsible pace, and we have to make sure we've got people looking out for the blind spots that we have. It's always going to have to be there, unfortunately. But that's life. We're human beings and I don't know about you, but I have friends and family members who have probably saved my neck more times than I could count because they saw something that I didn't see.
Brian Sutherland:I mean, I would be lying if I were to say, like, oh yeah, I've never had a thing come up. We've all been there. And call it our guardian angel or whatever you want to call it. But yeah, it's just part of human nature.
Christopher Hutchins:Well, if I could have you just think about this as we wrap up. If you think about looking ahead in your crystal ball, what do you think the next five years is going to be the thing that's going to make the difference between organizations that really implement effectively and manage well using these AI technologies versus ones who may fall down and not have the success that they're aiming to have?
Brian Sutherland:In my heart of hearts, I think the biggest differentiator is going to come down to the organizations that can effectively structure themselves to treat this AI as an employee, regardless of where you put it in your configuration, regardless of how you think it might be protected, treat it as though it's a person. Don't call it a person, of course. It's still not a human at the end of the day. But if you were to center your thought on that, those organizations that can build frameworks and governance around that, those organizations will have staying power from now until the next big disruptor comes out. And then we get to do the merry-go-round all over again.
Christopher Hutchins:I like how you frame that, and I think it's an important thing to note as well. We are dealing with a period of time where people are pretty uncomfortable and they're nervous about their jobs being replaced and all that. And so when we're talking about treating it as an employee, the intent here is really around making sure that we understand that it's going to continue to evolve and grow. It has nothing to do with what the role or even the function is. It's that it needs supervision. It's artificial. Artificial means not real. In my simplest understanding of the term. Maybe it's not exactly correct, but we do need to treat it that way. But not framing it as just a human being, well, this is evidence you're going to try to replace me with a machine. Not what we're talking about.
Brian Sutherland:To put it bluntly, it knows how to mimic to a certain extent. Kind of think of it like a parrot. Parrots are incredible creatures that can mimic human speech, human sounds, but it itself is not a human being. It's not effective at interpreting all of our language. An animal is going to be a little bit closer to having emotions and feelings, but it's parroting back information that it has been fed. So it is the ultimate imitator of human language and human engagement that we have seen in the entire history of the human race.
Christopher Hutchins:That's it for this episode of the Signal Room. If today's conversation sparks something in you, an idea, a challenge, or perspective worth amplifying, I'd love to hear from you. Message me on LinkedIn or visit SignalRoomPodcast.com to explore being a guest on an upcoming episode. Until next time, stay tuned, stay curious, and stay human.
Podcasts we love
Check out these other fine podcasts recommended by us, not an algorithm.
Practical AI in Healthcare
Steven Labkoff, MD and Leon Rozenblit, JD, PhD
AI and Healthcare
Tensor Black
The Business of AI in Healthcare
Robert Kaiser
The Future of Healthcare AI
Bain Capital
The AI Healthcare Podcast
Dylan Reid
AI Governance with Dr Darryl
Dr Darryl
The AI Rules Podcast
Council on AI Governance