Health Care Corporate Governance: Critical New AI-Related Issues for Health Care Boards

Rob Gerberry, Senior Vice President and Chief Legal Officer, Summa Health, speaks with Michael Peregrine about how health care corporate governance may be impacted by the rapid acceleration in artificial intelligence (AI) sophistication and utilization, and the implications for its future development. They discuss how the board should approach its fiduciary responsibilities and interact with senior executive leadership in the face of this rapid AI acceleration.

Watch this episode: https://www.youtube.com/watch?v=4FykGUMuYcQ

Essential Legal Updates, Now in Audio

AHLA's popular Health Law Daily email newsletter is now a daily podcast, exclusively for AHLA Comprehensive members. Get all your health law news from the major media outlets on this podcast! To subscribe and add this private podcast feed to your podcast app, go to americanhealthlaw.org/dailypodcast.

Stay At the Forefront of Health Legal Education

Learn more about AHLA and the educational resources available to the health law community at https://www.americanhealthlaw.org/. 

Transcript

SPEAKER_00 0:04

This episode of AHLA Speaking of Health Law is brought to you by AHLA members and donors like you. For more information, visit American Health Law.org.

SPEAKER_01 0:17

Hello, everyone. This is Rob Gerberry. I'm the Chief Legal Officer of Summa Health and the President Lick Designate of the American Health Law Association. I'd like to welcome you to the latest in our continuing series of podcasts on corporate governance issues affecting healthcare organizations. For today's episode, we're returning to a topic we've addressed before: the role of healthcare governance in supervising AI deployment within an organization. We're doing that not to repeat what we've discussed before, but rather to focus on possible new areas of interest being highlighted by current development, of which many of you are already aware. More specifically, this episode will discuss how corporate governance may be impacted by the rapid acceleration in AI sophistication and utilization and the implications for its future development. This acceleration is changing the script for how board should approach its reducier responsibilities with respect to an organization's use of AI and how it should interact with the senior executive leadership team in that regard. Through all this, we're not trying to undermine the extraordinary benefits that the healthcare sector may achieve from AI, but rather to point out the related governance challenges that are ongoing. Help us discuss this issue. We're joined by our HLA colleague, Michael Perrigan, who's both an HLA fellow and a fellow of the American College of Governance Council. Hello, Michael.

SPEAKER_02 1:38

Hello, Rob.

SPEAKER_01 1:40

So, Michael, a lot of our members have been able to brief their boards on AI issues over the last few years. And I'm sure, like myself, they've used capital to get on the agenda to host those briefings. What are we telling the chair and the CEO as far as why they need to come back to the board again on the topic of AI?

SPEAKER_02 1:57

Rob, I you know, I think in one sense it's pretty simple that this is a technology that's expanding in its potential potential utility and benefits, as you said, at almost exponential rates. And the board simply's got to be able to keep up with it if it's going to be able to do its job. The alternative is just to have the board simply respond with a, you know, whatever you say to every AI issue and proposal that management offers.

SPEAKER_01 2:25

So, Michael, if that's your simple answer, let me dive a little deeper. What's your complex one?

SPEAKER_02 2:29

Well, I'll use multi-syllable words now. Um, I think it's that the fiduciary expectations of boards of directors in business sectors across commerce that are aggressively applying AI, like healthcare, are increasing at rates that will really challenge their ability to satisfy those duties. My view is that you know effective AI oversight and decision making could require a whole new level of training and management support if the board is really going to be able to maintain its finger on the AI pulse of the organization. My biggest concern, Rob, and something, you know, we're we're not to be what they call a pannequin, uh, but my biggest concern is the directors are going to recognize that they might not have the time to keep up with all the changes in AI and the developments and the evolution. And they might just not have the ability in terms of engagement to do their job. And they'll they'll say this, you know, this is a risk to me, this is frustrating. I don't know enough to weigh in here as things move forward. And they're going to either resign from the board or just become totally engaged on AI issues and becoming far more uh dependent on the representations of management and outside advisors. That's a problem, because that's the point where you're essentially not to say that management would leave anybody in the wrong direction, but you're essentially defaulting from your role uh as a board if you say, I just don't have enough information to make an intelligent decision. And again, I can see board members coming to that conclusion in the near future.

SPEAKER_01 4:13

So boards never like to hear on their agendas, Michael, the sky is falling, and they ask us for specific concerns. So we're gonna ask you what's your specific concerns?

SPEAKER_02 4:21

Well, yeah, absolutely. You got to list them, and what are our concerns? And I think again, we want to start with the basics. And and you can pull, you know, I think Rahwi, I want to realize that for everything I'm gonna be saying in one respect, there's probably an opposite perspective. So I'm reacting to what I see and I read. I'm certainly, as you know, I'm happy with the push-button uh phone on my desk, Ada. So I'm not Mr. Technology, but my sense is that the rate of acceleration of AI is moving forward at a rate that's just far surpassing that of other technological developments in years past. One of the things that I was have been reading uh in preparation for today's presentation suggested that AI skills are actually doubling uh every six to eight months and then may actually increase in pace over time. Uh you look at the January essay that written by the uh uh anthropic CEO uh Dario Amadai, um it's entitled The Adolescence of Technology. And he makes comments about what he regards as the scaling laws of AI systems, how they begin to move and move and move and really accelerate. Yeah, and then there are the increasing concerns, which are unsettling but legitimate, that AI systems can go rogue and whether individual organizations and their leadership have the ability to identify and address those concerns on their own. Again, not to make a political statement, but we're we're seeing a little bit of this in terms of the uh uh use of AI in uh war-making capabilities. And then from a very practical perspective, you see many of the disconcerting warnings about the potential AI fallout that are coming from those who are directly involved in developing the technology. As Peggy Noonan of the Journal described, the inventors and executives are warning of widespread consequences that they don't begin to understand. Uh again, for every apocalyptic comment uh is is there's an alternative argument. But the board's real responsibility is to balance both of them, to take them into account. Uh, and I think the real area of concern is is the board going to be able to have the time to have the broad-based discussion on what changes are uh are coming to the board as uh AI moves forward at such a rapid pace. And it's not just a, you know, let's have a retreat on AI, it's some serious decision making on whether we've got the capacity to exercise oversight.

SPEAKER_01 6:58

So, Michael, let's take acceleration as a given. What are the areas in which you feel like the greatest level of board education really needs to occur?

SPEAKER_02 7:05

Well, I think first I would start with the fact that that, and maybe maybe the the chief technology officer would have disagreed with this, but it seems to me that we're in a space of the of in the development of the technology that's far surpassing that which uh uh was in place in healthcare institutions when the current board got its first education on AI. Things have changed. I would focus on four or five areas uh that I think our members, our listeners, uh can zero in on and will get. You know, first, I think the areas of uh education should be on the board's overall plan for AI engagement going forward. Second, uh is our old friend compliance oversight, both as externally as to law and regulation, and internally as to what is usually referred to as AI governance at the management level. Third is the whole thing about workforce culture and workforce displacement. Fourth is the extraordinary important question of trust as it relates to healthcare organizations, and that kind of picks up on the rogue AI issue as well. And that's the you got to have a discussion on that. And then fifth is ultimately whether the board should have a system in place should it need to exercise the corporate social voice to speak to any of these concerns. But I think those are the kind of areas where I would focus my conversation with board and executive leadership.

SPEAKER_01 8:32

So let's start with your first one. Let's start with AI engagement. What do you think a board plan should look like?

SPEAKER_02 8:38

Well, I think this is really in my mind. Um and I think it's really the the question of whether the current board structure, and by that I mean the size of the board, the composition, the expertise involved, the committee structure, and the support that Rob, you and your executive colleagues provide to the board, whether that works now, whether that's going to be enough to put the board in a position to exercise its tech-related, AI-related responsibilities going forward, given that the rate uh of the tech is accelerating in sophistication. In other words, does the way we have the board set up right now continue to make the directors capable of keeping up with the change? You know, uh, I think on one of our podcasts in prior year or two, we talked about the NECD's annual Blue Ribbon Commission report that focused on tech leadership in the boardroom. And that provided a detailed but very practical template from which boards could organize themselves into relation to the tech issues that they expected to confront them in the future. You know, in many respects, it envisioned what I call a whole of board approach the way the board engages with technology. I think the question now is given the accelerated development of AI, is that template still work? Does it need to be refined? Um, are we dealing with issues that may or may not have been foreseen a couple of years ago? So, what what I would encourage boards to do is to ask themselves whether maintaining this whole of board approach where every director is engaged with AI is doable going forward? Or whether handing off the responsibility in a very significant way to one or more board committees with board delegated powers, and I'm thinking technology committee, human capital committee, does that make more sense? Um, you know, we've we've all seen studies that talk about the annual commitment of directors to their board service. And, you know, I the what I the numbers I see don't really vary. It's 300 to 350 somewhere in their hours per year. The question is, would uh does that leave the board uh enough time to expand the attention that they're given uh to accelerating AI development? Does that commitment of 300 to 350 hours can that be expanded to increase board service as it relates to AI, especially when you compare it to all the other uh of the board's duties, of a healthcare board's duties, and especially if board members have demanding jobs in the daytime, or whether they serve on other boards. So I think it's important to ask the question: the way we're set up, the way our terms are set up, the way we are um allowing executives to serve on uh other boards, are we creating an environment in which the board is given enough time to dig into the stuff and to keep up with it or not? I'm not sure. But I'm not saying yes or no. I'm just simply saying it's probably time to reimagine board's oversight of AI in a more efficient manner. That's and so that's the engagement question.

SPEAKER_01 11:53

And so when we think about the board's duties around compliance, how do you view that related to AI?

SPEAKER_02 11:58

You know, compliance has never lost its importance to me on the board, and I think we're seeing this in a particularly interesting way, which I which could bring in the chief technology officer to help this the uh chief compliance officer and the general counsel. I think the concern with compliance is twofold. Uh uh first, it is the concern is grounded in a fact. I think it's fair to say that there just simply is no overarching federal regulatory structure or or set of guidelines for AA deployment. The Biden administration was going down that road, but obviously then left office. And I just don't see, again, I'm not making a political uh commentary here, but I don't see a plan from this administration to create a specific uh uh uh federal scheme for uh uh regulating AI. Uh I think it's you know, again, it's not politics, it's just the way it is. That's the the administration isn't oriented that way. Now, that's not to suggest, though, Rob, that there's no AI regulation at the federal or state levels. Uh that's absolutely not the case. You know, if you there indeed there are some federal laws, and I think you can think out right off the top of the head, uh, the financial services industry uh regulatory uh scheme, the EEOC, the FDA. Um these are three areas of federal regulation where you are seeing them address AI issues, and certainly there's a growing patchwork of state laws that regulate the use of AI. So I think we start with the fact that yes, there there is no overarching federal regulatory scheme relating to AI. Nevertheless, there's enough out there, including at the state level, uh, that there's a high likelihood that a company's use of AI is regulated in some manner. Um so we have that as a given, and I think it's important that the board and the compliance committee and the chief technology officer understand that. The second compliance concern is I think it's the board has an increasing responsibility to make sure that management has empowered the CLO and the CCO to provide advice on all the laws at both the state and federal levels that may intersect with AI deployment. Is this a seminar? Is this a briefing? Is this a paper to the compliance committee? I don't know what it is. Um, but it's it's some way that the the portfolio of the chief legal officer and the chief compliance officer definitely and formally includes uh an educational briefing or series of briefings to bring the compliance committee up to date on what's going on out there, separate and distinct from traditional compliance goals, you know, which could be complicated too if the Department of Justice comes out with new enforcement guidelines this spring, as they've been hinting. You've got a compliance committee with a whole lot on their plates, uh. And then I think part of this too is for the board to make sure that all of this is coordinated with management's AI governance structure so that the board's compliance responsibilities are integrated with the compliance activities of management through that AI governance apparatus. Absolutely right hand, left-hand um uh uh coordination.

SPEAKER_01 15:11

So, Michael, that sure is a lot for boards to think about. Where would you say the trust component comes in?

SPEAKER_02 15:16

Well, I don't want, you know, I think that's historically been the issue in healthcare organizations is uh, you know, what are the risks? Uh can we rely on what we're deploying to do the job? What's the accountability? What's the measuring? And I don't want that to get lost in all this broader question of whether the board um is able to exercise oversight. You know, time was uh when the matter of assuring trust in the outcomes of AI technology was, you know, that was one of the primary tech responsibilities of the board. Um but I think that job has become a lot more demanding as AI capabilities and utilization expand rapidly. You know, governance lawyers like you and I have long assumed over the last couple of years that the board liability for AI oversight is going to reflect some element or aspect of the CAREMARC doctrine, which, as we've talked up on this podcast, kind of is focused on a two-pronged test. One is that you've got an oversight system in place, uh, and second, that the it'll that the board is given that oversight reporting system is able to spot and respond to uh to red flags of risk. And part of that of a care mark response that uh should be addressed by what we just talked about, upgrading and integrating existing compliance mechanisms at the board management level that satisfies that first test. And I think that's an important consideration for the compliance officer, Rob, to make to the board. Does our existing compliance program work to pick up the trust issues, the risk issues? Do we have something else in place that does that? How do they integrate? How does it get to the board and in what kind of a timely manner? I think from an accountability for management perspective, is we need management to increase their briefing to the board on trust and reliability features and concerns that are just inherent in the AI tools it currently uses. You know, dashboards, all that. But what I'm talking about is an upgrade on AI-specific trust and reliability features, separate and distinct from traditional compliance issues. And then I think there's a clear understanding that management must absolutely timely report signs of risk exposure upstream to the board or whatever committee is monitoring AI so that appropriate responses can be pursued. Rob, you know, we've talked on on in the podcast before that one of the elements of um AI liability that's we we've learned from the Delaware courts is well, what do you do once you get a red flag, a warning board? What do you what do you do with that information? And and I think that it's really important that uh as we are dealing with new signs of risk exposure from AI, that management is drilled in. I'm not sitting on this. I have got to get this to the tech committee and I've got to do it right now. I've got to let them know, and I've got to tell them what it means. Uh, because we're talking about a level of risk, Rob, that is less familiar to many lay board members. And we we we need to make sure, and and I was about to use the word double down, but then I was ready to hit myself. Uh, one of those words I want to give up forever. But we we really need to make sure that we invest the time in upgrading the compliance reporting mechanism so it works to put in the hands of the board uh an awareness of what might constitute a yellow flag or a red flag. Um as I said, identifying red flags of traditional compliance risks from a healthcare organization, you know, regulatory risks, that's gonna, I think most boards that comes relatively easy to them. We need to ask, though, whether identifying red flags from AI-related risks come as easily as well.

SPEAKER_01 19:03

So, Michael, I've been offering you softballs this entire podcast. Why don't I throw the issue now at you about workforce displayment? How does that sound for boards?

SPEAKER_02 19:10

Well, you do we have another hour or so? Or is this, you know, I I think you and I have talked over the last year, and we've, you know, put this in AHLA publications and other platforms, that the board really should perceive itself as having a fundamental obligation to monitor the impact of AI-related decisions by management that carry the risk of big-time displacement of employees. It's an obligation, I think, that's grounded in the board's general responsibilities over human capital. And there really is a lot of solid thought leadership out there that suggests that those responsibilities are enhanced or heightened when you're going through a significant deployment of technology. Uh, Martin Lipton, uh, the founding uh partner at Wachtel has said, and I think this says it the best, boards should consider in a balanced manner the effect of technological adoptions on important constituencies, including employees and communities, as opposed to myopically seeking immediate expense line efficiencies at any cost. It's just my view, but I do have my doubts as to whether Mr. Lipton's really sage advice is being widely followed by boards. So, you know, on the workforce displacement issue, I think task number one is for the board to have a healthy discussion as to what it believes its responsibilities are uh when it's it's informed of or asked to approve significant uh investment in technology that the management says will likely result in also substantial displacement of employees. What's our job here? What are we gonna do? How do we exercise our responsibilities? That's a that's gonna be a very interesting board conversation, and it has to, I think it has to happen.

SPEAKER_01 21:09

So, Michael, that's the complete universe of concern that you have, or is there anything broader when we look at the potential massive layoffs that will come to entire industry sectors?

SPEAKER_02 21:18

Yeah, you know, I we we've certainly seen it, haven't we, Robin, over the last month in terms of it seems like every uh tech billionaire is coming out with his or her own statements of uh of an apocalyptic uh scenario. I did, I don't know about you, I can remember. I think I studied the Industrial Revolution, maybe at TCU, but also in high school. I'm no expert, but I I I'm not sure I buy the the uh references to uh a new industrial revolution so much. But I I think you know it's very difficult for the board to be prepared to address workforce culture issues and not at least have read some of these. Statements. The Andy, very detailed essay. The Suleiman of I think Microsoft, you know, the gentleman who was suggesting 25% of the workforce, or at least the white-collar workforce, being wiped out. And then there's the the, I think it's the Citrini research memo that a couple of weeks ago really shook the financial markets to its core. They're all saying the same thing. They are frightening to everyone. They are persuasive, and their perspectives are growing in acceptance. But we don't know their projections. We don't know how reliable they are. Now we're aware of the importance of healthcare sector to employment levels. We've seen that in the Bureau of Labor Statistics reports the last two months. But this is a bit of a broader commercial and social concern. So I guess what I'm saying is at the healthcare board level, uh, these projections uh of an AI jobs apocalypse, uh, you just can't blow them off. You can't just dismiss it as saying this is you know uh skies falling stuff, we should ignore it. Uh it's part of the board's oversight responsibility, Rob. It's a huge big picture issue. The board can and should be able to have a discussion with management on this without accepting the underlying premise. You're the question simply is we've all read about this stuff, they're coming again from the people who are developing the technology. Should we pay attention to it? The answer is yeah, you should. You pay attention to it and kick the tires and ask experts and say, is this a realistic concern or is this just pie in the sky stuff? Um and you know, you treat them as serious projections made by serious people, fairly presenting serious potential interests, business interests, and societal risks. But then, consistent with the way we've always looked at oversight concerns, we kind of dig into the alarm bells or have the tech committee do so and ask ourselves the question that are do these risks, are they really valid? Do they demand our response, or do we wait and see how this all plays out? I think you know the question is do what are the potential labor and social costs? Not just on the company and its shareholders or constituents, but again, on the workforce, the communities that we serve, our vendors, and you know, I hate to say it, but you know, on society, because you know the projections are uh and all the horrible things that they're projecting about the impact on society. If you if you use the uh Rumsfeld uh risk matrix, are they known knowns, known unknowns, or or something else entirely? If those risks can't be confirmed in the board's normal oversight process and discussion process, exhale and go back to the normal agenda. But if they can be confirmed, but they look like a yellow flag or or maybe a red flag, uh then the board's got to move forward in some kind of response.

SPEAKER_01 25:06

So, Michael, when you say they need to move forward with some kind of response, do you have thoughts on what that response may look like?

SPEAKER_02 25:11

Yeah, I and I think again, we we we wrap this into the CAREMARC um analysis. Is this a yellow flag or a red flag? And um I I think there's a lot the board can do, but the clock is is is ticking. Um I think internally the board can work with management to make sure we've got the broadest possible AI training and risk monitoring dashboards possible. Are we doing as much as we can to train our employees to be able to adapt to the new technology? Uh do we have an enhanced level of executive accountability for the results of our initiatives? We're pouring a fortune into AI. What are we expecting in terms of uh return on investment? What should we be asking for in terms of accountability of our executives on how this is working? We need to establish a record that we're holding management accountable to this new tech. Um I think we have to that the board can establish kind of an authority matrix of sorts as to um uh when management should be coming to the board with respect to deployment issues that that are gonna disrupt the workforce. And what's the board's role there? Is it authority to approve? Is it we're we're alerted to it? You know, what what is the level of sign-off that, if any, that the board has when we're gonna be um uh impacting the workforce? Uh and uh again, supporting expanded AI training for employees, uh, but I think also in that regard, to keep a really firm finger on the pulse of workforce culture as the company moves forward with AI. What is the workforce feeling? What is their sense? Uh are they buying in? Are they concerned? Where are they? That's the board has a role there. Um you know, uh externally, I think the board can encourage management to increase the level of its AI engagement with peer groups in a legally appropriate way, uh consistent with the antitrust laws, talk to your peers, what are they doing, what's going on, what are their concerns again? The CLO has to be involved in those and structuring those. I think certainly that they can recommend that the company's trade association take more formal positions, share corporate concerns with federal and state legislatures, and make, and then I think the the company can also, the board make uh preparations to more formally exercise the company's social voice if it if if all these other steps aren't moving forward.

SPEAKER_01 27:36

So, Michael, I know the importance to you of organizations exercising that corporate social voice. What does that specifically mean here in the context of AI?

SPEAKER_02 27:44

Yeah, I guess I would say not necessarily exercising the social voice, but understanding it's a tool to be used and having a program in place to um to exercise it should the need occur, um, and should it be in the best interest of the business to do so. Uh my recommendation these days, Rob, given the general volatility of the environment, not just in technology but generally, is that boards have a process in place to evaluate whether, and if so, when it makes sense, it's in the best interest of the company and its constituents to speak out as certain tech issues critical to their business and to the mission, pop up? Uh I can absolutely see that happening with AI. If a company believes that the issues we've been discussing in this podcast are serious, if the company uh believes that some of these projections on job loss, on social change, on um evolution of the economy are serious, I think the question is: does this implicate our business? Does this implicate society as a whole to the extent that will implicate our business? Um, and if we speak out, will that make a difference? Or are we just going to get grief from people? Because there's a lot of data out there that suggests that consumers don't like their companies to exercise a social voice. And we know there's a lot of people in Washington who really get irritated if companies oppose the administration. But but I think you know the the conversation has to happen because as we know, the cavalry isn't gonna be coming from Washington, D.C. anytime soon to deal with these issues. Um it you know, oftentimes the public will look to commerce, to business to address issues that um the government does not want to address or has chosen not to address. And and to be honest with you, Rob, I think you're gonna see a lot of companies doing this in the near future if they find that they're unable on their own to address these giant big picture AI issues. Uh they're gonna say, look, we can't do it on our own on health system ABC. We see these issues, we think they're realistic, but we're just uh, you know, a voice in the wilderness here. We they need to be raised to a higher level. How do we do that? And do we help support a broader-based uh concern by the business community? Will that get traction to the conversation? Will that start a conversation? And interestingly enough, I think a door to all this has been opened a bit. Um, and I don't know if you had a chance to read it, but uh the the former commerce secretary, uh Gina Raimondo, contributed a guest essay to the New York Times recently, and she proposed a grand private public partnership to focus principally on reskilling labor in response to the workforce displacement issue. Um, and she also provided a suggestion on how to incentivize business to do so, an economic incentive. And I thought, you know, it was it was an interesting proposal uh to deal with what she described as an extraordinary risk to society. But it's the kind of proposal that says, well, you know, I may not agree with this, but she's raising some issues, and let's still further talk about it. So, you know, ultimately, I think a couple of things. First of all, by responding to your question, I know you and I both are saying, we're not dumping on AI. We we absolutely believe um that the that AI has an extraordinary, profound, and compelling future in healthcare, without a doubt. Uh what we're saying is uh that from a corporate board perspective, you can't buy into that and blow off at the same time uh the warning signs that have been popping up over the last couple of months. You've got to take them both into consideration. Um, and uh I think part of the ability to do that lies in fostering broader internal and external conversations on the pros and the cons and figuring out how to way to work it all out.

SPEAKER_01 31:56

Well, Michael, thank you very much for sharing those thoughts and driving the conversation today around this important emerging topic. AI issues continue to be important to healthcare organizations throughout the country, and chief legal officers can play an important role in helping to work with their boards and their leadership teams to address this. To our listeners, we look forward to being back with you again next month, or our next episode will cover how to separate from problematic directors. So, Michael, until then, thanks again for our conversation today.

SPEAKER_02 32:23

Thank you, Rand.

SPEAKER_00 32:29

If you enjoyed this episode, be sure to subscribe to AHMLA Speaking of Health Law wherever you get your podcasts. For more information about AHLA and the educational resources available to the health law community, visit AmericanHealth Law.org and stay updated on breaking healthcare industry news from the major media outlets with AHLA's Health Law Daily Podcast, exclusively for AH MLA comprehensive members. To subscribe and add this private podcast feed to your podcast app, go to americanhealthlaw.org slash daily podcast.