Why AI Governance Is Failing in Most Organizations

Only 1 in 4 organizations has fully operational AI governance — despite nearly all of them having a policy document that says they do. In this episode, Dr. Marilyn Carroll breaks down the three failure modes that turn AI governance into compliance theater: governance by committee with no authority, policies that live in shared drives instead of decision workflows, and frameworks that lag deployment by months. If your organization can't answer "who is accountable?" in sixty seconds — this episode is your starting point. Real governance names people, embeds in processes, and gets tested before something breaks.

[ Hashtags ]

#AIGovernance #ResponsibleAI #LeadershipAccountability #AIPolicy #ExecutiveLeadership #AIRisk #TwoDoctorsAndATwist #FutureOfWork #AIStrategy #OrganizationalLeadership #EthicalAI #DrMarilynCarroll #AICompliance

Stay connected with 2 Doctors & A Twist – Just What the Doctor Ordered!

• Subscribe on Apple Podcasts, Spotify, or wherever you listen.
• Follow us on LinkedIn for clips, insights, and upcoming episodes.

Transcript

SPEAKER_00 0:01

Hi, welcome back. This is episode two of our limited series. As I stated before, Dr. Jamie Chessler is off on another assignment right now. And so she couldn't be with us for these this limited series. So we decided to bring something a little different to you, to you for these next um, there are a total of 12 podcast episodes. They're small, not a lot of time, maybe 10 to 15 minutes. So let's get started. So, why AI governance is falling or failing really in most organizations? Let me ask you two questions. Does your organization have an AI governance policy? Now, my second question is this Can anyone actually tell you what it says? Uh-huh. Let's see what you come up with. Now, pause there for a minute, okay? Because that gap between having a policy and having governance that actually functions is where most organizations are sitting right now. Yeah, that's where most organizations are sitting. And it's more dangerous than most leaders realize. So let me say this to you. So in the last episode, we talked about how AI is exposing leadership systems, right? Now, today we're going to talk about why those systems are failing to govern it. Yeah. Why are they failing? Because here's what's happening across organizations. AI is being deployed at speed, but governance, wow, governance is lagging behind, guys. And leaders are starting to feel it. They're feeling it. Oh my gosh, are they feeling it? Not because they don't have frameworks, not because they haven't uh read the reports, but because governance is being treated as something you write instead of something you design into decisions. And that distinction is everything, right? It's everything. So a lot of people are really experiencing or companies that have AI are experiencing some things they really did not expect to experience when putting in AI into their process. So what I'm going to do next, I'm going to talk to you about uh before we go through the rest of this series. I want to talk to you for a minute and tell you some information. Only one in four organizations have fully operationalized AI governance. Despite nearly all of them having a policy document that says they do. I want to in this episode break down three failure moves that turn AI governance into compliance theater. Yeah, you heard me, theater. Governance by committee with no authority, policies that live in shared drives instead of decision workflows, and frameworks that lack deployment by months. If your organization really can't answer who is accountable in 60 seconds, then this episode is your starting point. Real governance names people, embeds in processes, and gets tested before something breaks. So maybe asking, okay, Marilyn, I get you. I get you. Well, here you go. And lagging behind, as leaders are starting to feel it, not because they don't have frameworks, but really because, again, they haven't read the reports. That's not it. But because governance is being treated as something you write instead of something you design into your decisions. The distinction is everything. Let's ground this into some reality. Research from the World Economic Forum shows 81%, 81% of companies are still in the early or nascent stages of AI governance. Another study from IBM found that 63% of organizations that experienced a data breach had no formal AI governance structure in place. Research from audit board says shows only one in four organizations has operationalized AI governance. Not documented, but operational. Meaning, what does that mean, Marilyn? What are you getting at here? Well, guys, most organizations can point to a policy. They really can. Very few can point to point to a system that actually governs decisions. And I find this is so important. If you start with your processes when you build them in and say, with the rules there, you can build something that is tremendous help to your team. So what's going wrong? There are three failure patterns I see over and over again. I said these earlier: governance as compliance theater. The document exists, it's been reviewed, it's been approved, it gives, uh it lives in the shared drive somewhere, okay? If you're like us, you could get a chat box, you put together and bring it forward. Yeah. But it is not connected to how decisions actually get made. And organizations still have this horrible flow process on how decisions get made. Did you know that I may be a little off on my statistical information here, but most days that employees are working, your team is working, 50% of it is trying to find a decision. Find to find the right rule to put that's in place to govern something, especially when you're moving in a fast-paced environment. That shouldn't be. That's ridiculous. So when AI is used in a real situation, no one references the policy. They just say, oh, I think that's how it goes. They don't say, well, according to this on page 29 of policy number 1856, this is what it says how we should work. And if this doesn't work, or for some reason we have to go to someone else, it says that also. So number two is governance by committee. Everyone is involved, involved. Legal, compliance, technology operations, which sounds good until you realize no one actually owns the decision. It's floating around from Paul, Mary, and Sally. It's ridiculous. Who does this? What are you wasting money like this for? It's not authority, guys. It's not authority. And in high-speed environments like we're in right now, decisions can't scale. They don't. They can't. The number three reason, are you ready for this? Governance that lacks reality. Yeah, I said it. Governance that lacks reality. AI gets deployed. Teams start using it, and decisions start to shift. And governance shows up 18 months later. 18 months later? Oh, come on, Marilyn. Yes, 18 months later, decisions show up. So trying to regulate something that's already embedded, that's not governance, guys. That's cleanup. Okay? We don't want to do cleanup. The reframe here is this. This is where I want to shift your thinking. Because governance is not a document, governance is a decision architecture. Why? Well, in my work and in my book, Human Government AI, uh, and this book it'd be out, I think it is scheduled to release April 15th. I made this very clear. Governance must be built into the system before AI is deployed, not layered or after the fact. And here's here is the simplest way to understand it. A policy tells you what should happen. However, governance determines what can happen. Let me say that again. A policy tells you what should happen, whereas governance determines what can happen. So if you, if your systems allow a decision to be made without accountability, you don't have governance. You don't. What you have, if your system allows AI outputs to be used without review, you don't have governance. Now, I'm not saying that everything has to stop and and we have to review things. That's a hold up of the process. And a lot of my friends in this particular territory and and uh in consulting practices and building frameworks and all, there's Marilyn, we can't stop. This is why we own technology. We're not here to stop. So you embed them into your process, you embed them into the system where the system creates checks and balances so that when it finally gets out, it works, and that you have a tally of any financial, especially decisions that you're going to make or need to be made based on that. You have somebody who checks that before it's released the next day. You don't just leave anything alone and and expect that, oh, all of that worked. I think one of the best systems out there uh is uh stripe. Stripe, you a governance is built into that process. So if you have your processes made along the line, Strike did a brilliant, brilliant job, is embedded in Stripe, the rules that they have there, they know before any money leaves that account that it has already gone through all of the processes it needed to go through because they were embedded in the system. All right. You um don't have governance if you haven't done something to that nature, like a stripe platform that's been set up. If your system allows authority to be unclear, you don't have governance, you have documentation. So even using the Stripe example, uh, I have several accounts through Stripe, but reading through all that documentation and trying to decide was just a very laborious process for me. Yes, Stripe, I'm talking about you on this end, but I think you do have a great system on the other end. And here's what I mean. Here's what I really mean by this. It should automatically say or check, have a check in place. You have one account. Do you want all of these or you should change to an umbrella? These things should be embedded in the system because nowadays a lot of us own more than one company. We do. We need it for survival, some of us. So we're gonna have more than one company. We need cash flow to go through the way we need them to. It could be five companies. I don't need five Slack accounts for that, okay? Uh uh strike accounts for that. So let me give you a real pattern. An organization deploys AI is uh in its hiring. I think I wrote a newsletter, uh, posted a newsletter about this one day on LinkedIn. So an organization deploys AI for hiring. They have a policy, right? It says avoid bias, ensure fairness, maintain oversight. All the right language, right? Yeah, it's all the right language. However, now hiring decision is made. The AI ranks candidates, the recruiter follows the ranking, right? The manager approves the hire. Six months later, there's a problem with the hire. A pattern of bias emerges. And now the question is: who is accountable for making that decision? Well, HR says we followed the system. The manager says we trusted the process. The vendor says, we built the model, you deployed it. Here's where we come in at. We built the model. What does the model have in it? And what are the checks and balances that were in the model? The managers need to ask that, or HR needs to ask that of the vendor before they go through. They should understand what checks and balances they want. The vendor could have provided them a system that was for some other company based on that. The average. You may not be the average, you may be the above average or something else at your company. So we want certain things to be in our system, right? So the manager says we trusted the process. Did you ask questions? Did you see? Did anybody check on the end of it to say if we implement this process, how would it work? And what do we look for? And what do we need to make sure it does for us? That's what I mean by parts of governance. And then real uh leadership in the situation realizes something. The policy existed, but governance didn't. Because, really, because guys, governance would have answered the question before decision was ever made. It's like it striped in the payment processing world. Uh, I'm getting a process. It has already answered the question before it was made. So what good looks like what does real what does real governance really look like? You may be asking. There are four characteristics. Get this down now. One, clear decision rights. Someone is named, not a group, not a committee, a person. Number two, embedded checkpoints. Governance lives inside of the workflow. Okay, you got that? Governance lives inside of the workflow, not outside of it. Number three, define escalation paths. When something goes wrong, everyone knows exactly what happens next. Okay. And then number four, continuous stress testing. Governance is not static, it is tested, guys. It has to be regularly tested. So this is what I call governance as operating architecture. All right. This is governance with an operating architecture layered in it. Not oversight after the fact, not structure before the decision, but it is. And if your system doesn't enforce clear authority, named accountability, and decision boundaries, then AI is not being governed, it's being trusted. And trust without structure is where risk lives all the time. And we see this happening in real life scenarios before AI even came to play. I have been in many situations, many situations, where it was being trusted by a person that was going to carry out the vision and mission of the top leader for that particular area. And we find that it really wasn't. What was carried out, it didn't have a structure. And without that structure of the leader, the visionary, knowing, okay, what happened here and understanding what that's supposed to look like, there was failure over and over again. And for me, even before AI, this is why we see a lot of companies collapse. Now, I uh once owned a security company. And if I had left things by themselves for the team, and this is new to me, and I need to know how things work, I wrote the procedures, I wrote the process, I wrote the HR rules and regulations. And for me, I wrote the exam. Yes. I actually wrote the exams that my entry-level people would be taking in order to get a security license as well as senior management in the security area. So beginning levels, middle, and senior. And all of that was approved by the state in which our companies, the states in which our company operated. So that's not typical. The entrepreneur, the owner of the business, usually don't do that. They usually have a team. But what I'm saying to you is this leadership is no more about sitting back and saying, uh, Sally has to, somebody has to be accountable. Now, the CEO may be out waving, making contacts, making um setting up negotiations and things of that nature, playing the role they need to. But your senior, your most senior leadership in the office, should understand everything that needs to be done. And there should be a 101 each week with the leader that gives them snippets of what happened, what major thing happened that we needed to talk about. Is the systems operating the way they need to? And what are adjustments we need to make? That shouldn't take all day. Heck, this pet podcast is only, I think, about 10 minutes, uh, each of them. So we should be able to look at situations that we understand and know what will or will not happen and how it rolls up to the major initiatives of the organization. Anything with money to it, you should know about it. Anything with people to it, you should know about it. Anything with a process or an operation footprint and a community within it, you should know about it. So before the next episode, I want you to test something for me. Are you ready? I want you to take your three most important AI-driven decisions and ask, who is accountable by name? Where does governance show up in the workflow? And when was it last tested? I need you to do those things for me, as well as what happens if it fails, and who decides when the AI is wrong? Those are five important questions. I know I said three, but I gave you five. Okay? There are five most important AI decisions. Who is accountable by name? Where does governance show up in the workflow? When was it last tested? What happens if it fails? And what, or who rather decides when AI is wrong? And if you hesitate on any of those guys, that's your gap. Yeah. Bottom line, that is your gap if you hesitated. So on our next episode, we're going to dive deeper into something even more subtle. Authority. Yes, we're going to talk about authority because right now in your organizations, authority is already shifting. And most leaders don't even realize it. Yeah, it's it's deep. It's just that deep, guys. So I want you to take these questions with you, and I will see you on our next episode of two. Doctors and a twist as we go deeper into something even more subtle that we see operating inside organizations, small and large and small. Take care.