Decision Accountability in the Age of AI

When an AI-assisted decision goes wrong — who owns it? Not in theory. Not across a committee. Who, by name, is accountable? This is the question most organizations have not answered — and regulators, boards, and affected individuals are starting to ask it loudly. In this episode, Dr. Marilyn Carroll separates responsibility, accountability, and liability — three terms that get conflated in ways that create dangerous gaps — and lays out what it looks like to design accountability into AI workflows before deployment rather than assign it after failure. Boards are now personally liable for AI failures. The organizations that built accountability structures early are ahead. The ones that didn't are catching up under pressure.

[ Hashtags ]

#AIAccountability #DecisionMaking #AIGovernance #ResponsibleAI #ExecutiveLeadership #TwoDoctorsAndATwist #AIEthics #LeadershipDevelopment #FutureOfWork #AIStrategy #DrMarilynCarroll #CorporateLeadership #AICompliance

Stay connected with 2 Doctors & A Twist – Just What the Doctor Ordered!

• Subscribe on Apple Podcasts, Spotify, or wherever you listen.
• Follow us on LinkedIn for clips, insights, and upcoming episodes.

Transcript

SPEAKER_00 0:01

Hi guys, welcome back to Two Dodges and a Twist. And as you know already, Dr. Jamie is out on an assignment. Uh, some assignments she's working on, so she can be with us for this limited series that we're working on. Okay. But I hope you're enjoying it. We're on episode today, is episode four of 12 part series. And these series are not that long, but I, as I stated before, you go through all 12 of these, you will be more knowledgeable about how AI governance may be not, not may not be working in your organization, but can work in your organization and what you need to do about it. So this is episode four, and today we're going to talk about decision accountability in the age of AI. Um, when an AI assistant decision goes wrong, who's accountable? And we talked about this now, we talked about it in the uh previous two episodes. Uh not who was involved, not who touched it, not who reviewed it. Who by name is accountable? Is it Marilyn? Is it Jamie? Is it Cindy? Is it Bill? Is it John? Who is it? Because right now, in most organizations, that question does not have a clear answer. And that's not a technology problem. That's a leadership failure. Yeah. So the accountability vacuum. Uh, let me walk you into something that's happening more often than leaders are really really willing to admit, guys, an AI system is used in hiring. A candidate is selected. I know we've been working with this candidate thing for a while now through the series. Everything looks fine. Months later, a pattern emerges, a bias. And we talked about this earlier in another episode. An investigation starts. HR says, we followed the system. The hiring manager says, I trusted your recommendations. The vendor says, We built the model, you deployed it. And leadership. Leadership goes silent. Because no one was ever named accountable. And that is what I call the accountability vacuum. When I did the CFO project or intelligence certification, I did it to help some friends who were CFOs of organizations. And they wanted to understand AI better. So in helping and building that system, that certification, which I didn't intend to do, but I said that's the only way they're going to know. Because I want them to know they're accountable for what it is that they're doing. I want them to understand, not only be literate at it, but I want them to understand the process, and I want them to understand how the process got there, what it took to build the process. So when you can blend those three things together and look at the outcomes and test those outcomes in a testing environment against what your expectations are, and you're doing it alongside it, you can see how to build your system. If you're sitting there asking what ifs, and you're looking at your AI model as if it's one of your team members, and the way you would question a team member and the rights you would give to the team member, you're doing that. You're putting those information, that information, and the testing points within the system. Remember, the best system I could think of that has everything already is Stripe. And even that system has flaws. I could talk to them on a thing or two that I noticed as a customer that I see that could be fixed and improved. So this accountability vacuum that we have. And it doesn't happen because people don't care. It happens because the system was never designed to hold accountability. Yeah. So defining the problem, let's let's look at that. There's a reason this keeps happening. Leaders are confusing three things: responsibility, accountability, and liability. How so, Marilyn? Well, let's go here. I want to give you a little background. Because when an AI assistant decision goes wrong, who owns it? Not in theory, not across the committee, who by name is accountable? This is the question most organizations have not answered. And regulators, boards, and affected individuals are starting to ask it loudly. And then as we continue to talk about it here, I want to separate responsibility, accountability, and liability. Three terms that get conflated in ways that create dangerous, dangerous gaps. And it lays out what looks like design accountability into AI workflows before deployment rather than assigning it after failure. Boards are now personally, get this, boards are now personally liable for AI failures. The organizations that built accountability structures early are ahead. The ones that didn't are catching up under pressure. This is a known fact. So I want to make sure that you understand that because I think when we do the next section, you're going to understand why we probably got there with something we call the human in the loop. I don't think a lot of people understand what we mean by human and the loop. So let's break this down and defining the problem. Responsibility. Who is involved in the process first? Accountability. Who owns the outcome? Liability. Who is legally exposed? And those are not the same. There are three different things: responsibility, accountability, liability. Responsibility can be shared. Accountability cannot be shared, guys. Accountability must be single. It must be a singular uh thing. One person named, visible. Because without that, without it, when something goes wrong, organization produce meetings. Yeah, we don't know about them. Uh sometimes some of us are in meetings probably 50% to 75% of our day. Is that reasonable? No wonder we're failing. We're in meetings and we have a lot of high-level people in these meetings all day. They're spending time that don't need to spend time there. They're not answers in those meetings. They're just finger pointing things happening, or people listening that don't aren't aware. Or it may be people in the meeting, which some most times is the case, that don't need to be in those meetings because they have no skin in the game, no end result, no name accountability, no influence about the process. They're there. Or no way that they're going to change it. This isn't hypothetical. Global regulators are already addressing this. So, how so? The global policy assembly states, and I want you to listen carefully. This is the global policy assembly. Ultimate accountability remains with the organization, regardless of automation. The Federal Trade Commission, along with the Equal Employment Opportunity Commission, has made it clear. Yeah. And analysis of EU AI acts highlight this: a growing gap between system builders and system deployers. There is a gap. And this is why we do the CFO certification, AI educator certification. We do this because we know that there is a gap between the builders and the developers. Meaning, responsibility can be become fragmented unless leaders design it intentionally. How are you going to design something and you don't know what you're designing? You don't. So this is why you have to go through a training that teaches you what's really going on in the background. This is just like if I'm a manager, and before we had to uh at a certain level, we had to go to school and get an MBA. It was recorded, it was recommended we get an MBA before we got promoted to that, from that uh manager to a higher level manager grade, especially senior management. It's the same thing here, it's the same thing. Meaning, responsibility can become fragmented unless the leader is designing it intentionally. You have to know what you're talking about. So before, the more we went to the top, we didn't understand things. Now, the more you go to the top, you must understand when you're using AI, why you're using it, how to use it, what it needs, what it means, and who's accountable. And normally, if you're in the senior leadership roles, you're accountable. You cannot delegate accountability to a system. You can't do it. And regulators are making sure of it. Here's the three failure moves I see. So, how does accountability break down? There are three patterns, guys. Diffusion. Everyone is responsible, so no one is accountable. You heard that one before? It happens even before AI. This was going on. The flexion. We got diffusion. Now we're going to deflection. The AI recommended it, Marilyn. It recommended it, so we went with it. Blame moves to the system. How can the system decide something if I didn't put it to design it so that it decides something? Displacement. That's the third one. Responsibility strips upstream to the vendors. Well, the vendor designed it, Carol Beck designed it, and PowerDead Pro designed it. Really? To the vendors? This is why I was so excited when some friends and a colleague came to me saying, I need to understand AI. I really do. And one came to me with tears in their eyes saying this. And I understood what they meant. But after it was all said and done, I think we spent about six to eight weeks together. They had assignments. They probably didn't want to do the assignments, but it was important to have the assignments so I knew that they understood and they understood not only the literacy part of it, but how to recommend, how to process your work, what their system was about, how it interacts with everything else that's going on in the organization, what those things meant. And if you had to design this, how would you design it? So once these people understood it, we could design a system for them or work with their team on designing it. But if you're already buying this from a company that designed these systems already or designed financial systems, then you need to know what you're asking them. You need to know how or how what questions to ask and what things you want to have look for so that you know what you're buying is the thing that you can uh stand behind and say, I was in that decision process. Okay. Then the vendors to the data, to the model, and it goes on and on. No, all three of these cases, accountability disappears. Not because it's not needed, but because it was never designed. So let's look at this. Let's let's see how to reposition this in the right way. This is where leadership must shift, guys, because accountability is not something you assign after failure. It is something you design before deployment. It really is. If human in a human governed AI, I make this explicit. AI may assist decisions, but responsibility for those decisions must remain human. This means before AI is ever turned on, you must answer. Who is accountable? What decisions uh require human validation, and what happens when the system is wrong. If those answers don't exist, you don't have governance. You have risk. And here's your leadership decision moment. Yeah. So let's make this real. A financial institution uses AI to flag fraud. I've been there, worked at financial institutions. Uh, a transaction is blocked, but the customer suffers a loss. The system worked as designed, but the outcome is damaging. Why it's damaging because the customer suffers. We all know in financial situations the customer can never suffer. I remember at the bank, um, my team uh had to was assigned to uh the payment processing area. There were some challenges with payment processing for retirement payments. If I have one client, an older client, or some of them had plenty of money, were probably the ex CEOs, CFOs, COOs, CTO, some high-ranking official at an organization previous that's retired, and they may have five homes they live in, different areas, and they request their money, and the system flagged it because it didn't know that this person had embedded in it five homes. They only knew about one. So when the customer called in and called from the wrong uh area code that the system wasn't familiar with to request a transaction, or uh use a computer that the system wasn't familiar with and tracking the large amounts of money that they were asking for. Oh boy, oh boy. I tell you that we hear about it. We were called in, we were questioned, we were ran through the ringer because it can't happen. It damages. You got to ask yourself what happens in each of those scenarios. I'll give you another one. My parents are retired, and they needed to uh get one of the pension checks from one of the mines that my dad worked in, and uh stepdad worked in, and the whole scenario was just so messed up. My mom kept going back and forth. She called me on the phone, Marilyn, this isn't working, this is and that. And these people don't use technology. I want her to, I don't know why she doesn't. She came from banking and she knows how things are supposed to work in the old-fashioned world or the world previous to AI and previous to online technology and things of that nature. So she really didn't want a hard copy check coming to their home. And I don't blame, I don't know what this company was doing, sending hard copy checks. So she asked for direct deposit, and I would have asked for that too. I don't want them going to the mailbox. Pretty soon the mail uh may not run every day because of cost constraints. Everybody has some constraints these days. So who owns that outcome for that question? Who was damaging? My stepdad was uh worked for the mine for what 30, 40 years, and all he had to do was go to the union and start complaining. That would have been damaging. And he goes to his other buddies that retired with him, and they start complaining. Oh, we got a mess on our hands then. So the question becomes: who owns that outcome? The analyst, the system, the vendor, or the leader who approved the system without defining accountability. And that's the moment, guys. That's the moment we're sitting in right now. Because accountability doesn't show up when things go right, it shows up when things go wrong. Okay, and if it's not clear then, it was never clear to start with. So, what does good look like, Marilyn? What does it look like? So, real accountability is named accountability. Every AI influenced decision has a single owner. Decision logging that you can trace what the system recommends, what the human decided, and why. Those are all in the loop going back to uh strike. There's a human override authority, leaders have the ability and the expectation to intervene, and there are predefined consequences. If the system fails, the response is already designed. That is accountability as an architecture, not reaction design. So, in my closing, let's bring this back to the core truth. AI does not remove accountability, it makes the absence of accountability more dangerous. Yeah, because decisions are faster, influence is broader, and consequences are amplified. So the real leadership question is not can we trust the system? It's how we design accountability for what the system influences. Because if you haven't, you don't have uh control. So before this next episode, do this. You're learning. Learn with me now. Identify three most critical AI influence decisions. And I need you to answer who is accountable by name, what happens when the system is wrong, and where is that accountability documented in the workflows? If you cannot answer this clearly, that is your leadership gap. So until next episode, we take one of the most comforting and most misleading ideas in AI governance. Human in the loop. It's time to talk about human in the loop. Because the most uh because what most organizations call oversight, it isn't. I'll see you in the next episode of Two Doctors in a Twist. I'm Dr. Marilyn Carroll, and this is part of our limited series on AI governance and leadership. Take care, guys. See you next time.