Building Systems That Don’t Break Under Pressure

95% of enterprise AI pilots fail to deliver measurable business impact — not because of the technology, but because of governance gaps, integration gaps, and systems designed for normal conditions rather than real ones. In this episode, Dr. Marilyn Carroll examines what happens to AI governance and leadership systems when the pressure is actually on: speed, uncertainty, high stakes, and incomplete information all at once. She identifies the three failure patterns that collapse systems in a crisis — and lays out the three design principles of resilient organizations: simplicity, pre-commitment, and rehearsal. The organizations that hold under pressure built for it before they needed it. This episode shows you how.

Stay connected with 2 Doctors & A Twist – Just What the Doctor Ordered!

• Subscribe on Apple Podcasts, Spotify, or wherever you listen.
• Follow us on LinkedIn for clips, insights, and upcoming episodes.

Transcript

SPEAKER_00 0:38

So, welcome back to Two Doctors in a Twist. I'm Dr. Marilyn Carroll, and Dr. Jamie Chessler is out, like I said, this quarter. We have three more episodes with Dr. Chessler. So, girl, if you're hearing me, get on back, finish that project so you can get on back to business, and we can get back to having our girl conversations that we have on this podcast. You understand? All right. All right. So let's get started on our episode today. This is episode 10 in our um series. And today we're talking about building systems that uh don't break under pressure. And so I want to talk to you about how 95% of enterprises who use uh who go through AI pilots fail to deliver measurable business impact. Not because of the technology guys, but because of governance gaps, integration gaps, and systems designed for normal conditions rather than real ones. So in this episode, I'm going to examine what happens to AI governance and leadership systems when pressure is actually on, when the pressure is actually on, guys. Now, speed, uncertainty, high states, and incomplete information. Ooh, ooh, all at once. My gosh. It's a bit much. So I'm hoping to help you identify three failure patterns that collapse systems in a crisis. And I want to lay out three design principles of resilient organization: simplicity, pre-commitment, and rehearsal. Yeah, those three. The organizations really that hold under pressure are built for, are really built for it before they need it. And this episode shows you how. The winner was built to perform, okay? To show how to perform under pressure, under unrealistic uh expectations, unrealistic conditions. Those are the people built for performance. In a leadership, we often say that we need different types of leaders. And people wondered, I even wondered when I was getting my PhD in organization and management with a specialty in leadership, because I really want to understand leadership. So I took all these extra courses to become a specialist in that area. And when you when you when you have that, um I there's dark leadership. There's probably about 12 different types of transformational leadership. There's transactional leadership, there's spiritual leadership, there's um, oh my gosh, laissez-faire. We have all types of leadership, okay? So that being said, every type of leader is not going to make it with every type of situations. Uh, for me, being the type of person I am, I was brought in and I'm normally uh do well, even in new companies where I'm brought to uh diagnose and resolve an issue when nobody else can do it, and you need a different type of transformational leader, may not be the most charismatic in nature at that particular point, but knowing what type of leadership is needed is my specialty in companies when they bring me in. What do I need now in order to achieve the goal I want to achieve on the next five years? Within the over the next five years. And that's very important. A lot of people don't think through that, and so they miss out on it. All right. So every system works when conditions are normal. Really? Okay, that's when you can have a normal leader, one that can do well no matter uh if if if the conditions basically are normal that they're operating in, right? Yeah. Well, when the problem comes in, it's with leaders who are not built for that. Uh processes flow, decisions feel clear, governance holds solid. Um, some people do uh a lot of people do well with that. But if the conditions, if everything is not working, or it looks like it's working, hey, that's when we have real problems, guys. When the pressure really hits, then we got some challenges. We have some challenges, right? And those challenges uh hits at and in that moment you find what your systems were actually designed to do, what your management was really capable of. The other day I heard somebody say they took um an assessment, and um what is it, uh, where we do this um NPS on uh how people are doing in companies and how leaders are doing. And the leadership was 60 or something like that. And listen, that was when you were talking about those numbers, you only get the numbers that matter to you that HR sends out. They're sending those to you because that means that you have that challenge at your senior leadership levels and below that. That's what's going on with your group that reports to you and what your group thinks about you in that particular scenario. All right. So I want you to understand that. Because here's the truth because when those numbers go to the board and they see those performance numbers, if that was high up and they said that about the most senior leaders in the organization, the the ones, the CEO, the chairman of the board, and all of that, if that was the case and people didn't believe that, then the investors would have a huge problem with that. Right? Yeah. The investors would have a huge problem with that. And so would your board of directors because they're gonna say some things and say this is not gonna work. Now, I want you to listen to this carefully. Here's the truth: all AI systems don't fail most often in control environments. They fail when speed increases, uncertainty rises, stakes are high, information is incomplete. And we are in this moment at this particular time where it's not just AI that's disrupting things, is the financials are disrupted, money is tight, the environment has changed, government has changed. Ah, we have a lot of things happening at one time, and it's just one thing right behind the other. And this, this, this in and of itself is causing pure chaos for organizations, especially those, excuse me, providing education. This is when systems are tested. Yeah. This is when you're gonna know who's working, who's worth it, and who's not, and who can handle it. This is when you have the horse we had in the Kentucky Derby and the rider we had on that horse. All right. All right, or Indiana football team and how they won out of nowhere, or how uh Seattle uh won the Super Bowl based on the people they had there and and what was happening there. I watched this, I watched this in sports, I watch it in business, I watch it in education, and I watch it in enterprise um accounting or business systems. So let's ground this in reality. You're saying, well, Marilyn, what is the reality here? What is going on? So let's start with this. So, research from Massachusetts Institute of Technology, MIT, said up to 95% of enterprise AI pilots fail to deliver measurable business impact. Why? Well, not because the models don't work, they work, but because really, because governance is unclear, yeah, integration is weak, and systems are not designed for real conditions, guys. They're not, they're designed for the best conditions, and data from audit board shows 44%. The audit board now, 44% of leaders say governance is too slow, 24% say it's too complex. And I get you, I've I've faced this too with um back and forth through my LinkedIn conversations with different people who are building governance, especially UK. I think they're ahead of everybody, and that the UK wants specific things that they're requiring of organizations that are touting how AI is great and works, and that they have governance. One said to me, Well, Marilyn, I can't stop and put a person to opine on something at every step of the way that a decision has to be made. That in and of itself is not going to increase speed. If I'm using AI to increase speed, I gotta do something else. I I get you guys, I get you. I get what you're saying. So, what happens under pressure? Well, I'm gonna say this to you: the things that handle that happen under pressure are it gets wrong, we stop using AI. Oh, it's a whole list of items I can go on and on with you on. Because AI, it confuses a lot of us about what it is that's happening. And so, in the grand scheme of things, we have to be prepared for those things. Because if you bypass governance, we have these challenges that organizations are not having, not because the people don't care, uh, but because the system can't keep up. It just can't keep up with what's going on because of the constraints we put on it, because we never did build governance into the process, because the system didn't keep up. There are problems, yes. Um, and many of you understand what those problems are. So, how do these systems break? Well, we see three patterns. I know, I get what you're saying, and this is what has happened, and this is how a lot of us came to build these governance uh processes that you see all of a sudden a lot of governance out here that companies are saying they have. Uh, we have the complexity cats, okay? And you're saying, what is this, Marilyn? Too many steps, too many approvals, too many dependencies. All right. Under pressure, the system collapses when this is the case under its own weight, because it can't take it, because it's not built for those things, it's not built to keep stopping. And then we go from complexity cascade to authority confusion. So, what do you mean by authority confusion? So, what I mean by that is no one knows who decides. So, everyone waits or defers, right? Uh, oh, wait a minute. That's Susie's fault. That's Paul's. Let's get Paul in because that's Paul. Oh, I just had this the other day in a meeting. Oh, that's Maryland. Maryland needs to take care of that. No, no, no. When speed is required, and guys, listen carefully. We are at a time when speed is definitely required. We can't take it moving slow. We are like a train, and we have to be, we have to have our precision, we have to get to a certain from point A to point B by a certain time. We gotta, even in bad weather and with bad conditions, we still have to get there. This brings us to number three accountability. Dissolution, I'm sorry. Something goes wrong, right? And instead of action, there's hesitation. Why is that hesitation? Because accountability was never clear to begin with. Why was accountability never clear to begin with, Marilyn? Why was it? Because under pressure, we didn't, we just built the system and we we thought it just moves naturally because in a regular system, day-to-day operations, the system moves naturally, right? Well, under pressure, ambiguity ambiguity doesn't hold, and that's what you're gonna have. Uh, when things stop, there's some ambiguity going on, it breaks because of that ambiguity, right? So, what do we do? Well, I think we come to four pressure variables, yeah. Four pressure variables. So I want you to write these down. Get a pen and paper and write these down because I I do want you to remember this, okay? We have speed is one, decisions must happen quickly. Every system that is tested across these four dimensions must be done because it's very important. So we start with speed, which means that decisions must happen quickly. Uncertainty is the next one. Information is incomplete. The third one, I'm sorry, speed is the first, uncertainty is the second, the stakes are the third. Because consequences, they really do matter, right? And then we have the fourth, complexity. Multiple factors interact as one at once in these AI technology driven patterns and systems. And and to be honest, here's the reality, guys. Most governance systems are designed for stability, predictability, and time. I'm going back to my Sarbanes Oxley days and uh auditing days, tax return days. You gotta get those right before you get under pressure from the auditor. Thoughts the same thing. Any type of system must must address things before pressure points are put there. And they must be able to operate under those pressure points, and that's why that's why these new systems are failing because they don't have those four guardrails in place. So leadership decision moments is this part I want to get to because I need to make this real for you if you you haven't felt it already. So I hope you got those four guardrails down at first uh that I just spoke about. And I want to make this real uh for you, and I want to type it, I want to type so bad because I'm writing on notes here to make sure that I'm covering everything that I want to cover. An organization faces a sudden operational disruption. So we're gonna start there. Say an education, um, like we had this week, uh, or last, where the system went down, where the learning management system was hacked. What was your cause of operational disruption? The AI systems are generating recommendations, and the teams are waiting for direction. So, say you had that. That system is not operating, but you have other systems that's watching your learning management system, and it's not it's not operating the right way. The system, the situation is evolving quickly. How do you get done? How do you move? What do you have to do? Millions of people will without their educational system. And if you're in college, we all work at a certain pace. If I'm a college student, I'm not here to work when your system is working. I need your system to work always for me, and that's the beauty of online education. That's the beauty of education today. So, this situation that happened when the learning management system was down was evolving quickly. And leadership has to decide, what am I going to do with this? And you can just imagine the number of people that were in backrooms, corners, and all of that saying, what are we going to do? Because we got to get this back up and running, right? But no one, imagine, no one is sure. You have this learning management system at your organization, and nobody is sure who to hold accountable for the results or who to get answers from, and what's the next steps. Now, in banking, because I worked in banking and finance and wealth management for 17 years, and we always had somewhere to go in that practice. We were preparing for guardrails, we had governance processes in place. And so when there came a problem, when the pandemic came, even though I wasn't there at that time, they were prepared for that because we practiced for it. We practiced in advance. We knew to start having people work from home at least one to two days every week, so that they were accustomed to working for home in case we had a crisis. We had three other centers that linked to somewhere else in case all of the utilities went out in the primary system. There were two other systems where we would get uh our energy from to operate those systems. And we had offices set up, warehouses set up in other places where our team could go and work. So for us, that department was flawless in implementing when the system went down during the pandemic. They never ceased working because when you're working with people money, you cannot take it that you're going to be down when it you got to keep their checks rolling in, these retirees, uh, as well as investing money and exchanging money or selling assets and things of that nature at that time. So we couldn't have it where who has final authority. We had to know. Whether AI can be trusted or the system could be trusted, we had to know. And what happens if the decision is wrong, we had to know that as well. And we have to implement those things in the system. Same thing happened in 9-11. We knew who needed to be on deck and who would have the final say in those situations. Okay, so people don't hesitate because in if there is hesitation, the organization loses momentum. It really does. And that's what happens when these systems are going down because of cyber acts, cybersecurity information, all of that. We should have a process in place. That prepares us for that. Not because they lack data, but a lot of companies don't know how to utilize the data that they do have, how to put that together. And I explained that in our last episode that we had, because they lacked design. They really did. Those individuals lacked design in what they were doing. So as we bring this to a close, resilience systems are built differently. And I and I hope you got that from me. I hope I'm not sounding boring or anything like that. They are designed for pressure, guys. Pressure to perform through these three principles that I'm going to talk to you about now. Simplicity. Complex systems fail faster. Okay? Make it simple. Clear structures hold, right? And through that simplicity, complex systems fail faster, but clear structures hold. And then pre-commitment. Decisions are defined in advance, not created in the moment, right? They're they're they're designed. You know what's going to happen. Just as I said to you in my example, with how we prepared through socks for in case there was an emergency. And that all started with 9-11. So uh, and then rehearsal. We rehearsed every week because that was embedded in our process. Okay. Systems are tested before they are needed. All right. Not when they need it, and then or not after they break. You know what's going to happen if the system does break. You know what's going to happen if the learning management system goes down. This is exactly why governance must be architectural. I I couldn't give it to you any more simpler than that. This reactive stuff hurts everyone. So the goal is to eliminate reactions, reactivity, or reactions, and have a plan in place. So you may be asking, so Marilyn, what is the leader's role? Well, here's what leaders need to understand, guys. When systems are under pressure, you become the stabilizer, right? You just do. You become the stabilizer. Because if you hesitate, the system fragments. It does, it really does. And we can't have um fragmentation, right? We don't need that. Because what happens when you have fragmentation? What happens with that? Because the moment really, people are not looking for data at that moment, in that moment, they're looking for the reaction of the leader, right? They're looking for direction. I remember when 9-11 happened and we were looking for a new daily valuation system. I was in, we were all in Denver, Colorado, but I was working in Atlanta and all of us was there. So my manager, my manager's manager or leaders, all of them stayed back. And something inside of me said, go ahead and fly back tonight. Because I have a family, I have kids at home, and I wanted to make sure my children uh saw me because I had been gone on the road for so long. I had we had achieved everything. We struck the deal, we wrote out the process, we agreed, we were all there. And I decided to take the what is it, the flight was a flight at 2 a.m. in the morning back to Atlanta from Denver. So I hopped, I went to the airport, hopped on that flight right after dinner and conversation, great time. Okay. And I got on that flight, I made it back to Atlanta, went home, took a shower, got ready, saw my kids, made breakfast for them, all of that. Got in my car. It was had been sitting in the garage. I got in my car to head to work, which I had an hour and a half drive to work. Atlanta traffic. You know how that works. I stopped to get gas at the QT, and um I had the news on. It popped on the news. One plane had been hit. By the time I finished pumping my gas, another building had been hit. Immediately I knew this is not good. And the crew that was with me wasn't coming back till the next day. And if that's happening, then the airports are going to be chaotic if anyone ever gets out. Because as I was driving, I got in my car to head in, driving to work as fast as I could. I noticed the roads were not as condensed as they usually are. I did make it to the office, but I got a phone call. Yes, cell phones were great at that time. I got a call in my car. Marilyn, what are we going to do? What's going on? Do we need to go to work? What? Yes, we need to go to work. Because again, in banking, you can't stop. You have to be available and have people's money available to them, especially if there was something more dynamic happening at that time. You cannot just quit the process, right? So being that I'm the only one left there in the building that can sign off on money transferring between parties and to listen to the investments we had just submitted for sale or purchase and to pool money. And this happened to be at a time when we wrote checks to retirees. And that in and of itself comes to lots of money, millions of dollars. Because we had a huge trust department at the time. And oh my gosh, I have to be calm, remain calm at that time. But I would not have been able to remain calm in all honesty if we didn't have a process in place, if we had not practiced the process, if there was not a governance process in place. All right. Because of that, we were able to move. I'm able to get people done first, remain calm myself, because people, as the leader, are watching me and what happens in my personality and what I'm doing. My kids, the school lets out at the time I was married. And so my um husband at the time was able to be home first because he worked in the area. So he's able to be home with them when they come home from school and watch all of that. For me, and people knew the type of work I had, that meant for me, either I was going home or I was going to stay at my place in the downtown area at that time. I went home. I did drive home. I didn't stay at my place in the downtown area. But I went home at that time to my home in the suburbs with my kids because they're gonna need me after I got everything structured and in place and settled and mapped out a plan for later on. Now I'm able to talk to those that's still in Denver at the time because they didn't have any way home. They were stuck in Denver for a couple of days because they couldn't find a car to drive out of Denver back to Atlanta. So they were out of the equation, other than uh saying or authorizing what could go down. So there you are. You were stuck with you having to, or somebody having to be the person that answers the call, that answers what's going on, and able to articulate and put together what is going to be next, right? That experience in and of itself taught me a lot. And because I was prepared for that, I paid attention, I listened, put governance in place so that for my team, they didn't have to worry. So, because they worked from home before, we're able to put in structures because we needed to make sure another center was up and running. Um, since all of this was happening on the East Coast, we had to make sure our center in the Midwest or on the West Coast was operating. So a lot was going on at that time. But if you don't have governance in place to start with, you won't know how to operate or do the things that need to be taken care of. I think we forget a lot of things and think, oh, we don't need all of that. We need to cut down on staff, we need to cut down on this, that, and the other. No, you don't. You need to always make sure you have governance in place first. Because you're gonna have some 9-11s, we're just gonna have them. We're going to have uh some cyber attacks, we're going to have uh some pandemics, we're going to have all of that. If you study history, you know we're going to have something that happens. So you need or must be prepared for those things. Because systems don't fail unless they weren't designed for uh for realities that may come into place. As I just shared with you, there are realities that are there that will happen in these processes. Okay. Now, AI increases speed, guys. It increases complexity, it increases consequences, and if your system cannot hold under these conditions, it will break. Not eventually, exactly when you need it most. So, with that said, I would say to you, and this as we close out this segment, before the next episode, I need you to do this. Take one AI-driven process in your organization and ask yourself this. What happens when speed increases? What happens when information is incomplete? What happens when the tasks are high? And who decides under pressure? If you don't have clear answers to this, your system is not ready. So in our next episode, we step into the future of leadership itself. Because systems are changing, if because if systems are changing, then the role of the executive, yeah, is changing with it. And most leaders are still operating from an outdated model. You can't, you have to change. And we thought education was dead, it is alive and well. So for me, I'll see you on the next episode of Two Doctors and a Twist. I'm Dr. Marilyn Carroll, and thank you for joining us.