Learn to navigate your next project retrospective with ease!
Galen Low is joined by Payson Hall—Principal Consultant at Catalysis Group—to talk about project retrospectives: how to run them effectively and how to ensure that the lessons you capture in your retrospectives actually translate into improvements for future projects.
Learn to navigate your next project retrospective with ease!
Galen Low is joined by Payson Hall—Principal Consultant at Catalysis Group—to talk about project retrospectives: how to run them effectively and how to ensure that the lessons you capture in your retrospectives actually translate into improvements for future projects.
Hey folks, thanks for tuning in. My name is Galen Low with The Digital Project Manager. We are a community of digital professionals on a mission to help each other get skilled, get confident, and get connected so that we can amplify the value of project management in a digital world. If you want to hear more about that, head on over to thedigitalprojectmanager.com. Okay today we are talking about project retrospectives: how to run them effectively and how to ensure that the lessons you capture in your retrospectives actually translate into improvements for future projects. With me today is Payson Hall, founder and lead consultant at Catalysis Group, who has been helping organizations learn from their project missteps for well over 30 years. Payson, so great to have you on the show.
Payson Hall:Hey, glad to play and glad to meet the audience.
Galen Low:Yes, actually and I'm glad you mentioned that because we are doing something a bit different today. We are live with our members as our studio audience, and we're going to be taking questions on the fly via our Slack community. And Michael's in the studio with me as well, so he's going to fill some of the questions. We're going to work it into the conversation and anything can happen. So let's get lively. All right, we're going to dive in. I thought I'd start with a juicy question because based on, we've been talking about this and you've been doing project retrospectives as a consultant for decades now for all sorts of projects, including software projects and large scale systems integration projects. And I wanted to ask you this, when you come in as an external consultant to unearth all of the project's dirty little secrets, do project teams, do they view you as a friend or a foe? Is it like them getting mentorship? Does it feel like they're getting audited?
Payson Hall:Normally, it's a friendly exchange. I'd really like to make a distinction because one part of my consulting practice, I'm doing project autopsies. If a project goes severely off the rails and there's a concern that people knew it was going off the rails and didn't say anything, that's much more like an audit. In fact, I work with a state auditor in California and I've audited billion dollar IT projects. And there, what they're usually trying to do is build a court case for the lawyers to go after a systems integrator. A more common and friendly retrospective is just people saying, Hey, we finished this project. Let's get someone in to facilitate a look back, right? That's what a retrospective is. A look back to say, what did we learn? What did we do that was clever? And maybe we didn't notice it at the time. What did we do that in retrospect we might've wanted to do differently? It's really a risk management session looking in your rear view mirror. What can I do better the next time? What did I do that was clever and I want to make sure I do more of it.
Galen Low:I really like that distinction too, because I think some folks listening might not actually feel that distinction as much, like where every retrospective feels like an autopsy, feels like they're going to be like evidence in court, feels like a bit of a blame game versus just like you said, looking in the rear view mirror and learning from your mistakes.
Payson Hall:One of the only rules, big rules about retrospective, there's a variety of ways to do them. But one of the big rules that I would suggest is you never want to, as a project manager, conduct your own retrospective, because some of the things that didn't go well are things that you did. And a good facilitator can make it, you don't get nailed to the wall, good facilitator makes it about what happened, when did it happen, what did we learn from this. Not, well, Galen, you really screwed up, right? That's not constructive. People get defensive and gets off the rails. So whether you bring someone in from the outside, or you've got someone you love and trust, who's a good facilitator internally, you do want an external party, somebody who wasn't involved in the project that has no skin in the game to facilitate the conversation so that it's, they're not, we don't want a lot of blood on the floor when we're finished.
Galen Low:That's fair. And actually that's a really interesting lens because I know a lot of folks crack this open to listen to it, to try and figure out how they're going to run their own retrospective. And by all means, I know that's a reality for a lot of project teams and we will cover off on some of the tips and things that you've learned along the way. But I love that advice that it can be somebody else, if not should be somebody else to give that perspective and someone who can make it something that won't devolve into a blame game. And there won't be a lot of blood on the floor, as you say afterwards.
Payson Hall:Well, someone who can come in with a different set of assumptions. I mean, think about projects that you've been on. As you went through the project, you made assumptions about what the causes of grief were. And they might be right or they might be wrong. Somebody coming in without those assumptions might be open to other information or other ways of interpreting.
Galen Low:I love that. Yeah. I love that sort of a perspective of someone who wasn't in there for whatever 12 to 18 months on a long haul project, feeling very close to it and not necessarily seeing the forest for the treats. I thought maybe we can just start with the broad stuff. Maybe let's zoom out a bit and just to give our listeners their bearings. Can you explain from your perspective what a project retrospective is and why it's worth it to do one?
Payson Hall:Great question. Okay, so a retrospective, the word means to look back. And the idea is we get to a significant moment in the project, or we get to the end of a project. And we look back and say, all right, let's catch our breath. What did we learn in this last segment? What lessons learned do we want to apply to the next segment? Or do we don't want to apply to the next project? What did we do that was clever accidentally, right? Or what did we do that, Oh, if I had it to do over again, right? If I could send a message to the time machine, send a message back to myself, here's something I would have wanted to do differently. So the idea is really just take a breath, look back on project events in context, right? Like you said, zoom out and get the big picture so that we understand what's important and what's not important. Some things, it's just, they're black swans, right? They're, Oh, this was an unexpected event, COVID, right? I am not going to plan for pandemics in most of the projects that go forward, but boy, if that happened to your project, it was a hit. So, I don't know, there might be some lessons learned there, but some things are just, no. Some things, it's maybe you needed a dozen laptops, so you ordered 14, in case a couple were DOA. And it turns out one of them was DOA, and that was really clever, because you were on a tight time frame. That's oh, okay, you know what? Contrasting the cost of the other two laptops with the delay avoided, that might be a really good lesson learned. Right? And it's not that the extra laptop is scrap, right? We might be able to return it. We might write a contract that says, I want the rights to return it if I don't open it. And it might be, it's just, that was a thousand dollars really well spent. It saved us three week delay. If you've had to work with hardware in the last three years with supply chains all screwed up, you know exactly what I'm talking about.
Galen Low:For sure. No, I love that example. And I love something really important that I think that folks don't normally think of is that yes, retrospectives can be a moment of reflection on something that went really well, that clever thing that was awesome. And I, you get it in retrospect to sometimes you're like, okay, what went well? And they're like, okay, yeah, the team really worked well together and we communicated well and blah, blah, blah. But in terms of a crisp lesson of oh yeah, remember when that person booked that other alternative meeting room? And our number one venue got canceled or something happened to it and we had a backup? That was amazing. That should be something that we can learn from and we can bake into our process next time. And to your point and we make sure that we have a term in the contract that says if we order an extra laptop, we can return it if it's unopened and we've covered our butts and we can use that in future projects as something that was a clever learning. Other thing that I really loved that you said was you said retrospectives happen either at the end of a project or at a significant moment within it. And I think a lot of folks that I talked to are like retrospective means it was the end and it's too late. We can look in the rear view mirror but there's this kind of like forlorn tone, right? Where you're like, yeah, this could have been better, but it's too late now. We have to wait for the next one. Versus actually some other spots within the project to have a retrospective. I'd like to come back to that later on as well.
Payson Hall:Yeah, that's actually one of the interesting things that we get from the agile approaches because they typically do a small retrospective at the end of each sprint. I've worked on projects where we did them on a milestone basis on larger projects, but I think Agile said, Hey, wait, why are we doing, why don't we do these episodically? Because we're doing it more frequently, they're smaller. The team ends up understanding the process that can get into it, get it done. I think that was a really nice innovation that the Agile guys really crystallized when they talked about doing retrospectives at the end of each sprint.
Galen Low:I love that sort of mindset shift of, actually, we can do this more regularly and it becomes a smaller thing. Because just to be the devil's advocate on this all, I mean, I think it stands to reason that a lot of folks, especially a lot of organizations, they're going to say, oh yeah, learning. We want to learn from our mistakes. We want to fail fast. We want to do all these things. But when it comes down to it, like not every organization is willing to invest in this, like looking backwards, this rear view mirror, they're like, what, we learned it along the way. We don't need to have a meeting about it. Let's just go on to the next thing. But I thought just like in terms of the overall investment, and like you said, bring in someone from the outside as a facilitator. And if you're not doing it, sprint over sprints or at every milestone, then maybe it's a big huzzah at the end. But what is involved? What does it take to conduct a proper project retrospective? And is it worth it? If I was playing devil's advocate as an organization being like, I don't know. I think people felt the pain during the project. We should just move forward. Let's not bring in Payson. Let's not have a retrospective. We'll just keep moving forward. What would you say to that?
Payson Hall:That's a great question. Whenever we talk about process improvement, we want to make sure that the cost of the lessons learned, the cost of the improvement doesn't exceed the value of the improvement, right? So what's it take to conduct a retrospective? Let me give you an example, walk you through. And then the first thing I'd say is context matters. If we're talking about multi-million dollar or billion dollar project, it's going to take a lot more effort than a small project. There's going to be a much more data. If it's a troubled project, it's going to take more effort than one that went relatively smoothly. If a project crash and burn and lawyers are involved, you're not doing a retrospective, you're doing an autopsy and those can be very expensive. So here's a data point. I conducted a retrospective recently with a team of about 12 people for a project that went on for about 18 months. It was an internal project, but a dozen people on the project team project went about 18 months. Met requirements, ran a little bit late and that was acceptable. No lawyers were involved. The question was, okay, Hey, we've gotten to the end. Let's catch our breath. What did we learn from this? What can we do differently? So individual team members probably spent four to six hours each in preparation and conducting a meeting and doing the debrief. As a facilitator, I probably spent a dozen hours preparing and facilitating and writing stuff up at the end. The project manager probably spent 8 to 12 hours getting me information and walking through stuff. And so, that retrospective probably consumed about two person weeks of effort. Okay? Two person weeks of effort. Now, if we imagine a dozen people, plus the project manager working for 18 months, that's about a thousand person weeks. So what we're saying is 0.2% of that was spent with a look back, trying to understand what'd we learn from this? What can we do differently? What do we do that was clever? So think about it. All you have to do is find a way to say two person weeks worth of effort, and you've got a positive return on investment. And that happens in a couple of ways. It can be structural, right? Hey, here's a recommendation to the PMO to do this thing differently that's going to save you time and grief. And it also can be organic. It's Hey guys, you know what? We went out of our way in this project to make sure that we did a code review, just picking something at random. And that really helped. We experimented with it. It really helped. Now, everybody in the team, the next project, a lot of times they explode like a seed pod. They go off, they might work on the same project together, might work on different projects. They're carrying those lessons learned and can advocate for them on the next project. So, I've never seen a retrospective where we didn't come out, we didn't at the end say, Hey, here's a list of findings, recommendations, and suggestions and I think that this is worth at least the two person weeks that we invest.
Galen Low:Fair enough. And I mean, we're talking about larger projects, but is there a small, like a lower limit for smaller projects? Do your project only lasted four weeks? Investing two person weeks into it seems like the ROI might not be there. Do you just scrap it and say, no, we can't we won't do a retrospective on these. Do you aggregate them? So you have five small projects do a retrospective together or?
Payson Hall:No, that was actually a mistake I made early in my career. You don't want to have anybody in a retrospective that wasn't involved in the project. Just because let's assume that the team is not at war with itself. The team is, if you're going to have a candid conversation about what worked and what didn't and all, yeah, I, I screwed this up. I could have done this differently. People are much more comfortable doing that with an intimate team that they've worked with than a bunch of strangers, so I would never would consider aggregating. After we'd done a series of small retrospectives, getting the project managers together and saying, what are the common threads might be an interesting thing that PMO wanted to do. But for a retrospective, I don't want any, I don't want any tourists. I want to make sure it's just the team. To your question about, can we do this on a small scale? Absolutely. The formal process that we'll talk about in a minute, I'll walk you through. The thing is on an 18 month project, people feel like they come in every day and they hop on the hamster wheel for 18 months and there's some significant events, but it's easy to lose context. It's easy to, like you said, not see the forest for the tree. And so taking some time out and saying, here's some significant developments along the way can be really helpful. And investment in two weeks, we were talking about. I could see that on a three or four week project, maybe getting together for an hour, we could probably take care of most of what needed to happen. I'd probably want to prep a little bit and make sure I understood the project, understood, from the project manager, what might they be looking for? But yeah, you could probably do a retrospective like that pretty informally in an hour or two.
Galen Low:I think you raised it earlier, right? And in an agile sort of framework, you might be doing a retrospective of some sorts every sprint and yeah, maybe it does require a sort of reflection at the end of the project to bring all those learnings together and make recommendations from them. But I like that notion that those two person weeks, it doesn't all have to happen, at the end of a project. And that was even a project that is whatever a team of 12, and pretty large scale, there are smaller ways to do that. So, yeah, I want to dive into that when we get into the process as well.
Payson Hall:Yeah. We'll figure if you had a team of eight getting all the wires in one room for an hour, there's eight person hours. Maybe a couple hours of prep, a couple hours to the project manager after you might be able to pull the whole thing off in 12 hours. And that includes coming up with a one page bullet list of here's what we learned that along the way.
Galen Low:And there's the opportunity costs here, right? I mean, I know I was playing the devil's advocate earlier, but you know, if you spend that hour and uncover something, a lesson that will save you half a million dollars every time, well, I mean, it's paid for itself tenfold. So there's that sort of the cost of not doing a retrospective. It's something to think through. You mentioned something earlier, you were talking about internal teams as your example. I know a lot of folks listening, they work at agencies and I know it comes up a lot where, they're working for a client. The client doesn't want to pay for the retrospective because they're like, that's how you get better as an organization. Meanwhile, their leadership team doesn't want to invest in it either because they're like, well, this is not billable time. So what we're going to sequester eight or 12 people, for X amount of time to talk about how a project went and we won't make any money. No, let's not do that. And I've actually been in that. I've actually had a retrospective. I've had a CEO walk in and be like, everyone get out of the room. This is a waste of time. Get back to work on billable stuff. And that was a sort of big wake up call for me to be like, Oh yeah, not everyone finds this valuable in like the business mechanics of an agency. But just in terms of people who are leading a project and seeing the value of a retrospective, like how can they advocate for this type of learning and how can they convince their owners, like agency owners and the leadership teams to see the value of a retrospective that's not billable?
Payson Hall:I want to be polite, but if I was working somewhere where the CEO came in and said, we don't need to do the next project any better than this one. And there's nothing better to learn. I get my resume together. That would trouble me greatly. Before I left to start Catalysis Group, I worked for IBM professional services for four years doing multi-million dollar projects. And you're right. Typically, the client doesn't pay for this. The organization pays for it. And the organization pays for it because can we, what do we, what could we do better in terms of customer satisfaction? What can we do better in terms of efficiency? Where did we misstep so that we can do process improvement? Particularly consulting firms, you stop and think about, what are you charging by the hour? How many hours do you have to save before it has value? And just to point out for people that don't do consulting, Mike Billing rates $2000 a day. That's not what I get paid. So when I'm doing administrative time, it's not costing the organization $2000 a day. If I do admin time with a group of, six people for an hour and we figure out, Hey, here's a way we can save four days on the next project. We just massively paid for the retrospect. So, I don't know the context. It could be the organization you're talking about was bleeding money. The CEO needed you guys generating revenue, but in general, in my experience, it's not a hard sell. And one of the things that you can do is you can start small. You were saying, okay, what if it's a short project? Great. Let's get together over lunch. Let's get together. Set up an hour. When I was a puppy, I worked for a Price Waterhouse and it was amusing to me that every month the partners had a special breakfast for the staff and it started at seven in the morning. And we all showed up at seven in the morning and they bought us breakfast and we thought we are skating, man. We are so bougie. They're buying us breakfast in this fancy place. And the partners are all thinking, I just tricked all these dupes into coming into work two hours early for a staff meeting. And they're happy because I fed them. So yeah, I think professional services agencies generally, they're trying to build things up. There can be mitigating circumstances, right? If something's on fire, I can understand the CEO saying, no, put out this fire, then come back. But I think start small. I think the value retrospectives will demonstrate itself. One of the things similar to risk management too, is when you get a nugget out of a retrospective or you get a risk management session, saves you a bunch of grief. Tell people, Hey, we did this retrospective, came up with this idea, we applied it on the next project, it saved us two weeks. That's how to convince the execs that it's a good use of time. But start small, so it's not a huge investment.
Galen Low:Start small and get some evidence, right? Proof points to be like, Hey, this created value last time we did it. Maybe ask for forgiveness rather than permission in some cases, if you're doing that, and then get the proof points that I think that's huge. Because I think a lot of the time it's just that they're like, decision makers might not have proof that this is going to create value and there's money on the table there to grab, right? There's a project happening right now, billable hours, all eight of those people in the room could be doing that and not yet seeing the proof that a retrospective can actually evolve a business, can actually optimize a business in a very valuable way.
Payson Hall:Well, one of the things that's frequently an output of a retrospective is a document or a slide presentation that says, here's the highlights. And that can be sanitized for the execs. Maybe there's the internal version and then what you share. But if it's well formed, it can help let the execs understand that sometimes they're the cause of problems, right? Redeploying resources with minimal warning or, saving the wrong dollar. We don't want to pay that night of hotel, so we're gonna fly you on a red eye from here to the east coast. It's okay, you know I'm gonna be brain dead when I arrive. So it seems like gentle ways to cover those kinds of lessons learned, it can help steer the executive's thinking towards more productive interaction.
Galen Low:I like that as a sort of strategic communications thing, especially the, maybe they don't get the same version. They get the leadership team version with the learnings and not all of the not all of the dirty laundry necessarily in the team version.
Payson Hall:Absolutely. I don't like the dirty laundry metaphor so much. It's like one of the smartest things anyone ever told me is, most people are trying to do the right thing the best that they can with the information available, and it's very easy to come in. If you want to be a jerk, you can come in a retrospectively. Oh, you guys didn't do this. You didn't do this. You fail to do that. It's that's not the point. You assume people are doing the best they could with the information available. Then the question is, what information did you have available? How can we get that for you sooner? We need to have a constructive approach to this because if people think you're looking for someone to crucify, you're not going to get anywhere. It's not going to have any value.
Galen Low:Fair enough. And actually, I want to dive in there. We said, we'd go through the process. And so I figured let's just assume that we've got everyone on board. We're going to do a project retrospective. What does your process for project retrospective look like? Could you walk me through it, as if I was the PM on the project in question?
Payson Hall:So let's assume we're finishing a project that's been going for a year. Okay, so it's a big project. We had a team of 10, 12 people, just so we've got some scope. First thing I want to do is have a meeting with you. You're the project manager. I want to have a meeting with you. Basically, it's establishing rapport. And if you invited me in, this is easy, right? If I was inflicted on you by executives or by the project office, one of the things I need to do is encourage you to trust me. You need to explain what the process is so you know what to expect. And one of the things I'm going to try to do is say, Look, I'm going to promise you this. I am not going to surprise you. Any findings I have, you are going to see them before anybody else sees them. And you'll have a chance to argue. You maybe changed my mind. You can give me other data. You can give me another way to look at this. So that takes a lot of the stress out of it for the project manager. That's good for both of us. I don't need them to be afraid of me. I don't want them to hide the ball. That's actually one of the biggest problems I'll have is if people start hiding the ball. It's imagine you're talking to a cop and you start acting really suspicious. Their ears perk up, the hair stands up on the back of their neck, and now they can cheese the dynamic. So much better if we're having a conversation. So once we've established rapport, I'm going to explain the process to you and tell you the very first thing I want to do is I want to understand what did you set out to do? Are there any definition documents? Are there any, reports or memoranda? What was the definition of the project? Now, you know, good project management practices say we would have some kind of charter or project definition that said, what are we trying to accomplish? And some people do better jobs that than others, but I need to understand what did you think you were trying to do? And then I want to understand how did it change? So typically what I'm gonna do is I'm looking for documentation, right? I mean, I'll have a conversation. What did you think you were trying to do? And they'll say, Hey, is, was that written down anywhere? Was there a written definition of what you're trying to do? It could be a requirements, could be a project charter, whatever. And I'll talk with you. How did that change along the way? Were there change orders? Can you share those with me? What I'm really trying to do is understand where did you start and how did that evolve over time? All projects evolve. The longer the project's going, there's gonna be tools that get swapped in, versions of software that get updated. So it's really helpful for me to take a step back and understand those. And jumping ahead, one of the things I can do as an expert is imagine that you were working on a one year project. You hit a speed bump three months in because they needed to do an update to PeopleSoft and it, you know, all the king's horses and all the king's men took a month to put it back together and your project's a month late. One of the things I can do as part of the findings is say, that's not on you. This is an unfortunate event in the future. Either we should look to say, are we going to update that software during this project? And if we are, make sure we allocate time in the schedule. Right? So I can help put a spin on that, that you can't, it'll seem to you like you're being defensive. If you say, well, wait a minute, they updated this in the middle, I can come in this next year old party and say, no, that's that happens. Well, that asked was terrible. Yeah, but it was reasonable with the information available. But anyway, so I'll chat with you, set what to expect. My first step is to try to get as much homework as I can so I don't have to trouble you. Definition, change orders, risk management, any status reports that you've done, hit to your listeners, one of the best ways to get a project management education is sit down with a year's worth of status report. Just sit down, get some coffee and read through a year's worth of status reports. And what you'll see is sometimes, in month two, there's this little event that happens. And in month three, it's actually a slightly bigger event. And in month four, it turns out it's a full blown problem. And it's not that people were hiding the ball, they didn't realize the significance. I was working on a project in a project review. This was actually more like an autopsy. I didn't work with the team. I just got the data. And it was a hospital that was supposed to be installing a new system. And the first month they said, yeah, we're going to be moving to our new quarters at the end of the month. And the second month they said, okay, there's a problem moving to the new quarters, but we should be able to move there at the end of the month. And in the third month they said, okay, they hit another snake, but we're going to be, and it's this is a one year project and they need to have the facility they were supposed to work in. And the project manager, I'm not faulting the project manager had no sense of urgency. What they were doing was responding to the facts of the gramble. It's this outside of my control. It's going to be another month. Where if I were in that position, I might want to send up flares and scream bloody murder. It's hey, we got an issue here. We either need to escalate this or I need to reset your expectations about the one year project. But anyway, so I'm going to look at status reports. I'm going to go off sequester myself. For some reason, I always think of this happening on a Saturday, go through all this and I'm making notes about anomalies. What were the places where the schedule jump? Was there any conversation in the status reports or in the risk log or in the change log that explained the jump? So I'll make notes off to the side and I'm going to build a crude timeline in Excel or Word just to get myself working because I wasn't there. I'm like, after the fact historian, trying to write down the highlights of what happened. Then I'll have a conversation with the project manager, try to fill in the gaps, maybe get some additional information. Once the project manager understands I'm there to help and I've gotten myself oriented, we'll ask the team to spend 60 to 90 minutes preparing. And what I wanted to do is think of three things. What were interesting events that happened, like people getting married and having kids. During the project, what are some interesting things that happened? What are some things that happened that were fortunate or clever, right? What'd we do? Oh, we did this accidentally, but it was a really good idea. Or, we went to the site and we didn't bring a tarp. Fortunately, Galen had a tarp in his trunk accidentally. Boy, it'd be a really good idea if we carried a tarp because when it rains, it just fills in the hole of water and whatever, right? And what were some of the speed bumps we get? What were some of the things that were issues we had, boy, this software problem, we thought it was trivial and turn out it was really hard. Or, they updated the software in the middle and it set us back. Or Mary Sue, the database administrator, got COVID, was out for two weeks at a real crucial time, and it screwed us up. So ask the team to identify just two or three interesting events, things that stuck out for them. Because everybody, it's the Rashomon effect, right? Everybody's going to have a different perspective on what happened. So then what I'll do is, when it comes time for the, I'll paper a wall and build a crude timeline with significant milestones and events that I could glean from the status report just to keep folks oriented. People show up, I explain the process, give them all some post-its and say, Hey, so what were some of the events? When Susan got married, that was great fun. We had a good time at the wedding. Great. The nice thing about those is they orient people to time. There's the stuff that happened before Susan got married, the stuff that happened after. They may not remember it was May. Susan will know when she got married, but yeah, the things were going great, and boy, right after Susan got married, this thing over here happened. So we can put those events up here. It's a warm up. There's no significance assigned to that other than they're moments in time. Then we could talk about, so what's some of the good stuff that happened? Oh, we hired Jim, and he was the new kid, and he had way more skills than we thought. That was really helpful. Cool. Let's put that up there. What were some of the speed bumps? What were some of the things that happened that slowed us down? The rainstorm, the power outage, whatever. And what we're doing is we're building a timeline so that people can see visually how the project evolved. And what happens is that starts spurring more things that people talk about. I can think of a specific example where we were building the timeline and somewhere near the middle, the project manager said, yeah, and we really bogged down here. We were starting to get a lot of questions from a couple of key stakeholders and it really soaked off a bunch of time. And we talked about what else was going on and what they realized and they hadn't really put two and two together, is they'd swapped in a new stakeholder in the middle and hadn't taken the time to orient the stakeholder. That's not a fault. That's easy. You're dodging bullets are running a hundred miles an hour. It's like you get the memo. Oh yeah, Mary Sue's out and Jim is in. It's easy to, yeah, okay, yeah, that's great. I need to call them and chat with them. But it's no, actually you may want to go spend an hour with them saying, here's what we're trying to accomplish. Here's your role. Here's my role. Here's where we are. Here's the challenges. And it's easy to skip that. It also can be very dangerous. And so we started building a timeline. They realized that stakeholders swap caused ripples and turbulence for a couple of months because the team kept getting soaked off and was getting asked all these different questions. And they hadn't realized it. It's one of those things that as you see the timeline and someone says, oh, that's when we changed the sponsor. It was like, oh, now I see. So the idea is when we build the timeline, people get oriented to some of the significant events. And then we're going to do risk management in hindsight. Easiest risk management session ever. We can go to a flip chart or a whiteboard and say, okay, so here's some significant things that happened. Good news, bad news, whatever. And then we can ask three questions. What could we have done to detect this sooner? To detect the opportunity or to detect the risk or to check the problem sooner? Is there anything we could have done? Sometimes there isn't. Is there anything we could have done to increase the likelihood of the good stuff or to decrease the likelihood of the bad stuff, right? Sometimes there isn't. We can have that conversation. We can take notes. And is there anything we could do to reduce the violence of the bad things or to reduce the benefit of the good things, right? This is where we say, oh, it was great that we bought those 14 laptops. In the future, we should get the right to return them, right? They were completely clear. We got no inventory at the end. We don't have to sing and dance about why'd we spend that extra two grand. So that really, the results of that, the team contributes, it's not speculative, right? It's not imagining the future. It's saying, okay, in the past, it'd be nice if we could send a message in a bottle to ourselves and do these things differently. And then that's the rough draft of the final report. Here's what the team set out to do. Here's what they accomplished. Here's how much it costs. Here's how long it took. Here's some things they did along the way that were commendable. And here's some things they did along the way that were unfortunate or the things that happened that caused, a schedule scope or resource stress. And here's what you might consider doing in the future to alleviate or reduce it. So the report for retrospective, you don't want to create a phone book, no one's going to read it. You want it to be a page or two or, quick summary slide saying, Hey, here's some a couple of good takeaways we could do process improvement.
Galen Low:Talk to me about depth because we're talking about getting all of the sort of milestones and things that happens throughout the project. Everyone's getting oriented around the timeline. Some folks might come in with 10, 15, 20 things and are you discussing them in depth as people are putting them up on the board? Or is it mostly identification and circling back and asking those three questions later on?
Payson Hall:It ends up being a judgment call. I want to give people enough time to have an aha moment. And I don't want to go down a rabbit hole that makes my meeting run super long or cuts the conversation short. So, it's like, how much root cause analysis do we do? Let's have a conversation if it's being fruitful for three minutes or five minutes, great. At the end of three or four minutes, or it stops being productive, say, okay, let's take this and we can do that little bit of research offline with the people that are most closely associated with this. We can feed that back.
Galen Low:And that's why you need that facilitation skill. We talk about it a lot in the community about like the, does a project manager or does anyone on a project team need to be good at facilitation? And, I'll keep coming back to the idea that it helps because even if you're not facilitating, you understand that not all of the world's problems are going to get solved in that session. Some of it will go offline, but you want to be able to talk about the big things and look at areas of improvement or how you can capitalize on things that went well or clever ideas that came about during the project itself.
Payson Hall:Absolutely. That's one of the benefits of whether it's someone from outside the organization or just someone from outside the project. If they're a good facilitator, I mean, I'm a recovering engineer. If you present a really good problem, I'm tempted as much as anybody to do a deep dive and talk about it. I've seen this before, and here's how it was solved. I got this great book at home, and that's not helpful. You want someone saying, okay, are we making constructive progress? And how does that fit into the larger picture of what we're trying to accomplish? And so, yeah.
Galen Low:I like the sort of lens of looking at it as yeah, this is a rough draft of a report that you might share with others who are going to take on other projects or, future you, who's going to revisit this and probably not want to, dive down some big Ishikawa diagram to look at the root cause. And that's the only problem we solved in the entire session. We had one learning from our retrospective, but at least we know the root cause. Yeah. I hear what you mean in terms of yeah, making a judgment call on the depths. I really liked that. The other thing you said is and I don't know if a lot of folks look at it this way, but I think it's important. You said this is risk management in hindsight. And a lot of folks, they're looking at risk management as a future thing, right? How can we help good things happen in the future and stop bad things from happening in the future? And I don't know if everyone always connects the retrospective to risk management, but I think what you've been saying about iterative retrospectives or, at a certain stage gate in your project or even after. But it's really, it's that sort of message in a bottle to, you wish you could send it past you, but instead you're sending to future others about how to avoid this thing that happened that was bad or how to take advantage of something good that can happen.
Payson Hall:Absolutely. Well, and the key idea here is it needs to be about what worked and what didn't, not who worked and who didn't. That's the whole job of the facilitator. This is not about blame. And let me tell you, because, you it's easy in a podcast to say, let me tell you about all the time how smart I am and how I've done all this clever stuff. I was working with a team and the team had just finished one project. I was helping them start the next project. And they were having, going to have a retrospective about the project they just finished and I did something really stupid. I invited myself, I said, it is okay if I come. And I told you, no tourists, you only want people that were involved. And I'm talking with this team, I'm chatting with the team and one guy on the team, I said, so yeah, you're in the retrospective says no is I'm not going. They're just gonna blame me for everything that went sideways. That really perk up my hearing. I was, okay, well, I'm curious about this. So I went into the retrospective and the person facilitating the retrospective, because I'm just an observer, was terrible. Okay. They were terrible facilitator. They wanted to know who was responsible for this, which is not the question we're asking in a retrospective. And I was amazed by the team. So they talked about, well, blah, blah, blah. And then this happened. And then this piece wasn't ready and the facilities, well, who was supposed to build that? And the team for, I give the team immense credit, just said, it's not really important who did it. Here's what happened. And they refused to throw the guy under the bus. I discovered later, the guy was an alcoholic. He had a real problem. And he was really responsible for all this, but the team was aware that's not something to share with this facilitator. And it's not relevant, right? There's a personal issue someone has to take care of. Let's talk about what we learned on the project. You never want to rely on a type of, amazed I have the utmost respect for that team. You never want to rely on a bunch of tired, grumpy people to have that much discipline and empathy to not crucify the guy that wasn't there. So it was an amazing team. And so, yeah, you don't want any tourists and you want the facilitator to know what the heck they're doing that this is about. Let's talk about proving the process, not who do we blame? And again, I think the rule guiding rule is just assume everybody was doing the best they could with the information available. They may not have had the right skills. That's the project manager's fault, right? Unless you gave me a resume that said, you're an expert at AI, I gave you a simple AI task and you were overwhelmed. Then that's the project manager's fault. It was a wrong a skills mismatch. And we don't have to say it's the project manager's fault. We say, you know what? We didn't have the skills that we thought we did in this area. It's not, Simon's not tall enough for this ride. It's, we didn't have the skills that we thought in this area. We need to bolster those skills going forward if we're going to do a similar project.
Galen Low:So much of this is like in the, it's a woven into the fabric of how you've set it up. A lot of folks will come to me and they'll ask me, okay, well, how do I create psychological safety in a session? And part of the answer is, well, before the session sit down, talk to the people who might have reservations, build trust, build rapport, explain the process, give the do the housekeeping to say, you know what, in this session that we're about to dive into, we're not going to ask who, we're going to ask what. And getting everybody on board with that idea and then just reinforcing it. I mean, if you were, let's say you were the facilitator in that session, and the person who was facilitating was just an attendee going Oh, that was probably Brad's fault. How do you reorient when people start the blame game and start pointing figures and it becomes a witch hunt?
Payson Hall:The first time you yellow card them, right? They say, well, no, this is not really about who did what. We're going to assume everyone was doing the best they could. What we want to talk about is what happened, what are the consequences? Second time, depending on context, I might yellow card them again. I might say, you know what? Let's take a break. Take the person aside and have a word with. It's no, this is not about who broke. We can have that conversation. You can have a broad conversation. The project manager, you and I can talk about it later. But this meeting is about what did we learn? What worked, what didn't? Not who worked and who didn't. Because it's not constructive. That we can't fix the past. All we can do is learn from it. Right?
Galen Low:I love that. I guess the other side of it is, we're talking about framing. We're talking about, okay, well, maybe we didn't have the right skills. And it goes into that report. And then someone on the exec team reads that and reads between the lines. And they're like, yeah, that was probably Brett's fault. Is there anything that you can do to avoid that interpretation?
Payson Hall:If Brett's not tall enough for the ride, that's a personal issue we need to take care of. That's not something I want to end up in the report. I'll defer to the project manager to handle that. The kinds of things that end up in the report, a lot of times, like we said, if the execs are, this is not faulting execs have priorities that might be different than the project manager. So, if a higher priority project starts to boil over on the stove, and they pull one of your superstars to solve that problem, that's their prerogative. The idea behind retrospective is to say, Hey, there was a consequence to this project. Without blaming the execs would say, Hey, you know what, we had a roster for the project and for a variety of circumstances, people got redeployed in the middle, or if someone got COVID or someone eloped, whatever, and we had resource issues on the project. In the future, what we need to do is make sure we have better agreements and maybe think about, this is a personal bias. If your schedule says, but no one can get sick, everyone has to show up for a 40 hour week. You're already smoking dope. That project's not going to happen, right? Because someone's going to get sick. Their kid's going to fall off a bicycle. The parent's going to get ill. They, they got to go to graduation or something. So if your resources are overcommitted, the project schedule becomes very brittle. And that can be a finding. We were overly aggressive in how we allocated resource. We really needed more resources to do this. We maybe we didn't know that here. We knew that here. We should have said something when we, we discovered that here, we probably should have filed a change order there to make sure everybody knew that we're going to need more resources, more money, or more people than we thought, or more time.
Galen Low:I like that sort of criteria of, you know, what you want in the report. And of course, deferring to the project team involved, but there is a difference between project issues that we can learn from. And for example, personnel issues that just have no business being in that report because it is something that is not specific to how projects are run. It has more to do with staffing and, vetting, I guess. So I like that sort of lens up.
Payson Hall:Well, the report needs to be diplomatic too. As a consultant, I get paid to tell people if the baby's ugly, but there's ways to deliver that news that don't inflame the situation. So you want to be diplomatic in the language that you use to describe lessons learned. Basically, what you're trying to do is make it as positive as you can. Hey, we did this in the future. Here's recommendations for doing it differently. Not, Oh my God, we really screwed this up. But that's not helpful. People are doing the best they could with the information available at the time.
Galen Low:I like that. One of the things that, you know we've gone through this process of, setting up, preparing for retrospective, reviewing the documentation, getting those milestones out on a timeline, discussing them. Talking about ways that we can either take advantage of good things that happened or avoid bad things that happened in the future. And then it goes into this report, but I think one of the things that I hear the most from folks is they're like, yeah, we're not going to do a retrospective because what are we going to do? We're going to produce a report, no one's going to look at it or they're going to misinterpret it. And then it's going to sit on a shelf somewhere. And we're going to watch as other project teams, including ourselves, keep hitting the same issues over and over again. But like, how can you, I guess, as a facilitator, but also how can organizations overall make sure that these recommendations actually lead to change instead of just gathering dust?
Payson Hall:Well, let's unpack what you said. One of the big problems is if I produce a 30 page report with findings and recommendations, no one is going to read it. So the idea is to be brief. We want to prioritize, right? So if you're the project manager, you say, here's six things I think we could do better. I'm going to say, pick two or three of them and I'm going to try to polish those up and get that message through to the exec. Okay, so we got to make sure we don't overwhelm people with action items. That's a hard balance, particularly for young organizations that are still learning their way. But we want to pick and choose. We'll get those others next time, but you'll still know them, right? And the team will still know it. So there's the formal, Hey, TMO or executive team, here's two or three findings we have. We think this is something our organization could do better. You want to make sure you prioritize that list so that they're not overwhelmed, that they don't just be, Oh, fine, that's good. Set it aside. The more comprehensive findings of things, Hey, here's what you did that was clever. Here's stuff you could do differently. Everybody in the retrospective carries that information off as a suggestion to their next project. So there's the formal and then there's the informal. And the informal is really, I think, the way to turn the battleship. The extreme, there's this mythology that says you do retrospectives and you write this up and it goes into a big database for everybody in the organization. I have never seen that pulled off ever in any organization I've ever been in. It's an idea and Oh, we're going to send it to the PMO. They're going to have this list of lessons learned. It's like never seen it done because it's too overwhelming. It's too context specific. So the best thing is, I think, the two or three recommendations to make the executive team aware of. You talk about rescheduling rooms. One of the things that is an often lesson learned for the non trivial projects is they should have a dedicated war room. And if you stop and think about having a place to be able to conduct meetings on the fly, particularly, I don't know, now with COVID, a lot of vacant space. But you think of most offices being really packed and you got to hunt and search and destroy for, to find a training room or a meeting room. Small dedicated war room can be a huge savings for a team. Think about how much times people have been wandering around or trying to figure out where is the meeting and real quickly you're saving person weeks just having a dedicated room. So that's the kind of thing that can bubble to the top of a list and once, execs may not hear that the first time or the second time, the third or fourth time they see it, hey, you know what, we keep saying this and we're not doing it. Maybe they can get a clue.
Galen Low:And I'd like the separation as well. And I mean, it might be logical for some, but yes, there is a strategic recommendations that strategic communications up. And to your point, don't make it a lengthy novel, pick the high priority things to address. And then there's also this informal level of learning that happens regardless. And I don't know if a lot of people appreciate that as much. And I think you earlier described it as everyone who participated in that retro is going to take a seed with them and carry it to their next project. And I think that is massive and probably unmeasurable, but absolutely something that needs to be talked about as part of the process. Yes, we're going to have a report, but guess what? All the things we talked about, we're learning just as humans in our brains without having to write it down on a piece of paper necessarily. And we're adding to our shared experience, even that timeline where people are going to remember different things. That Rashomon effect, you said, like they're learning from that too. And they're taking that experience and bringing it with them, whether they've remembered it at the time or not, they actually got the chance to discuss it. And find out what the impact was and how they can either do good with it or avoid bad with it in the future. I think it's super cool.
Payson Hall:Well, to come full circle with our conversation where we mentioned risk management, one of the things that occurs in most planning processes is you get the core team in a room and say, Hey, let's talk about a prospective, right? Let's look forward. What are some of the risks that you see? And if in the last two months you just came from a retrospective, you're prime to think of those occurrences as risks. And so you're ready to have a much more, Hey, our last project, we did this and we realized we shot ourselves in the foot. We wish we'd done it this way. And not only do you have that top of brain, but it's from a real, it's not hypothetical. It's a real world thing that you experience. So you can speak with some authority, Hey, we ran into this, we did this, and it was a great idea. We ran into this, we did this, and it was a horrible idea. Getting those things on the table does two things. First, it improves the risk management on that project. And second, everybody in the room hears your story. So it continues to propagate organically through the organization.
Galen Low:And although I love the word story as well, because it really is that, right? It's like the database is our noggin and what happened in a project is a story that we can share and share knowledge with one another. It's true that. I actually think it was probably a good segue because I was like, okay, well, let's reserve some time at the end to take some questions from the audience. And one of them actually was about the notion of a prespective or what some folks call it, like a pre-mortem. And I think you did answer that in a way that, yes, this is risk management. You can have a pre instead of a post. But what are your thoughts on doing both? How does risk management tie into like iterative retrospectives? If you're running agile and you're doing sprints and you're going to do a retro, are you also then doing a sort of risk management looking ahead, meeting separately, or does it blend in together?
Payson Hall:I think they blend in together. A lot of times when you get into an agile retrospective, what they're saying is, what are the barriers, what are the things you, gee, I wish we'd done differently, or we're doing this and it's not efficient, those become to dues to mitigate those risks in the next sprint. So in a specific context of agile, we're talking about retrospective and risk management, at the same time. We're taking those lessons learned and say, how can we apply this to the next sprint, which I think is really healthy.
Galen Low:For sure. And to have those conversations, yeah, because that relationship, right? And you're going into the future as into the next sprint, there is a future. It's not all just rear view mirror stuff, but it is still the same conversation. I thought I'd also grab this other question. I know we don't have time for too many, but I thought I managed to touch on a lot of them along the way, but this one really stood out. We're talking about this notion of like things that happened during the project. And sometimes it's choices that people made who are on the project team, but sometimes the issues go beyond these decisions that the project team is making, and it might be more environmental and the example given, and I think we we touched on it as well, but you know, if the root cause of an issue was that project team got poached away, some of the key resources were taken to another project, like how do you capture that and report it up so that you can influence change beyond the aspect of that single project where it's it wasn't something that the project team did, something external came in. And then how do you like the surface that in the recommendations in the reports, but also how can you get that to influence change outside of project context?
Payson Hall:So I tend to think in terms of finding. So the finding would be the project experience challenges because team members got redirected to higher priority project. That's a pretty neutral way of saying this. Now it's making an assumption that they're higher priority projects to give benefit of the doubt to the exec team. So that would be the finding. The project experienced resource challenges because some of the resources got redirected to other issues in the middle. What's the recommendation? The recommendation would be to consider the adding, increasing the staff on the project so that you got a little more staff flexibility. The other thing might be to say, make sure to provide as much advance notice to the project manager as possible if you need to redirect their staff so that they have more time to respond and to try to backfill. So if you frame it that way, you're not saying anybody did anything wrong. What you're saying is this happened, it had a consequence. Here's what we could do in the future to try to avoid that.
Galen Low:I like that notion of findings. I'm going to tackle one last question.
The question is:what are the most important topics to focus on in a retro? Because so many folks seem to be interested in so many different aspects like financial or the process or execution or methodology, but like you only have so much time. So with that limited time, what would you recommend focusing on to get the most out of it?
Payson Hall:Some of that's going to be dependent on context. But if I give you from my experience, a lot of it has to do with, is there a clear definition that's shared and approved with the stakeholders, the sponsor and the client? And as that evolves, do we have an effective change management process? So it's a conscious choice. Hey, if you want to add some ornaments to the Christmas tree, it's going to cost more and take longer. It's not the only sin, but it's one of the original sins that cause a lot of cascading problems that masquerade as other stuff. Wow. We really underestimated this. No, you didn't. You did a fine job of estimating it. And then the project grew and you didn't throw a flag on the field saying, Hey, you guys are increasing the amount of work without giving us more resources.
Galen Low:Yeah. I love that. That's like there's a relationship between all of it. And I like what you're saying about what is the common ground with the stakeholders involved and what's going to be the most valuable lens to apply to that conversation. Very cool. I think that's us for our time, but Payson, thank you so much for hanging out today. It's been a real honor having you on the show. Before we let our listeners go, how can folks find out more about what you do at Catalysis Group?
Payson Hall:If you go to catalysisgroup.com, you'll find information about training and consulting. And actually, if you go to the resources, there's some a couple of articles you can download about risk management that talk about how to conduct a risk management session, and you'll find that'll be really useful fodder for a retrospective as well.
Galen Low:Love that. Attests to the fact that Payson is a very entertaining writer. Every article I've read from you, I've laughed out loud, but I've also taken that story with me as a result. So, yeah, for anyone who thought retros and risk management are dry, well, check out Payson's stuff. I'll include the link in the show notes below. For our listeners, I hope you took a lot away from that. And, there were a couple of questions that we might not have gotten to, and we'll surface those in the community and have a gab about it as well.
Payson Hall:And I'm also, but if you want to send email to me with questions, I'm happy to answer any questions people have, they think of right after we hang up.
Galen Low:All right, folks, there you have it. As always, if you'd like to join the conversation with over a thousand like minded project management champions, come and join our collective! Head over to thedigitalprojectmanager.com/membership to learn more. And if you like what you heard today, please subscribe and stay in touch on thedigitalprojectmanager.com. Until next time, thanks for listening.