AA140 - The Backlog Has 500 Bugs!

HELP! Our Backlog has 500 Bugs and we feel like we're never going to dig ourselves out!!!

Remain calm...

On this episode, Enterprise Agility Coach Om Patel and Product Manager Brian Orlando walk through how a company or shop might get to this point, and discuss some strategies to help dig yourself out of this deep, dark hole!

0:00 Topic Intro: 500 Bugs in the Backlog
0:28 Initial Reactions
2:44 Reasons
3:55 Process
5:10 Rogue QA, or the Perception Thereof
7:18 It's Done, But...
8:53 Customers Pay the Bills
10:12 Culture - the 3 (or 4) "P's"
11:28 The Business of Prioritization
13:39 A Strategy for Digging Out
15:59 Involving Customers/Stakeholders
17:41 Another Strategy - Categorization
19:30 Pushing Back on Customer Involvement
21:04 Rank Ordering and Closing Bugs As-Is
24:28 Definition of Done
28:36 Program DoD
29:11 The Restaurant Analogy
33:04 Customer Validation
37:10 Surveying Customers
40:02 Podcast Summary
41:44 Wrap-Up

= = = = = = = = = = = =
Watch it on YouTube

Please Subscribe to our YouTube Channel:
https://www.youtube.com/@arguingagile
= = = = = = = = = = = =

Apple Podcasts:
https://podcasts.apple.com/us/podcast/agile-podcast/id1568557596

Spotify:
https://open.spotify.com/show/362QvYORmtZRKAeTAE57v3

Amazon Music:
https://music.amazon.com/podcasts/ee3506fc-38f2-46d1-a301-79681c55ed82/Agile-Podcast
= = = = = = = = = = = = 

Transcript

0:00

I guess we're on a kick of related QA podcasts. In this podcast we're gonna explore a exclamation that I heard once at a networking event saying, I have 500 bugs in my back. And we all recoiled and, I don't know, made a protective sign of whatever our religion of choice was. Yeah, exactly. 500 bugs in your backlog. And I don't remember the summary of that conversation, but for the purposes of this podcast, we're going to make up a new summary. I like it. Listen, when I, when I first heard this as a concept, theoretically, metaphorically, philosophically I said like who, who hurt you, who, who inflicted this on your culture? Because like, if, if there's 500 bugs in your backlog, like something is wrong with your corporate culture. Or development, culture, whatever. The culture of your organization. Something is wrong with your organization. Why do you have 500 bugs in your backlog? What has gotten out of hand? That was my initial, immediate initial take. Coming from being a BA, QA analyst real early in my career. Being a manager of QA. Hiring a bunch of QA analysts and engineers over time. Yeah, I this was my immediate response. I like your immediate response better than what mine might have been right? I have 500 bucks. What you been doing? instead of Right. So like why do you have 500? Why did you let it get this far? It's like saying look my entire house is burning down and now the chimney is on fire. So what have you been doing this time? Right? That, that would have been my reaction. But anyway, let's give people the benefit of the doubt. Somebody just said, Look, I just realized we have 500 bugs in my backlog. Now what? So I think the context is important here. What is your backlog? What are you doing with your team? Is your team a firefighting team that is simply just taking these bugs from somebody? Somebody's and just whittling those down and if you have 500 hopefully they're prioritized in some way So that's one kind of easy out right if you like Assuming that's not the case and you're just another Development team that just happens to have a deep backlog because you have 500 bugs in your backlog This person who is vocalizing. I have 500 bugs. Who are they? What persona? Is that a product person? Is that it? It really doesn't matter in the long run. It could be anybody on the team that says, yikes, 500 is a big number. What now? Well, to me, the product manager, I don't really, it doesn't really matter to me who it is. Vocalizing that they have, I also assuming it's 500 open bugs. Sure. Cause there's 500 bugs and you select all and close them. Then congratulations. You have no bugs anymore. Yeah. You close them all but , to the QA manager inside of me, it worries me that your product is built on a mountain of constantly shifting sand and that everything's going to collapse , at any moment. So like, I think that there are, cultural problems. That's, that's my initial, suspected area to start poking at, which is either your development team is constantly under pressure to deliver features. And nobody cares about the technical infrastructure, architectural runway, whatever term you use for it. Nobody cares that your application has to get rebooted. Nobody cares that they, that your people do care. Just not the, just not the right people care that your developers are up in the middle of the night every other day. You know, rebooting your servers or dealing with memory issues or trying to calm the database, talk the database off. Like, listen, your database is at the club drunk and it needs to get convinced not to drive home and get an Uber. Yeah, exactly. Yeah. You're so right on that. Right. I mean, look, there's. Different scenarios here, right? One is if people have watched our other podcast about, well I don't have a QA on my team. That, that podcast, yeah. Maybe you really don't have a QA on your team and you have 500 bugs and your developers are just insular and they're thinking that all they do is develop. Maybe that's a scenario. I don't know. Well, that's, that's, it's easy to fix that. That's, but there is that. Yeah, it, well, you watch, watch that podcast.'cause in that podcast we were talking about, Basically the opposite of what we're talking about in this podcast. In that podcast, we were talking about a very advanced development team who understood quality, who had testing skill set, who, they were cross functional, and they were in control of all the variables that it takes to make and maintain great software. In this, I'm saying something's wrong with your culture. I guess when I'm saying culture, that might be, maybe that's too heady of a word. Something's wrong with your process. Let me, let me boil it down to your process. Yeah. Okay, maybe there's a lot of handoffs. Maybe there's a lot of people fighting for ownership of little pieces of the application. Maybe things are split up too much. You know, there's a million things of maybe, right? Here's another one. Maybe you have a rogue QA person Right. That's just writing bugs like crazy. And they don't really understand. They've not been brought in and aligned. With the product and or development teams to say, Hey, this is how we need to best utilize your skill on the team. You need to help us understand and not write our stuff up after the fact. We need you at the beginning to help us get things done in the right way. Yeah, I think, let's unpack a little bit of that, right? So, yeah, the previous podcast was about a team that owned quality, right? This podcast is about really we have a situation at hand and how did it arise and what do you do about it? So, I want to go in reverse order now. Rogue QA. I think that kind of does a little bit of a disservice. Service to the person because they are a QA person and their job is to write up bugs and that's what they're doing somewhat of a maybe a CIA type of situation. If they didn't, then they would get in trouble type of thing. But you know, if a field on a screen is misaligned by two pixels. Technically, that's a bug. Technically, you could argue that, but is, is the whole thing still usable, I see where you're going. But if you're drawing a circle around your QA department and then you're saying this, this QA department is measured and incented Based on the number of bugs they find and the number of bugs that slip out to customers and the number of whatever, of course, they're going to be kicking and streaming to, to not let a single piece of functionality out that could potentially come back on them because that's the way the numbers and the, the, the metrics go and you know what you know, as we talked about in the last podcast that I, I had to really hold myself back from digging into is the culture where, you The Sprint is two weeks, ten business days. Development works for nine and a half days. And on the last day of the Sprint, they put it into the QA status. Send it over to the QA column, team, whatever. And then, and then, and then And then the message is communicated from development to product. Not QA, not, not team, right? Cause we're not teams at this point. There's circles around all of our, all of our departments. The message is communicated is development finished in the sprint. And I don't like these QA people just can't keep up there. They're, they're being a bottleneck. everyone listening to this, hopefully everyone listening to this understands that what I just did, that's a game we're playing games. Of course, we're not being a team and maybe they do, maybe they understand it. But also when you're incented on things you might understand that it's a game, but also you gotta play the game Or else you gotta keep that resume updating Yeah, right. You gotta play the game if you want the incentives because you are incented that way Absolutely, but when you get up to like when you get up to the leadership's level and dare I say products level Those are the right people to push through all this stuff, to be like, look, you guys are a team. Like nobody cares that it made through one phase or another phase or whatever. I care that it's in production. I had a CEO one time talking about definition of done where I'm skipping ahead here, but that's fine. Talking about definition done. He's this was rampant in that culture at a place I worked one time of people saying it's done. It's done, but now it's got to go through QA. It's done, but it's got to go through documentation. It's done, but it's got to go through UAT. It's done, but people do that all the time. Oh, it's totally done. It's done. Everything's great. I'm so great. And the CEO one time, I don't remember where he was, but he stamped his little fist down on the table and he's like, I don't want anyone to ever say something is done ever again. Until it's in production and a customer and a customer can, and it's in the customer's hand and the customer can use it. And the customer is aware that they have the ability to use, maybe you can't control customers, right? But it's in the cost. It's the customer has the ability. To go use it and have it work end to end the way that we promised that it would work. Once he laid down that mandate that the culture started to get better. Quite honestly, the culture started to get better. We had no product people in that company at the time. So this is something a product person would do. Is say, hey, I don't care about all these games. We shouldn't be incented by these games. Right. We should be incented by value actually delivered into customer hands. Sorry, I don't know why I did it, Kevin Kirk. Value delivered into customer hands. It's gotta be in their hands at the end of the day. Absolutely agree with that. So, companies that just simply prize that done, right? They're looking at charts or whatever some kind of dashboard maybe. And they're just saying, well, how many done this sprint? Then the development teams... Plural. They're all going to play to that and they're going to say, we're done. We're done. We're done. We're done. Right? The bottom line is just that, like you said, we play games, right? But all of these things aside, who really pays our bills? It's the customer! So if the customer doesn't see value, they don't have to recognize value. Give the product to the customer and let them feel it touch it You know poke it whatever they do with it if they love it great if they don't great Some people are gonna go. What do you mean? Okay, fine. Look if they love it and they tell you they love it then fine if they tell you they hate it I say fine, here's why. You did what you were supposed to do. You got it in their hands. And now learn why they don't like it. Right? As opposed to saying, well we're done. Right? We're done. Customer doesn't know yet, but we're done. Really? Are you? The old, the old arguing agile pushback on what you're saying is like, I understand what you're saying and everything, but like, it was like, what are you going to do? Change, change, incentive. You're going to write a new incentive plan and go to finance and go to the CFO and get it signed off. Like, get out of here, kid. You're just some, you, you're, you're just some pointy head that deals with development teams. Get out of here with all your new fangled scrum mastery, product management. We know how to do software development. Yeah, yeah, you kind of do at that point, I'm just going to say, you kind of do, but your customers are leaving you by the droves. Whenever this topic, something like this comes up, I always tell people you better fix the money. You better fix the incentives or you're not fixing these problems. Again, which again goes back to the my point. Number one is a culture. It's like it rooted deep in the culture look you mentioned the c word So I just want to kind of just quickly say What is culture? You know, can you touch and feel culture? Can you change it? If you can't touch and feel it, how do you change it? Gently massage it. Can you gently massage it? Yeah, exactly. No, no, you can't. Right. And and so culture really is a product of largely three variables. Culture is a product of people, policies and processes. Now I will lay on a gentle fourth on there, which is politics. You can definitely impact three out of the four and the fourth one will take care of itself. One hopes, right? If you can take care of the three that you can actually impact people, the the, the processes and so on. You can definitely impact those things. So unless you're taking that into account, you're simply just playing games that, as we were saying earlier, that's all you're doing, right? And customers will find that out. And when you find out that they've realized all this, it's too late. Yeah. Well, usually it'll be too late by the time that you turn the ship around. You know, it's going to be like, that's the only, the only reason that people try to start correcting culture is because you have very, very, very serious. You have issues that are already there. Yeah, exactly. You started going down the issue real early of prioritization. I think you, you were the first one to mention prioritization. The other shop that I saw this type of thing have a problem with this type of thing is they had no, they had, they had I want to stop at the edge of saying poor. They, they had, they, they needed help with their product management facilities. They had suboptimal product management. They, they did, they did practices specifically with regard to prioritization. Mm-hmm. They needed a lot of help with prioritization and also like that, like if you think about like running a business, if you have this problem where You have 500 bugs it means the bugs are not getting worked on and I think of the same the same way I think of Where you have organizations that are on like super old technology like windows xp or when it's like I remember back in the day when people were on windows 2000 I was talking about Windows 2000 because I was trying to illustrate. Like many years later, even when superior operating systems, I don't know, were there a superior operating systems in the 2000? I don't know. I don't think so. When superior operating systems came along, those business systems were still run. On that old windows 2000 platform, because upgrading was deemed as like, well, there's no money in just retesting the software on a later O S you know, there's, there's no money in that. So I'm not even going to bother not until we wanted to do new features that required new operating systems. Where we forced to go through the upgrade, but again, this is an effect of product if you don't have product No one is looking at the future. No one is making these decisions. Maybe you have a really good tech lead CTO whatever right who's kind of forcing you to say hey this new technology is out and we can't run windows 2000 machines, right? On whatever, trying to build data models. This is where Microsoft said, look, if you're not going to upgrade, right. We're just stopping support. Right. Yeah. Right. Right. So a CTO would say, wow, we need support. Right. Yeah. So there's this side of it. Windows used to have a 10 year support life. Back then. Yes. Yeah. Right now, now AWS is like, Oh, Oh, it's, it's. It's, it's 7 p. m. It's Tuesday. Oh, we're, we don't support the previous version anymore. Exactly. They, they, they bring you along. Okay, the, the cloud like forces you along. That's good and bad also, but we're not gonna go there now. But the, the, the point of me bringing this up is, if you're in a situation where you have 500 bugs in your backlog and nobody's managing it and you're a product manager and you come in or you're a QA manager and you come in it's going to take a bit of rolling your sleeves up and getting into it, but you can get out of this situation here. Here's my, here's a previous Matt previous QA manager's suggestion for getting out of this very real sticky situation that you might find yourself in of, I have 500 bugs in my backlog. Okay. First of all you need a strategy to maintain your current architecture, no matter how behind you are to maintain your current architecture and move forward. To, to, to continually pave that architectural runway so you can launch bigger and bigger planes as technology grows and expands and et cetera, et cetera. Okay. Right. Here's what I would do immediately. Immediately, I would start going to product and saying whatever our capacity is for our team, all of our teams across the entire program. If you're a larger program, right? Right. I demand the 20 percent of your capacity. To be put aside for whatever the team, by proxy of a CTO development leads, whatever it's up to the team to, to, to decide what to bring. But basically the, the CTO and or their staff of tech leads. Or development matter. If you're a small company, you have development manager, right? Whatever technology solutions that need to be upgraded, maintained, implemented that 20 percent services that component. So they need to build things into the roadmap and they need to maintain those things. I would say, look, you have some. A bit of product management function. We need to bring you in and you need to be responsible for maintaining and curating and refining first of all, but that 20 percent is going to be dedicated to whatever you say, let them work on and figure out how to do it. So the remaining 80 is now owned by product management to do the normal feature development. Cause normally when I see this, it's because the business. Is undisciplined and just wants 110 percent of the velocity dedicated to new feature development. Bring me new features, bring me new clients, spend your nights and weekends doing upgrade work or whatever. I don't want to hear about it or, or worse, send all that stuff over to some kind of architecture platform infrastructure team. And all they do is this grunt work that nobody sees in the middle of the night and think that you can ship off maintaining our infrastructure to another team. You can't do that. You cannot do that. So sorry, you were, I think you were about to push back on No, no, I wasn't going to push back. I was going to say something here which is, If you have a situation where you have 500 bugs, clearly it got that way for a reason. So, do some RCA on that and figure out why. Now, now, now that you've come up with your RCA and, and, and it's still, let's just assume the worst case scenario. You just say, well, because, well, it doesn't matter. You have to deal with it now. You could. Like you said, I'll allocate capacity, 20 percent whatever, right? And just say, go whittle that down. That 500, the backlog of 500, go whittle that down based on some kind of prioritization, based on severity, etc, etc. Whose claims are loudest, you name it, right? Whatever scale. But, here's what I want to add to this. If you want to be effective at this, not efficient, effective, right? Why not involve your customer in the prioritization exercise? Get them in the room and say, We got 500 things that have been reported, some by us internally, some by you, which of these are your top pick a number 20, 50, whatever it is, right? Top 10 percent would be 50. Okay. So let's look at these 50 and then focus your effort on those, not the other 450, because they're not that important. Those that matter. I got sidetracked because what I was going to say But the first thing I would try to do is I would get a panel together. Yes. If I'm the QA manager, like I just got hired into Brian and Om's software development company. Brian and Om are harsh taskmasters and only care about features that they can sell. They don't care about fixing things because they plan to sell the company in two and a half years. That's right. And they, they don't care if there's a thousand bugs or five hundred or whatever the number is. As long as they cash out and bro down at the end of it. So, what I would do, as a new QA manager, product as well, I would do the same thing as product, I come in, I sit down with my panel of people, maybe a QA person, product expert, customer support, product manager, whatever basically a nice cross functional team, people that know the product, people that know the customers. Okay. And or maybe I would invite a customer to that panel. I'm not sure yet. I would advocate that if you can do it. I'm not sure yet. And what I would try to do is I would go through the 500 items. I assign some homework. I go through the 500 items. I try to break them up into categories. That's right. I try to break them up into categories because I don't know quite what those categories are because they'd be Kind of dependent on your business, right? You know, maybe, maybe like a production things or user facing things or back end thing. I don't know. I don't know what those categories would be. So I'd ask the team, Hey, 500 items, try to break them down, try to break them down into categories, try to group them, and then give me a recommendation of which category we should attack first. Okay. And this is assuming that I also don't have. Basic, like if I have in other places that I've worked the QA team and the development team together, along with product three, you got three teams, they will assign a severity, priority, criticality, frequency, the ability to replicate, they'll assign those type of things that, so it'd be nice to have a matrix. Of those numbers for each bug, but sometimes that's not possible because it requires you to actually take time to replicate the bug, assuming you can and you have 500 to do it and you have 500. So maybe you're not willing to spend that amount of BTUs to actually get this data. So, so that's fine. That's fine. You don't have to do that, but at a minimum, let's group them and let's set, let's set in what we think is a severity. That maybe we just aggregate all these numbers together into just one number of things. How severe do you think this bug is? Yep. Listen, I, I, I agree. I wanna stop you a little bit here for a second. Grouping and severity is good. I agree with that. However, here, here's what I want to like gently lean against mm-hmm.. So not really pushing back, but maybe a little we assign the severity. Who's we? Right? So you mentioned all these people on the panel. Sure. If the customer is optional there, you've missed the mark because we could say this is severe because we can fix it quickly, right? But a customer is the only person, the only persona who can decide how, what the impact is of this to them. So bring them in and say, how what's the severity level for you on this? If you do that and then you move on to the other things, I think you have a much better chance of satisfying that customer for whom it matters. Just saying. I haven't seen this very often I have to admit. I've seen it one time in a small very, very small, like a 50 person entrepreneurial startup that really engaged the customer throughout, not at the beginning, at the end, throughout, right? So every sprint, the customer was in there looking at everything. In that scenario we never built up a huge backlog of bugs because customers will tell you this is burning, burning, itching, irritable and barely feel it. Those were the four categories and burning and itching. We focused on in that order, right? Burning got to fix it. Itching will fix it next sprint or the sprint after everything else is. Okay. We'll just talk about it later because if it's no longer an issue, we'll just drop it off the list. Or if it becomes burning fine, we'll trade it that way. So I just wanted to add that in there. We'll talk through the bugs and we'll come up with an order Basically, we're doing all this so we can come up with the order of that's exactly it should be rank ordering things And maybe you just maybe things that you know, whatever if you do like priority, let's pretend for a second where you have priority one, two, and three, right? Well, one is like the most burning stuff, and three is like, ah, we haven't really seen it. It got reported one time, we never saw it again. Like, maybe you just cut the priority three stuff. And just close immediately close like 150 items sure. Okay. You're like, Hey, if we see him again, we'll, we'll note them. We'll reopen them. And we'll we'll, we'll have new evidence or whatever. They're really old. Like you know when I was early in my QA career, I, I hated to close anything. He did that. Yes. You're saying that, well, we only saw it once. There's a good chance it's a real item, but it was only so I was only seen once and and also like the Differentiation between was it seen by a customer was it seen? Oh, that's huge and they would say well It was only seen internally and we don't can't figure out how to replicate it again Just close it and that would bug me a lot Here I am in retrospect like almost 20 years later and and I completely understand Yeah, we only saw it once and and we did not do Any diligence at the time that we saw it once right to figure out Why it happened exactly what happened. We didn't stop and say hey, I have a problem I don't know what I just did because someone helped me and put some brainpower on it there, you know spend 45 minutes together, sitting together to figure it out. And now we don't know how we did. Here, just close it. So, a quick anecdote here for like five seconds. I, I worked in a, in a client, like a, a, a two tier, three tier, maybe, three tier architecture environment for a few years, some time ago. So, you had the you, you had the database, and you had the middleware, and you had the, the front end, the GUI layer. And, often, often, things will be reported that could not be replicated, right? And then you ask the person reporting. What happened after that? And they say, well, I just rebooted my machine. And then what happened? I tried it again. It worked. Those are classic for, we don't really know, but it's not worth the cycles to spend on figuring that out. So we just burn it. We just say, well, just archive it. Now, if you're really paranoid, don't delete them. You can archive them. That's fine. Right? And if they happen again, you can meter against that and say, well, it happened. One more time two years later, whatever it is, but if you do this an example of 500 and then you mentioned 150 could be burned. So that's 350. You've already made a reduction and you just rinse and repeat on the priority and severity plus frequency. So there's a little bit of that too, right? How often does it happen? In my scenario, 300 people. If it's, if it's a real issue. And it's a repeatability issue. Like people are going to use the same thing over and over and over again. People are calling in and taking orders. It's going to happen again. So I didn't want to delete them. So I put them away. Right. And I can tell you one out of 10, maybe one out of 50. I don't know. I'm guessing here because it was a while ago, would recur. And when they recur, you say, Oh, it happened six months ago or a month ago or whenever. So then I go back to my notes that were made in the system and say, when it happened, what was the agent doing? The person answering the phone. You know, what product were they selling? Oh, it's the same product as this one. Maybe the issue is how that product is configured because no other product has the same. So you're trying to like home in on the the RCA analysis is basically what's happening here. But on a need basis. So if it doesn't happen, you don't have to go in cycles on that. But you know, like to pivot us away from this topic I think if you're in this situation where you have 500 bugs in your backlog, either. You don't understand what the definition of done is or your definition of done is inadequate or you don't have a product person who can triage all these things. I was trying to stick with the, like the, the triaging kind of goes with prioritization. Yeah, it does. Probably, probably should have included it with our discussion of prioritization. But what I'm saying is like what I think of a normal definition of done Like basic stuff like it Sure. It has to have unit tests, it has to it has to have automation baked in. It has to go through passive pressure and all of that. Yeah, yeah, yeah. Security and so forth. Yeah, yeah, yeah. The ba the, the basics of an a definite, like we have to peer on things. It needs X number of code reviews, when I think of, of a culture where this has gotten so out of hand where you have hundreds of them. Yeah. I expect that you just don't, you, all the development standards we were talking about before, internal development standards, coding standards, maybe on the previous podcast where you talked about. It was on the previous one, yeah. Yeah, where we talked about like, well, what are your standards? Where you sit together and you write negative tests first and then your code emerges through the negative test. You're not doing any of that when you have 500. Yeah, you're not doing any of that. And also, it's very difficult to have a definition done when some of your teams Are nearshore onshore and some of your teams are offshore and they don't necessarily communicate with each other, but they should, but the definition done should be flat across all of the teams, which is where I think a lot of people get in trouble because they don't realize, well, my team has a definition of done and it's different from that team's definition of done and it's different that that's what they don't realize the definition of done should be flat so that when your team says it's done, that means the automation is done. The testing is done. Okay. It went to every environment. It, you know what I mean? We, we peered on everything. Our QA people were involved from day one. Our product people are following a million other things that potentially could be in the DoD. You don't even have a definition. Well, yeah, I agree. First of all if you have multiple teams, which is the scenario you're talking about, and if you have a program, the scenario I've been in for a while now. The teams have typically they believe they have a definition of done because every scrum master says, I know what I need to do and their checklist driven. So yeah, we need a definition of done and they get people together and do a little miro mural exercise and they come up with it. And that's when I come in. Somebody yours is there. Yours is there. What about our? Because you can't deliver anything by a single team. It's got to be a cohesive effort. Sure. So then a program level definition of done is created. So I think where you are going with that is it's okay if the team has a nuanced situation where they have some small things that differ from other teams that contribute to the program level definition of done. But I want to go back for a second here. When the customer sees something, it meets the program level definition of done. Meaning everybody's done their best. Everybody's met everything they need to do, including people like DevOps. All these people, if you have those, if you don't have them on your team, you have a separate team for DevOps, for example. All of those people have a DoD that combines up and meets up the definition of done for the program. The team level is simply another ingredient in the recipe. It isn't a recipe by itself, right? So where the teams falter is when they say, well, we've met our DoD. And we're done. You're not done. Done. Right? I hate to use that term too, because it's a cliche these days. Done. Done. But yeah, so I always, I always coach the teams to say, well, you might be done, but is the customer happy with what you've done? What's left? Let's talk about what's left. And they go, well, it's them. No, there is no them. It's us, right? And so if you're a coach, if you're anybody and anybody on your teams say us and them, they say the business changed their mind or the customer, it's them. Focus them on the fact that there is no them. This is us, including the customer. I know it's hard because everybody says it's us and the customer. No. It's us and only us because customers are part of you and if they're not then you're throwing things at them, right? And sometimes they don't land. You're saying program definition of done like in my head. I'm I'm interpreting it again I'm taking the word program out of it to interpret it as our death like an individual team that's plugged in To the product that all the rest of the teams are working on can have a more restrictive definition They can add things. Here's what i'm trying to say. They can add things to the definition of done In addition to what they're doing that all the rest of the teams agree with. But it can't be less. They can't decide like, oh, we're not going to do automation. That's right. That's right. It has to be additive. Absolutely correct. So just take a metaphor like a car manufacturing. Just because I like that one. Or you could take the restaurant metaphor if you want. The restaurant one, I think it was a cleaner. That's a clean example. Why don't you, why don't you tease that one out? You can say, If you want pasta with a bread sauce and a breadstick on the side. There you go. Every meal absolutely will have to have pasta with red sauce and a breadstick on the side. But some, some dishes I might bring you some parmesan cheese on top. There you go. Some dishes I might bring you some fresh you know, seasoning on top. It's an extra thing our team does that we think accentuates the product. That's right. That we bring to you. But. But we've all agreed like it's definitely gonna have pasta a certain amount. Sure. It's definitely gonna have bread sauce a certain amount That's made a certain way and it's definitely gonna have a bread stick Absolutely every time We've all agreed that all of I guess our restaurant only sells one thing. Well, we'll go to Olive Garden, I guess, but no, seriously, you're right. You're absolutely right. Well, you can't, you can't you can't just decide, well, I don't want to deal with red sauce for this sprint. You cannot. Right, right. You cannot do that. I'll send you the pasta out to the table first. And then in 10 minutes, I'll send the red sauce. Oh, great. When the pasta's cold and yeah, yeah, yeah. Now I can't eat the pasta at all. Somewhere along the lines you decide like The customer ordered pasta and red sauce. I'm gonna give him a bowl of red sauce and They can wait for the bus and somebody to the store to get pasta. No, you you should have a definition done. Yeah, it should be shared across all your cooks in the kingdom and and there shouldn't be compromising on that and if you're having problems with that You need to deal with the problems with like, don't, don't make a bunch of other policies, deal with the problems of that. That's right. I want to add a little bit to this. So this analogy is just talking about ingredients and dishes and whatnot, right? But, but it, it is, but listen, the thing that matters is every single step in the customer experience. Meaning, so let's just stick with the analogy, the restaurant analogy. So yeah, yeah, we make pasta, we make sauce, we make them at the same time or at the time when everything gets together. But then what if you don't have. Wait staff that can deliver it to the table in a timely manner, right? These people aren't creating the dish as such, but they are definitely contributing to the customer experience. You better believe that. So then if you have a, because restaurant is basically a linear system, right? They have orders that come in and orders that go out or, or. Products that go out. If you don't balance that, what's going to happen? Think about it. We've all been to restaurants where we get suboptimal service, where the food is cold, it arrives late, something that arrives before something else, your salad and your main course arrives at the same time, your soup and your dessert arrives at the same time. Like that is what we're trying to avoid here. You know, an an analogy that actually works really well in this example is, high-end restaurants. Oh yeah. High-end restaurants. Yeah. High-end restaurants will never bring one thing out at a time. The point here with the definition of done, the point here is At every high-end restaurant I've ever been to, they will not bring out one piece of food Absolutely. Without bringing the whole table's order out. And they commit to this. And that means that the cooks back in the kitchen have to line up all the dishes together to bring 'em out together. And that's a, that's a level of coordination that, that You know, I guess I'm not trying to say like lesser restaurants, but non high end restaurants lower end restaurants. They just, they just don't care about that. That's not, they haven't perfected the process yet. Let's just say that give them the benefit of doubt, but you're absolutely right. So I, those high end restaurants. accentuate the customer experience, right? In that way, thereby doubling the chances that you may go back there, right? The perceived quality. It's a real thing. That's exactly right again We started this podcast after the last podcast. Are you talking about quality? Yeah, I'm talking the through line of both. These podcasts is quality The DOD, like it's not pitched this way a lot of times, you don't hear it pitched this way a lot of times, it is a way to maintain that high end restaurant quality in your development process. Yeah. That is the way that it's maintained. I feel like we harped on this a while, but it, we ended up on a really good analogy we did of, of high-end restaurants and, and now we're both hungry. And now we are. I'm, I really would like some breadsticks. I'd like to move on to the next category, but Sure. But like we, we, we talked about, we talked about the definition of done, and I feel if we break down the definition of done into a more granular level, yeah, that's fine. So I think we should stick with our restaurant analogy going there as well, because this is like okay. You have your food coming out. Maybe it comes out all at the same time, but nobody's come back or even come to your table, which is really terrible, but nobody's come back to see if your drinks need replenishment. You need some salt or you need some, yeah, you need some other things. And then commitment to continuous improvement of the development process, which is retrospective. So, basically, not skipping retrospectives. And then, and then, Communication with stakeholders, which normally is wrapped in the retrospective, but if you're not doing scrum, I could easily see people pushing back to say, Well, I'm not doing scrum, so I don't need to sprint demo and a sprint retrospective because I'm not doing scrum. I've seen plenty of people do that. I don't need a scrum master because I'm not doing scrum. That's fine, but I need communication with stakeholders and I need continuous improvement spelled out in my definition of done to say, have we looked at the feature in the field in production to see if it meets the customer's actual need. And when that is finished. We can truly say that we are done with this line of work. I could not agree more I mean, I really i'm stretching with a lot of teams normal dods because these are not Some of these are some of these are csed Automation those are normal. Sure, but some of these are not normal to be in the definition of done But I think they probably should be they should be absolutely again I listen I agree with you when something is just put out into the field and you say I'm done. Yeah, you're not really. So going back to our restaurant metaphor, Brian Ohms restaurant, we want to call it right on software development company turned restaurant. That's it. So what happens when You go into a restaurant and everything is going according to plan, meaning you order something, it comes out your entire Party has all their food delivered at the same time. Everybody's ordered something different. It's hot great It's all correct, and it tastes great right, but then you don't hear from anybody. You know, your wait person is AWOL Yeah, that's not good. All your waters are empty. That's right So if you don't have frequent checkpoints from that, from that weight person saying, what else do you need? Right. Can I get, get you another drink? That's what we're talking about here, right? Put it out into the field, but that's not enough, right? You need to then survey how it's going. Is it going good, bad, ugly? Otherwise, how, and then learn from that. If it's going great, that's okay. But how can you make it better? What a terrible situation where everyone is doing. Everyone is working. The best of their ability did their best to deliver the most difficult part to you. They, they did all the food. They cooked all things with different cook times and they arranged it. So they all landed on your table, hot and ready at the exact same time. And then one person doesn't have silverware. What a what a, what a letdown, man. That's the easiest thing to resolve. You know what I mean? So as a restaurant, would you want ownership, right? Would you want to know about that? You should, right? Because that person or that party, in fact, if it's multiple people, guess what? They're going to go and tell people about that. And there's research out there that says a happy customer doesn't really tell more than maybe one or two people. But an unhappy customer... Talks to up to 10 people saying how unhappy they were this this is out there. If you as a restaurant owner don't want to hear about it, that's a huge issue, right, because you're ignoring your customer behavior right there, or the impact of what your efforts are to please your customer, so going back to the business at hand here, Your team has developed stuff, tested stuff. It's great, and they delivered it on in a timely manner. Great. It's now in the customer's hands. Great. You're not done. That's what I would say. And people are listening to this, watching this are going, what do you mean? We're done. It's in their hands. Isn't that what we wanted to do? Yeah, not quite. That's most of what you wanted to do. Did it really satisfy their need? Right? And that little extra step is huge in its impact. Yeah. Miss that at your peril. And that's this is the second podcast we brought this up on. And it's a real high concept for a lot of people. Of course. Because it's not like any of the scrum. No, it's not taught anywhere. We don't bring this up. You know, the idea that Communication, with your stakeholders after you delivered something like you have a real good chance inside the scrum model to do it as part of your as part of your sprint review. Sure. To demo the feature. You know, the, the, the more the customer puts their hands on it, the better in the sprint review. You have two opportunities. One is the interview where you actually walk the user through the functionality and you sit through it and say, what do you like about this? What's frustrating? Does this capture your need? What's left out? You, you, you prompt them with open ended questions, see where they go. That the synchronous method. The asynchronous method is surveys whether it's an after the fact survey that you send them or whether it's, whether you actually build surveys into your application in the world of mobile development, we used to do that. We would build surveys into new fun. So you'd hit new functionality. And the application would know whether this is the first time you've got to that screen or not. And, and it would know to prompt you with the, the pop up that says, Hey, did you like that? How is your experience? Thumbs up, thumbs down. And then if you have thumbs down, if you hit thumbs down, it gave you a little box to put texts in. People could, most people skip that stuff. Sure. But some people put a text in and you would capture that and you, and you're. Theoretically your product people would be reviewing that on a regular basis. Right? You get a lot of like gripes that you would expect. Yeah. But occasionally you get some diamonds. I personally like synchronous better than asynchronous. I agree. I think you get. You get to see the reaction on someone's face. And if someone gives you a sour reaction, you can say, well, why do you, what makes you say that? Or what makes you, you know what I mean? You can dig, you can dig, you can't really dig asynchronously. You can. And people fake asynchronous as well to a large degree. So the majority of people skip it. Yeah. And the other thing they tend to do is oftentimes you see these things like NPS queries, things like they just give you a scale one to 10. Right? How likely are you to recommend that? Right. Okay. But to your point, if you see people hesitating, wincing, You can actually dig deeper there. So I think there's no, there's no doubt. That's a much better way to go. A lot of times in coming back to our DOD discussion, a lot of times people don't even think about all of that. They just say, well, it's already delivered to production. So we met that milestone and we're done. Yeah. Like you really aren't done. It's not enough to get to a point that you think is the end point because it's not the end point. Think about... A lot of people get hung up on this and say, That's not us, right? That's the marketing people, the sales people, the product people. Like, that is the problem. That is the problem when people say, that's not us. Why are you here? You're here to delight the customer. Listen, not satisfy them. Because you satisfy them, that's okay. podcast... Take that the product manager on the podcast said listen, you can have 500 bugs in your backlog and your product can have, you, you might have to tip toe through your product in order to make it through a transaction successfully. However, if you're truly solving a business problem during your tiptoeing through, through, through the happy path through your application. And you're, you're truly involving your customers and, and, and really dedicating toward that customer experience, what we're talking about, the actual, when you deliver the meal, it is hot, it is fresh, it tastes good, and it's made exactly the way the customer wants every time. I mean, the customer will deal with a lot of pain in the application to get through that path. I worked on a lot of applications that are really, really terrible once you stray from the path. Sure. Problem that they solve is so painful. The customers are willing to come back again and again and again, no matter how bad the product is, no matter how bad the businesses run behind the scenes. So there there's light at the end of the tunnel. Like that, we, there's a couple of strategies that we gave segment your bugs, tackle them tackle them one at a time. It seems daunting to start with, especially when things are in the past. But you got to start somewhere, the quality starts with you. Absolutely. I want to just end with the tip, right? Which is you can have stories, you can have features, you can have this and that. Why don't we talk about putting up a simple statement? And this doesn't have to be a work item. So in the context of the time box, think about all the work within that time box and say, as a team, which is everybody, product, everyone, sales, even right. Customers get them involved and say, is this really what we're trying to do? Right. Add to this, please. So if you're missing that, they will tell you, and then you can course correct quicker because you're in the time box as opposed to like sometimes later when things are already out of hand. All right. Well, that's this podcast. I don't know if we'll do another one in our QA series of podcasts, but this is fun. We, Never said ever. We might, especially if you guys let us know in the comments below, never should ever and like, or like, and subscribe any which way you want but just hit that button.