#55 - Measureing Agile Progress Effectively

 How you measure the progress of an agile team will fundamentally effect how much value that team will produce, since that measurement will reflect that value. What should that measurement consist of?  That's the topic for this week. 

Transcript

Measuring Agile Progress Effectively

If part of your responsibility is to manage a team of software developers, and if you are adhering to an agile process, then how can you best measure the progress of that team?  This can be especially challenging if you are not, yourself, a technical expert. 

An old saying in project management is "you get what you measure", and so this is an important choice to make if you want to make sure that the progress indicated by that measurement actually reflects the value of the team's work. 

Traditional measurements probably will not do the trick. 

For example: if you measure the team based on how many lines of code they generate in an iteration, then you will certainly get a lot of code generated, but there is no guarantee that it is the right code, that it is good code, or that the product they've made has sufficient business value. 

That one might be obvious to you. Here's one that probably isn't:

Sometimes a team's effectiveness is measured by the rate of which they generate defects that have to be remediated in the system. The idea is that a low defect rate indicates quality work, and will also indicate that the team is not creating waste in their process. 

This seems reasonable until you ask yourself "how do we determine that defect rate?" The simplest and more straightforward way to do so would seem to be to divide the number of features generated by the team over the number of bugs that make their way into the product. 

Here's the problem with that: The bugs that get counted are typically the bugs that get reported back to the team by customers. But the only bugs that get reported are on features that actually get used in the field. What about features that nobody actually uses? It turns out that there can be a whole lot of those in a traditional process.

Think about Microsoft Word, as an example.  I'm not picking on Microsoft, it's just that most people are familiar with that product.  There are features of Word that I guarantee you have no idea are there.  You never need them, you never use them, and yet you paid for them when you purchased that software.  There are features in Word that I suspect nobody uses, or that are so seldom used that it's unlikely anyone would report a bug in them.  Why is this?

Processes like the waterfall tend to lead the team to develop features like that. This is because the waterfall process gathers requirements in the inception phase and discourages any addition of requirements later in the process of development. Stakeholders, realizing this, will front load all the requirements they can imagine since they know they will probably not be able to easily add them later.

In an agile process, on the other hand, each iteration of work provides an opportunity for re-prioritization of the requirements. That means there is no reason to front load all the requirements, because the expectation is that they can be added later as part of the process of validating the work. It is much less likely for features to be prioritized that few or no customers want, because they never are evaluated as the "next most important thing". That means that far fewer unused features are developed into the system.

But the math is the math and so if there are unused features, and if unused features never get a bug report, those features appear to be bug free even though they probably are not. This drives the bug down down for a waterfall team making it appear that they are doing higher quality work. I guarantee you that this is not true but, once again, you get what you measure.

So what is a good, valid way to measure the progress of an agile team that would show the value of their work as opposed to a waterfall team?

Lean theory says it is more important to eliminate waste than it is to increase velocity when you want to maximize the bottom line. One element of waste is delays. So one thing to measure is this: Once the process of development has begun, how long does the organization have to wait before they start getting value out of the team? 

In the waterfall process, typically no value is delivered by the team until the product is complete.

In an agile process value is delivered after the first iteration of work, and will continue to be delivered every time an iteration is completed.

So if the measurement is "time to value", then an agile team will always be measured as more effective than a waterfall team, which they in truth will be.

But this means that the visibility of that value must always be clear, and that means we need a way to radiate the information that indicates the work of the team to all stakeholders. I'll talk about that next week.