When KPIs Go Wrong: Lessons from Cobras and Rats (AI voices Arkaro content)

Welcome to the Arkaro Insights podcast. This episode is based on original content developed by Arkaro. At Arkaro, we're committed to innovation in everything we do—including how we share our insights. We've utilised advanced AI technology to transform our written expertise into this conversational format, making our content more accessible and convenient for our busy B2B audience. What you'll hear is a two-person discussion generated through AI voice technology, designed to deliver our insights in a more engaging way than traditional reading. As we continue to evolve this approach, we genuinely value your feedback. Thank you for listening to Arkaro Insights, where professional expertise meets innovative delivery.

Full article: When KPIs Go Wrong: Lessons from Cobras and Rats

Every organisation wants to improve performance, but what happens when the very metrics designed to drive success end up causing more harm than good? This fascinating deep dive explores Goodhart's Law – the principle that "when a measure becomes a target, it ceases to be a good measure" – and reveals how this deceptively simple concept plays out with sometimes disastrous consequences.

We journey through compelling historical examples, from the colonial authorities who inadvertently created rat farming operations in Vietnam to the British rulers whose cobra bounty program backfired spectacularly in India. These historical anecdotes illuminate the same psychological forces that drove modern corporate scandals at Volkswagen and Wells Fargo, where narrowly defined metrics and intense pressure to hit targets led to systemic deception and billions in damages.

But this isn't just a cautionary tale. We examine practical strategies for creating measurement systems that actually drive the right behaviors and outcomes: balancing multiple metrics instead of fixating on single numbers, carefully linking measures to incentives, regularly reviewing and adjusting your KPIs, testing foundational assumptions, and incorporating qualitative feedback alongside quantitative data. You'll learn how to use metrics as helpful guides rather than rigid dictators, and how to build measurement approaches that create genuine value for customers, employees, and organisations alike.

Ready to rethink how you measure success? Visit arkaro.com to learn how our consultancy can help your organisation implement effective strategies for measurement, change, and innovation. For a free consultation about your specific challenges, contact Mark Blackwell directly at mark@arkaro.com.

Connect with Arkaro:

🔗 Follow us on LinkedIn:
Arkaro Company Page: https://www.linkedin.com/company/arkaro
Mark Blackwell: https://www.linkedin.com/in/markrblackwell/
Newsletter - Arkaro Insights: https://www.linkedin.com/newsletters/arkaro-insights-6924308904973631488/

🌐 Visit our website: www.arkaro.com

📺 Subscribe to our YouTube channel: www.youtube.com/@arkaro

Audio Podcast: https://arkaroinsights.buzzsprout.com

📧 For business enquiries: mark@arkaro.com

Transcript

Introduction to Goodhart's Law

Speaker 1 0:47

Welcome to Arkaro Insights, the show where we explore the latest thinking and change and innovation to help B2B executives like you achieve better results.

Speaker 2 0:56

Absolutely, and today we're going to be diving deep into an article that you shared with me, called when KPIs Go Wrong Lessons from Cobras and Rats.

Historical Examples: Rat Tails and Cobras

Speaker 1 1:05

Yeah, it's a fascinating piece by Mark Blackwell who, as many of our listeners know, leads Arkaro, a consultancy that partners with B2B leaders on these exact challenges.

Speaker 2 1:15

And it really gets to the heart of something that I think every organization struggles with, which is how to measure success and how to drive improvement without falling into some pre-common traps.

Speaker 1 1:24

That's right, and the article uses this principle called Goodhart's Law to illustrate what can go wrong when you're not careful with your key performance indicators, or KPIs. Have you heard of Goodhart's Law before?

Speaker 2 1:36

I have. Yeah, it's the idea that when a measure becomes a target, it ceases to be a good measure, and it's one of those things that sounds deceptively simple, but when you start to see it play out in real life, it can have some pretty dramatic and sometimes even comical consequences.

Speaker 1 1:50

Yeah, and the article starts with these two historical anecdotes that I think really bring Goodhart's Law to life. The first one takes place in Vietnam in 1902, under French colonial rule.

Speaker 2 2:01

Right. So they had this huge problem with rats. They were everywhere, causing all sorts of damage and spreading disease.

Speaker 1 2:07

The classic public health nightmare.

Speaker 2 2:09

Absolutely, and so the authorities thought OK, we've got to get rid of these rats. How do we do that? Let's offer a bounty for every rat tail that's brought in. Sounds reasonable, right? You bring in a tail, you get paid. Fewer tails, fewer rats.

Speaker 1 2:26

Makes sense. On the surface Seems very straightforward.

Speaker 2 2:28

But of course it didn't work out quite as planned, because what started happening was people were finding all these rats without tails.

Speaker 1 2:36

Wait what? How do you have rats without tails?

Speaker 2 2:38

Well, it turned out that some enterprising individuals figured out that they could make a lot more money if they just caught the rats, chopped off their tails and then let them go again.

Speaker 1 2:47

Oh, so they could just keep breeding and producing more tails.

Speaker 2 2:50

Exactly, they turned rat control into a rat farming operation. And the officials? Well, they were completely baffled until they figured out what was going on.

Speaker 1 3:00

Talk about unintended consequences. The metric they chose, the number of rat tails, became the sole focus and it completely distorted the intended outcome.

Speaker 2 3:09

Exactly. And the second story is just as bizarre. This one happened in India under British rule, and this time the problem was cobras.

Speaker 1 3:18

Okay, so even more dangerous than rats.

Speaker 2 3:21

Definitely, and the solution the British came up with was pretty similar. They offered a reward for every dead cobra that was brought in.

Corporate Scandals: VW and Wells Fargo

Speaker 1 3:28

So again seems logical, instead of asking people to get rid of the cobras, problem solved.

Speaker 2 3:34

But once again, people found a way to game the system Instead of hunting down wild cobras, they started breeding them.

Speaker 1 3:40

Cobra farms.

Speaker 2 3:41

Seriously, seriously, they would raise these cobras specifically to kill them and collect the bounty. And when the British authorities finally caught on and stopped the program, guess what happened?

Speaker 1 3:51

I'm guessing they ended up with even more cobras on their hands.

Speaker 2 3:53

Right. All those farmed cobras were released into the wild, making the problem even worse than it was before.

Speaker 1 4:00

It's amazing how these seemingly straightforward solutions can backfire so spectacularly, and both of these stories really highlight the core of Goodhart's Law, don't they?

Speaker 2 4:10

Absolutely, because the moment you make a measure, a target, people will find ways to optimize for that target, even if it means distorting the system or completely missing the original goal.

Speaker 1 4:21

And it's not just a historical curiosity. This happens all the time in the business world. In fact, the article goes on to discuss some pretty high profile examples.

Speaker 2 4:29

Yeah, like the Volkswagen emission scandal, also known as Dieselgate, that one was a pretty clear cut case of Goodhart's law in action.

Speaker 1 4:38

Remind us what happened there.

Speaker 2 4:39

So Volkswagen was under a lot of pressure to meet these really stringent emission standards for their diesel cars.

Speaker 1 4:45

Right and they had made this big public commitment to being a leader in clean diesel technology.

Speaker 2 4:50

Exactly. But instead of actually investing in the technology needed to genuinely reduce emissions, they took a shortcut.

Speaker 1 4:56

A very expensive shortcut, as it turned out.

Speaker 2 4:59

Yeah, they developed this software that could detect when a car was undergoing an emissions test, and it would basically put the engine into a low emissions mode.

Speaker 1 5:09

So it was like a cheat code to pass the test.

Speaker 2 5:10

Pretty much. But out on the road, under normal driving conditions, the cars were emitting way more pollutants, sometimes 40 times the legal limit.

Speaker 1 5:18

So the target was low emissions, the measure was passing the test and the method they used to achieve that measured success was complete deception.

Speaker 2 5:27

And it all blew up in their faces. They ended up having to recall millions of cars, pay billions of dollars in fines and their reputation took a massive hit.

Speaker 1 5:37

It's a classic example of how focusing too narrowly on a single metric, especially when the stakes are high, can lead to disastrous consequences.

Speaker 2 5:45

And the Wells Fargo account fraud scandal is another great example. Remember that one.

Speaker 1 5:48

Vaguely Something about employees opening fake accounts.

Speaker 2 5:51

Yeah that's the one. So Wells Fargo had these incredibly aggressive sales targets for their employees, particularly around cross-selling, which is basically getting existing customers to open more accounts.

Speaker 1 6:03

So the more accounts they open, the better they looked and the more money they made.

Building Better Metrics Systems

Speaker 2 6:06

Right, but the problem was, these targets were so unrealistic that employees felt pressured to do whatever it took to meet them, and so they started opening millions of unauthorized accounts for customers without their knowledge or consent.

Speaker 1 6:20

Wow, so they were literally creating fake accounts just to make their numbers look good.

Speaker 2 6:25

And it went on for years before it was finally exposed. The consequences were huge fines, lawsuits and, again, massive damage to their reputation.

Speaker 1 6:34

It's almost like they forgot that the purpose of a bank is to serve its customers, not just to generate profits at any cost.

Speaker 2 6:41

And that's the danger of these narrowly defined metrics. They can create these perverse incentives that drive people to do things that are ultimately harmful to the organization and its stakeholders.

Speaker 1 6:52

So how do we avoid falling into these traps? How do we make sure that our KPIs are actually driving the right behaviors and leading to genuine improvement?

Speaker 2 7:00

Well, the article outlines several strategies, and I think one of the most important ones is to use multiple metrics.

Speaker 1 7:06

Instead of just focusing on one single number.

Speaker 2 7:09

Exactly, because if you're only tracking one thing, people will naturally find ways to optimize for that one thing, even if it means neglecting other important aspects of the business.

Speaker 1 7:18

So it's about having a more holistic view of what success looks like.

Speaker 2 7:22

Right, and one way to do this is to balance your lagging indicators, which are the outcomes you're ultimately trying to achieve, with leading indicators, which are the activities and processes that you believe will drive those outcomes.

Speaker 1 7:37

Can you give us an example?

Speaker 2 7:38

Sure, let's say you're running a sales team and your ultimate goal is to increase revenue.

Speaker 1 7:43

Pretty standard goal for a sales team.

Speaker 2 7:44

Right, so revenue would be your lagging indicator. It tells you how well you're doing overall, but it doesn't really tell you how to get better.

Speaker 1 7:51

Because it's a result of a lot of different factors.

Speaker 2 7:53

Exactly. So you might also track some leading indicators like the number of sales calls made, the number of qualified leads generated or the average deal size. So by tracking these leading indicators, you can get a better sense of what's actually driving your revenue and where you need to focus your efforts your revenue and where you need to focus your efforts Precisely, and it also helps to prevent people from just focusing on the easiest way to hit the revenue target, which might not actually be the most sustainable or beneficial approach in the long run.

Speaker 1 8:20

Another strategy the article mentions is to be careful about how you link your metrics to incentives.

Creating Holistic Measurement Approaches

Speaker 2 8:26

Yeah, this is a big one, because if you tie people's compensation or bonuses too closely to specific metrics, you can really amplify the risk of gaming.

Speaker 1 8:36

So it's like offering a reward for every rat, tail or dead cobra.

Speaker 2 8:40

Exactly, and it can create this culture where people are so focused on hitting their numbers that they lose sight of the bigger picture.

Speaker 1 8:47

So what's the alternative?

Speaker 2 8:48

Well, one approach is to use a more balanced system of rewards, where you're taking into account both the leading and lagging indicators.

Speaker 1 8:55

So you're rewarding people for both the effort and the results.

Speaker 2 8:58

Right, and you're also looking at qualitative factors like teamwork, customer satisfaction and innovation.

Speaker 1 9:04

So it's not just about hitting the numbers, but about how you hit them.

Speaker 2 9:08

Exactly, and another important strategy is to regularly review and adjust your metrics.

Speaker 1 9:13

Because what might have been a good measure six months ago might not be so relevant today.

Speaker 2 9:18

Right, and as you learn more about your business and what drives success, you need to be willing to adapt your metrics accordingly.

Speaker 1 9:25

Otherwise, you risk falling into the trap of measuring the wrong things.

Speaker 2 9:29

And that can lead you down the wrong path. The article even suggests that changing your metrics can actually be a sign of a healthy organization.

Speaker 1 9:37

How so.

Speaker 2 9:38

Because it means that you're constantly learning and evolving and you're not afraid to challenge your assumptions.

Speaker 1 9:44

So it's not about sticking rigidly to your original plan, but about being flexible and adaptable.

Speaker 2 9:49

Exactly, and another key point the article makes is the importance of measuring your foundational assumptions.

Speaker 1 9:55

What does that mean?

Speaker 2 9:56

Well, every strategy, every change, initiative is based on certain assumptions about how the world works, what your customers want and what will drive success.

Speaker 1 10:05

Right, and those assumptions might not always be accurate.

Speaker 2 10:08

Exactly so you need to be actively testing those assumptions and be willing to adjust your course if the data tells you that they're no longer valid.

Speaker 1 10:17

So it's about being data driven, but also being open to questioning your own beliefs.

Speaker 2 10:22

And finally, the article talks about the importance of incorporating qualitative metrics into your measurement system.

Speaker 1 10:28

So it's not just about the numbers.

Speaker 2 10:30

Right, because the numbers can only tell you so much. They can tell you what's happening, but they don't always tell you why.

Speaker 1 10:36

So you need to get the human perspective as well.

Speaker 2 10:38

Exactly, and that means gathering feedback from your customers, your employees and other stakeholders.

Speaker 1 10:44

Because they can provide insights that you wouldn't get from the data alone.

Speaker 2 10:48

And those insights can be invaluable in helping you to avoid the pitfalls of Goodhart's law and to make sure that your KPIs are actually driving the right behaviors and leading to the outcomes that you're looking for.

Key Takeaways and Episode Closing

Speaker 1 10:59

So, to sum it all up, it sounds like the key message is to move away from this narrow, target-obsessed mentality towards a more holistic, adaptive and nuanced approach to measurement.

Speaker 2 11:12

Absolutely. It's about using KPIs as a guide, not as a dictator. It's about understanding that what you measure is what you get, and so you need to be very careful about what you choose to measure.

Speaker 1 11:23

It's about creating a system of measurement that helps you to navigate complexity, to make better decisions and to achieve sustainable success, and it's about remembering that the ultimate goal is not to hit the numbers, but to create real value for your customers, your employees and your organization as a whole.

Speaker 2 11:40

Exactly, and if you can do that, then you're well on your way to avoiding the cobra and rat traps and achieving the kind of lasting success that we all strive for.

Speaker 1 11:50

Well, we hope this deep dive into Goodhart's Law has given you, the learner, some valuable food for thought. If you're looking for support in developing and implementing effective strategies for change and innovation, including designing impactful and balanced metrics, we encourage you to visit Arkaro website at Arkaro or follow them on LinkedIn.

Speaker 2 12:09

And if you'd like to have a free consultation to discuss your specific needs and challenges you can reach out to Mark Blackwell directly at mark@arkaro. com.

Speaker 1 12:19

Thank you for listening to the Arkaro Insights Podcast. We hope you found this deep dive helpful and, if you did, please share it with your colleagues. Until next time, take care.