A Job Done Well - Making Work Better

The World's Most Valuable Metrics – With Bernie Smith

• Jimmy Barber, James Lawther and Bernie Smith • Season 2 • Episode 25

Share episode

Send us a text

Bernie Smith, KPI guru, is back to help us find 'gold bars' - those vital measures that drive business success. 

Bernie shares a simple three-step method for unlocking those gold bars, using examples as diverse as predicting an international incident from pizza sales data, spotting a COVID outbreak using scented candle complaints, and calculating the number of piano tuners in Chicago.

Despite being an analytical love fest, this is one of our most entertaining and informative episodes. (They only briefly discuss celery cell growth and paper moisture levels!)

Hello, I'm James. Hi, I'm Jimmy and welcome to A Job Done Well, the podcast that helps you improve your performance enjoyment at work.

James (2):

those of you who listened to last week's episode, and if you haven't, you really should, we had a special guest, Bernie Smith, who, is a, um, a KPI guru, if there's such a

Jimmy (2):

was, I was outnumbered by analysts.

James (2):

Two of us, yeah. Bernie gave a really interesting talk about how you select metrics and how you then um, Narrow the field down But he also tantalizing gave us a bit of a spiel about The gold bar matrix the things that you really should be measuring that you can't So we're gonna go and we're gonna chat about that this week. Hope you enjoy it

Bernie:

This is where the really, really interesting stuff lies. So these are things that are game changingly important sometimes. but appear to be impossible to measure.

James:

have you got any examples of things you would regard as what you call goal bars that people will be able to relate to so they really understand

Bernie:

they're kind of peculiar because, They are so diverse. I mean, I actually keep a list of interesting KPIs. I know this is sad, isn't it?

James:

Jimmy is closing over a list of interesting KPIs.

Jimmy:

like, what's your favorite Excel function? It's like,

Bernie:

don't get me started. No. Um,

Jimmy:

for the two of you.

James:

Hey, you're not joining

Jimmy:

I'll sit, I'll sit that one out.

Bernie:

I'm 54 bit slow on recall, so I've pulled up my list and things like, Have we got a wave of COVID coming? really simple question to ask. people, they start off by going, well, we could look at hospital admissions. We could look at number of free hospital beds. Well, you know, if you're in the health service, and you are planning ahead, that's no good. You're in,

James:

it's a bit

Bernie:

you're in the middle of a storm. So, When you start to think about it, you go, all right, what, what happens when people have COVID? Well, there's a few interesting things. Firstly, people start buying tests, but before they even buy tests, they'll often Google the symptoms. And before you're symptomatic, you shed RNA in your urine and feces. So what you can do is you can sample sewage. and look for these RNA markers to tell you you've got a wave of COVID coming. and that's powerful because it's typically up to four days before people are symptomatic. You can also get a measure of magnitude by the concentration of those, markers in the sewage. And if you think about it, sewage, You take a whole city and you funnel it into a handful of sampling spots. So a grim job, not exactly what you call a dream measure, but it's a good way of getting a headstart. Now, there are other ways of doing this. someone looked at the correlation between scented candle complaints on Amazon and outbreaks of COVID, and it turns out you see a statistically significant uptick in complaints about scented candles, losing their smell. and an outbreak of COVID because a certain proportion of the population lose their smell, bought scented candles and go, these scented candles don't work. Um, so you could monitor scented candle complaints as a really rough indication of an outbreak of COVID.

Jimmy:

It's interesting because I'd heard, well, I guess on a more complex level, the Google search thing. So I know that they did a lot of work with the American health agencies. across North and South America to track things like flu epidemics and stuff like that by Google searches, where search terms around symptoms were very, very highly correlated to outbreaks. the other end of the sophistication, but quite often I've found that when you go into organisations, you start talking about what's important. There's sometimes there's things like you say, around customer behavior that you know is important. But you don't have an easy way of measuring at any one time. So your gold bar isn't necessarily hugely complex and sophisticated. And you feel like you should be able to know, are your customers happy with the service that you're providing, but there aren't, there, there aren't the KPIs around that you're measuring at that point.

Bernie:

got a great example from the pension industry. So, I was sat in an ops meeting in a pension company up in Scotland, and they'd seen a big uptick in the number of people transferring their pensions out. Now, that's bad news if you're at a pension company. You basically want people to come invest with you and stay there till they die. So, they're having a bit of a, bit of a flap about this. And During the ops meeting, one of the middle managers quipped that it's not like we can read our customers minds. And he started laughing at his own joke, which was quite irritating. Um, but then one of the, uh,

Jimmy:

James note.

Bernie:

Yeah. And then, then one of the younger, ops improvement people piped up and she said, actually, I think there is. and what she'd spotted was that. There are certain things that people do just before they transfer their pension. Now this is going back quite a bit before internet self service. And she spotted that pretty much the last thing someone does before they transfer their pension is they contacted the organisation for evaluation. Cause if you're going to transfer, you want to know what your pension pot's worth. And interestingly, that also explained why they'd seen this uptick in transfers because they'd automated that process. So, it went from talking to a nice. Edinburgh agent who would help you out to a robot reading out the value of your pension pot. So that had actually removed one of the last intervention points and they'd seen this increase. But the crucial thing here is that there are behaviours that that tell you about intent. So they're everywhere. I mean, I left.

James:

Well, so sorry, let me cut across there because there's two things there, isn't there? Right, so you're absolutely right. There are behaviours that show intent, but also if you don't think about it, what you do is you go and automate something, which is absolutely critical to your business. I mean, it's the last thing

Bernie:

Yeah. You put your very best agents on that call and, and you work really hard to keep them on board. And that's what they did. But yeah, I think what

James:

Yeah. So anyway, you've talked about three ways of measuring the immeasurable. So I've got these immeasurable, yeah, how on earth did I go about finding

Bernie:

Okay. So the three steps, the first one you're going to think this is, is a trick, but if you've ever done an academic project, you know, the first thing you do before you invest your life in, the bruising characteristics of potatoes or compression ratio in, uh, in turbine casings is

James:

celery cell growth for me,

Bernie:

So, the first thing you did, presumably, was go and see if anyone else has delved into the world of celery cell growth. Because if you go and spend three months figuring out how to do it and find someone who's already done it, you're going to feel a bit of a nana, aren't you? So, step number one on measuring the unmeasurable is, has anyone measured it before? Because if they have, they've done it. Bingo. You know, we go off, we'll borrow their method. And I've got one example of that. So I was working in the food processing industry. I won't be too specific to protect the innocent, but they were creating a very moisture sensitive product, 48 production lines in parallel, belting out product. And moisture was critical. Too dry and it cracked, too wet and it rotted. And it was a high care product as well. So you really didn't want it rotting. and their method of measuring moisture was to take it off the end of the production line, weigh it, put it in 24 hours and weigh it again. As you can imagine, you can make a lot of waste product in 24 hours, of

James:

yeah, yeah,

Bernie:

48 lines, so they're bringing their hands thinking, can we dry it out quicker? I'd worked in the paper industry. In the paper industry, you use infrared absorption as a non contact real time measurement for paper moisture content. Moisture is everything when you're making paper moisture and caliper, how thick the sheet of paper is. Those are two absolutely critical parameters. So get on the phone, call these, paper measurement, people. Within two days, they had an infrared absorption meter on their phone. On the line, measuring moisture content in real time.

James:

Yeah, and that's fascinating right because we have this whole not invented here thing and you don't understand our industry But actually if you can swallow your pride and have a go go and see what other people are doing Invariably you will find a solution to your problem because none of these problems are novel. Usually they're not novel

Jimmy:

and when you say somebody's done this before, I presume you are thinking universally as opposed to have we done it here in this organisation.

Bernie:

that's a good first port of call, but absolutely. you basically become more flexible as you don't find an answer. sometimes you'll find that literally the person in the next building has done it in your company. If they haven't done it, then you look around the industry. If you haven't done it in the industry, you might step outside of your industry and look for things that you can adapt or flex or, adopt, but invariably it massively de risks the whole process if someone's been through that development process already. So if you think about the infrared absorption, there was an entire company that spent several years ironing out all the wrinkles, getting it working. So you really don't want to have to do that again yourself. If you can just pay someone to come along and do it.

Jimmy:

So, so between moisture in paper and celery cell growth, you pair off must be fun at parties, aren't you?

James:

Oh, yeah

Bernie:

just run out the room.

James:

Yeah unstoppable, you know, the effect we have on women it has to be seen right? So there you go. So that's the first thing copy what somebody

Bernie:

Yeah. Yeah. It's basically, has it been done before? Yeah. But you're, you're right. So that's, that's tool number one because it's quick. It's easy. so it's having to think the next one is, does it leave footprints? So this is really just a fancy way of looking for what's called proxy indicators. you're looking for measurable things that happen before, during or after the thing that you care about. so there's absolutely loads and this is why I need to keep a list. So we talked about. RNA shedding is an example of, of COVID elevated heart rate. There's a really good leading indicator of whether you're coming down with a virus. It's not cast iron, but if your heart rates up by 10, 15 beats per minute, when you're resting, it's a pretty good chance you're coming down with, with some sort of ailment. that's a proxy indicator. There's loads of them. if you're in healthcare, you can, there's a billion different ways of measuring how someone healthy someone is, but a really good rough indication of someone's remaining life expectancy is how long they can stand on one foot. Now,

James:

At which point I have to say my wife and I have been practicing, although I don't think practicing

Bernie:

I thought you were looking

James:

I think the point is, We have been practicing,

Bernie:

Yeah. So, I mean, I guess, yeah, there probably is a causal mechanism around, nerve circulation, muscle control, strength and whatever, but it's, it's a very quick ready reckoner. so proxy indicators often aren't precise, but they are often leading and leading indicators are particularly hard to find. These are the ones that people really struggle with. So if you, you talk to finance people, They live in almost entirely in a world of lagging indicators. So they will dismiss leading indicators and no offense to finance people, you know, some of my best friends are finance people, but they're kind of like engineers often wired in to looking in the rear view mirror. leading indicators are often less precise, but still very useful.

James:

but this gets back to the whole point of target setting, right? Because if the aim is to live longer you can measure that by how long people stand on one foot. It is absolutely pointless then giving people a target to say how long can you stand on one foot, because that won't make them

Bernie:

Yeah, it's, it's, it's, it's, it's an outcome indicator rather than an input indicator.

James:

Yeah. The other one I read about which I thought was really quite interesting was books make you clever. Well books don't make you clever at all, but there was a measure done, and it showed that children who came from families which had lots of books in the house. invariably did better at school. Of course, it had nothing to do with the bloody books. It was the attitude of the parents who bought

Bernie:

Yeah.

James:

That was the key point. Yeah. So, and it's just, you need to be very careful about understanding the measure,

Bernie:

correlation versus

James:

is a great indicator. Exactly

Bernie:

So there's a, there's a classic book, How to Lie with Statistics by Donald Huff, I think, where he talks about the correlation between number of storks in a house and number of children. And the reason they're linked, um, is because more children, bigger house, more chimney pots, more stalks.

James:

Uh, there

Bernie:

so there is a causal link, but it's not the way around that you think. So you, you, you're absolutely right. Um, I can't, can't say for sure that proxies are always valuable, but you know,

James:

No, no, no, but that's not the point though, is that the proxy is valuable because it shows you what may well be about to happen. The trick is not to manage the proxy. If you see what I mean, it's to

Bernie:

Yeah, completely profits. Another one, you know, that's, I mean, it's not, it's not a proxy and it is lagging, but it's one where Setting a target on profits, fairly pointless, is actually setting a target on the activities that lead to profit.

James:

This is perfect, absolutely. Yeah,

Bernie:

I mean, you see proxy indicators used a huge amount in the world of intelligence. So I'll give you two examples. you can get a pretty good indication of whether, uh, there's an international incident brewing by the dominoes orders around the pentagon. So if there's a sudden surge in pizza orders in

James:

Yeah,

Bernie:

dominoes around. Yeah. there's, there's something going off now. Could you relate that to the exact severity of the incident? Probably not, but it's enough to make you think, Hmm, this is interesting. Um, another proxy indicator, they use the number of cars parked in car parks in Russia in military bases as a proxy for unit strength. So as the units become depleted, the car parks become emptier as well. So if you've got a military base, you can get some indication of how many people are in that base by the number of cars, driving to and from on a regular basis.

Jimmy:

I think, I think one of the things though about proxy indicators are the clues in the title, they're an indicator, quite often when we're talking about any form of indicators, we expect a level of precision, whereas like you say, the, the dominoes example, it gives you an idea that there's something going on, it doesn't give you the precision, and in our desire for precision and perfection, We can tend to dismiss things that help us understand are we going in the right direction versus are we pinpoint going to the exactly the right place.

James:

Oh, absolutely, but there's also, I mean, if I could be 10 percent at betting on the horses than everybody else, that doesn't mean I'd be right, but I'd clean up at the horses, wouldn't I? So it's all about competitive

Bernie:

That, that's exactly it. Uh, a slightly decent measure is of something very, very important is better than no measure at all. So, so we have to be pragmatic and say, okay, let's go with the best that we've got.

The show you're listening to right now. Brings you insights to help you improve performance and enjoyment of work. It takes a long time to produce, but we think it's well worth it. It's all in the name of helping you get your job done well. All we ask in return is that you share the link to this show with someone that you think would benefit from it. And if you haven't already, click on follow the show wherever you're listening to this podcast right now. Anyway, let's get back to the show.

James:

So there's number two,

Bernie:

The final one is called a Fermi Estimate. Now the reason it's called a Fermi Estimate is Enrico Fermi, one of the fathers of the atom bomb, super mega genius Italian, uh, physicist. after the Manhattan Project, he went off to teach physics at Chicago University. And for his first year physics students, his first challenge to them was, um, a microscope. I will tell you the population of Chicago. I want you to tell me how many piano tuners there are. And the way he expected them to solve it would go along these kinds of lines. Well, let's say the population is 500, 000 people. So how many families do we think there are in Chicago? You might say 100, 000 families perhaps. And of those, what percentage might have a piano? Let's say, you know. 20%. So we're down to 20, 000 pianos. Then we say, how often would a piano need tuning? You say every two years, how long would it take to tune a piano? Typically two hours, including travel. Um, so how many pianos could a tuner tune in a day? Um, how many tuners, how many pianos could a tuner tune in a year? Taking out holidays. Um, and you could very quickly. Get to an order of magnitude estimate of how many piano tuners are out there tuning pianos now There's a whole series

James:

Now I have to, sorry, cut across now. I heard this story before and I was sad enough to try it in London and I got the, the yellow pages, online yellow pages as it is now, and I was within about 30 percent. Yeah, so

Bernie:

You're sad. I did the same thing. I got a similar distance from it. And if I remember correctly yelp

James:

Jimmy's sitting there thinking I

Bernie:

Yeah, so

James:

that. Yeah, how many piano tuners in London,

Jimmy:

I'm thinking

James:

the answer by the end of

Jimmy:

30 percent accuracy isn't getting me that excited, frankly.

Bernie:

but it's great if you want to go into a market. So if you're looking at a market and you say, well, I sell 10, 000 units, a hundred thousand or a million units, that's where a Fermi estimate becomes useful, where you're looking for order of magnitude. And you have to remember, this is our weapon of last resort on the unmeasurable. So these are things where everything else has failed. Now, there are ways you can make,

Jimmy:

Sorry, Bernie. So you do, the steps are, you do them in this order. Has anyone else got

Bernie:

Yeah.

Jimmy:

What proxy measures have I got? And then the Fermi estimate. So it's in that, that's, those three steps are in order.

Bernie:

Um, cause the, one of the nice things about the proxy in particular is you manage to already have the data. so that's why we do it before the Fermi estimate. because there are often new ways of looking at existing data in organisations that. it can surprise you. Anyway, going back to the Fermi estimate, there's a few things we can do to make them safer to use because they are stacked estimates. So typically we'll do upper and lower bounds. So we'll talk about the most pianos that someone could tune in a day and the least. So we create upper and lower ranges. So we come up with an upper and lower bound so we can get some measure of uncertainty. The other thing is there's often multiple ways So, I get my, my groups to look at number of new cars sold in the U S each year, and you can do it on car replacement per year, or you could do it on the lifespan of a car. You can do it on percentage of the population that drive, or you can do it on number of families and number of cars per family. And I often encourage people to do it two or three different ways. Routes. And if all three routes come up in the same neighborhood, you have an increased confidence. And often elements of the estimate you have data on. So you can go out and find out certain parts of the estimate, or if you need to improve the estimate, you can go out and do some sampling.

James:

the whole point about triangulating in and doing it in a number of different ways. I read this on the internet, so you can take it with a pinch of salt if you want. But I am reliably told, I am reliably told, that if you want to work out how much a cow weighs, Yeah, you can walk into the center of nottingham And you can walk into nottingham town square and I assure you there are very few farmers and vets in the center of nottingham so they haven't got any idea how much a cow weighs But if you ask somebody they'll give you an answer and if you ask somebody else they'll give you an answer But if you ask 20 or 30 people and then you average that answer out You will get a remarkably good approximation for the weight of a cow, because everybody is an equal amount out. So by the time you average it out, you um, you get there. So the whole triangulation point is really important. There you go.

Jimmy:

your, um, what's your favorite internet quote, James?

James:

Uh, I don't believe everything you say on the internet, Winston Churchill.

Jimmy:

Rest my case.

Bernie:

excellent.

Jimmy:

but Bernie, just, um, If, if you were faced with a situation, um, where, the organisation you're working with or the team you're working with know that the, how satisfied the customer is with their service they provide, but they're not measuring it in any way. How would you, how would you approach that? Well, we've got to have surveys and we've got to do this. And

Bernie:

Nah.

Jimmy:

how would you approach that

Bernie:

So I'm, it's fair to say, not a fan of surveys and particularly MPS for the reason that we're going to. I, look at what people do. So for me, if I am super happy, I will recommend, I will refer. for me, the, the gold plated indication of satisfaction is the level of referral and recommendation, because I won't recommend or refer something unless I'm, I'm Really genuinely happy and confident in it. look at what you do on a personal level. I will buy again from someone or an organisation that I'm happy with. I will engage with them. I will go back to them and I will refer. Um, the other thing is language. Most conversations are recorded. We now have huge AI resources where you can, measure, tonality and sentiment, and you can also look for certain words as well. So look at what people are saying and how they're saying it, It may not work for every individual, someone with a strong accent, someone with an incredibly laid back vibe, but there'll be this enormous archive of live behavioural evidence of how someone's talking, if people swear, they're probably not that happy. Now, you might have one particular sweary person, but over a cohort, if you see an increase in sweariness, there's something going on. And there are certain words that people will use when they're not happy. And there are certain words that people will use when they are happy. So I would be really interested in looking at That kind of evidence rather than self reported because I'm relentlessly British if I've had shit service unless I'm really really angry I probably won't fill in a survey because I don't particularly if I think it's going to have a negative personal impact on the agent I dealt with because I don't want some poor sod in a call center becoming the lightning rod for a process failure So I often won't fill it in.

James:

fascinating conversation. So, in summary then, what would be your words of wisdom? What are the key messages you'd like, the people have listened to this to be thinking about

Bernie:

So the first thing is it's always harder. then it first appears to implement measures well. be brutal about what you choose to measure. Keep the list very, very short. Now, if you do that, you need to make sure that it, what you do choose really counts. So I would strongly encourage you to identify one or two of these. impossible, but important measures and try and implement them. and just finally, you know, particularly if you're looking at proxy indicators, Look around you. Watch what you're doing. Watch what's happening when the thing that you care about happens. we leak information about intent all the time. We're just not really trained to look for it. So if you're really interested in a particular event, unhappy customers, moments of truth, whatever it is, Really look at what's happening, really focus in at what's happening around it as well. And there could be some real gold bars in there that are measurable and that could be, you know, massively insightful.

Jimmy:

Absolutely. Love the love, the concept of gold bars, Bernie. And that really resonates with me in terms of my experience of, of looking at KPIs, but I think you've got to have the right mindset to approach it. So, that whole bit about, what are the things you're really trying to achieve and linking the KPIs into those, but also then having the, sense to discard a lot of stuff because in my experience, as soon as stuff gets written down and people start looking at it, it's, it's, it becomes a truth, and we find it very hard to Discard things. And then the three step process for trying to unlock that gold bar, I think is incredibly, uh, incredibly helpful.

James:

Lovely, and if somebody wants to learn more, how do they get hold of you?

Bernie:

Head to metameasurekpis. com there's plenty on the website. And if you can't find what you want, just contact me through the tab there. I'll point you in the right direction or jump on a call. Very happy to talk.

Jimmy:

And read three of your 21 books.

Bernie:

As many as you like, more the merrier.

James:

we will put all of that information on the website as well if anybody wants it So thank you very much for your time birdie. Very interesting

Bernie:

been a pleasure. Thank you for having me along. Cheers.

James:

Cheers now.

We cover a whole host of topics on this podcast from purpose to corporate jargon, but always focused on one thing, getting the job done. Well, easier said than done. So if you've got. Unhappy customers or employees, bosses or regulators breathing down your neck. If your backlogs are out of control and your costs are spiraling, and that big IT transformation project that you've been promised just keeps failing to deliver, we can help. If you need to improve your performance, your team's performance, or your organization's, get in touch at jimmy at jobdonewell. com or james at jobdonewell. com

People on this episode

Head Shot

James Lawther

Co-host
Head Shot

Jimmy Barber

Co-host
Head Shot

Bernie Smith

Guest