Measuring What Actually Predicts ERP Success

Bad ERP metrics are worse than no metrics at all.

Prosci's 2025 Unlocking ERP Implementations research found that organizations with poor metrics succeed 7% of the time. Organizations with no formal metrics at all succeed 26% of the time. Comprehensive metrics maturity delivers outcomes four times better than measuring poorly.

Most ERP dashboards are built from activity: hours trained, communications sent, milestones hit, and tickets closed. Those numbers are easy to generate, and they feel objective, which is exactly why they become the default. What they measure is whether the change management work was performed, not whether people are working differently.

In this episode, Prosci's Claire Guyot joins host Emma de-la-Haye to separate the metrics that drive decisions from the metrics that create noise. Drawing on years of work with ERP leaders across industries, Claire explains why training completion is the most misleading number on the dashboard, what five people metrics actually signal whether a rollout is on track, and how to tell a reporting metric from one that changes what leaders do next.

In this episode:
- Why activity metrics dominate ERP dashboards, and what they can and can't tell you
- Speed of adoption, ultimate utilization and proficiency as the human factors behind ERP ROI
- How poor measurement creates false confidence and hides real risk
- The five people metrics worth tracking, starting with readiness against the Prosci ADKAR® Model
- Reporting metrics vs. steering metrics, and the one question that tells them apart
- How to rebalance a dashboard partway through an implementation

Built to Change is Prosci's video and podcast series for the people who make change happen. Season 1 takes on ERP, with new episodes monthly. Subscribe to follow along.

New to Prosci? Be sure to follow us on LinkedIn or engage with our content further on our website. 

Transcript

Copy Transcript Copied

Speaker 1 0:00

Organizations with strong metrics are two and a half times more likely to exceed expectations on their ERP transformation. Organizations with bad metrics can actually do worse than organizations with no metrics at all. In this episode, Claire Guyot, an expert from Prosci Europe who has spent years working with transformational leaders on this measurement question, breaks down for us what to measure in ERP change and what to stop measuring. Plus the difference between metrics that report and metrics that steer. Tantalizing stuff. Let's jump straight in. Welcome, Claire. Welcome. My first question for you, if I might. Most ERP rollouts that both you and I have been involved in have dashboards. Most of these dashboards are full of activity metrics. Hours trained, training attended, communication sent, tickets opened. Why are those metrics so popular when they actually predict so little?

Speaker 1:25

Hi Emma, nice to be here with you. And really good to answer this uh this question. So I would say that activity metrics are popular in ERP dashboards because they are really easy to count, easy to report, and they feel also very objective. So organizations naturally track things like training completion, as you mentioned, communication sent, milestone achieved. But the challenge is that while these metrics are easy to measure, in fact, they are not always the best predictors of ERP success, as you slightly mentioned. In other words, they are popular because they are measurable, but not necessarily because they are meaningful. So there are a couple of reasons for that, based on also my uh experience with several customers. First, in fact, these metrics are readily available. So most systems uh within ERP can easily generate the reports on completed training, as you mentioned, project progress, you know, support tickets, number of tickets uh closed. So they are really quickly become the default dashboard. And this is what I've seen a lot of times. And second, they also create a sense of control because it's reassuring to say we have trained 2,000 users and we have completed 95% of the planned activities. But those numbers don't really tell us whether people are actually ready to work differently, whether the managers are supporting the change, or whether the users are able to perform the jobs, in fact, that they are they should do effectively in the new European environment. And this is where, in fact, I will not surprise you, but precise ADKAR model provides a useful lens. Instead of asking, did we deliver the activity? We should also ask, do people understand why the change is happening? Do they want to support it? You know, can they perform it successfully after GoLive? So that's a very different conversation. My practical recommendation on that is really simple. I would say keep activity metrics, but don't let them dominate the dashboard. We still need them. Project teams need them. But activity metrics whether tell you if the change management work was performed, but they don't really tell whether the change has happened concretely. So the one I would recommend using for the human factors of the return on investment for the ERP, these are the three ones we are really looking at. It's really the speed of adoption, how quickly people are getting up and running on the system, the utilization, so how people really actually use the ERP as intended, and the proficiency. So how effectively are they performing in the new system? Are they using it well? So that's very important to have in mind. And for me, the question leaders really need to answer is are we simply doing change management activities or our people adopting and using the ERP in a way that delivers business results?

Speaker 1 4:43

Thank you, Claire. So activity metrics have a place, and I think you described them wonderfully as change management indicators, but things like speed of adoption, ultimate utilization, proficiency, perhaps better indicators of change outcome, change successes. I love that. I think you articulated that really succinctly for us. Of those activity metrics, in your experience, which one often misleads leadership and sponsorship most frequently? Which one seduces them into thinking this is going really well? But perhaps it isn't.

Speaker 5:30

I would say, based on my experience, this is the training completion rate. Usually that is done right after the training, before go live. And if they see a huge number of people just attending the training, they think that people are ready for the go live. And they usually don't run any business readiness assessment or change readiness assessment to evaluate whether people have the knowledge to and the ability, desire, and knowledge, of course, to jump into the go-life. They just look at the training completion, which for me is really misleading because we need to look at, of course, the satisfaction, whether this helps them in completing their roles, if the training has really fitted really the role, the activities they are doing. So this is for me what I have observed as a metric which is always tracked, and this is of course important, but satisfaction, making sure that people get the most of the training so they feel ready to use the system is usually what is missing. And that's why for me this is one of the most misleading metrics in the ERP deployment.

Speaker 1 6:49

Thank you. You all have spotted a big smile on my face throughout that answer. Thinking back to my work with again lots of organizations and indeed having been client-side, as it were, delivering changes over the years, I think that's a trap I've fallen into once or twice as well. I also appreciate you just underscoring in your answer there, how the outcomes, as articulated in our ad car model, sure, desire to actually be trained, but then the knowledge, do I know how to do this and the ability? Am I proficiently able to use our new ERP capability in the way designed? Brought that through beautifully as well. Thank you. And I I tantalized with some data from our ProSci research, and I'm going to ask you to unpack that a little bit for us. So just a reminder. So the question for you: how does measuring badly actually become worse when not measuring at all?

Speaker 8:11

Yes, thanks, Emma. When I looked at this data in the research, I was surprised as well. So I can understand this may be surprising that organizations with no formal metrics outperforms those with poor metrics. But when you think of it, in fact, bad measurement really creates false confidence, misdirected effort, administrative burden without value. So this is what I have observed or what I can say. And in a certain way, it creates, it creates false accountability. So the team can say, we are measuring, you know, but if the metrics are incomplete, inconsistently tracked, or really not used to make decisions, that's what is important, they create the appearance of control without the discipline to drive the outcomes, as I mentioned. So that's one point. A second one I would like to highlight is that it also shifts attention to the wrong work. So if you have poor dashboards, um, that those dashboards often overemphasize the activity metrics. That's what we have just discussed. So training hours, communication sense, tickets closed, as already mentioned. And so teams will also be willing to optimize these activities instead of adoption, utilization, and proficiency. So that's also how it shifts attention. Another point is that it hides real risk in a certain way, because a dashboard can look green. This is what leaders can observe and they would like hearing or seeing. But in fact, users may still lack awareness, desire, knowledge, or ability, as you mentioned. So that really delays intervention because the leaders think that the rollout is really under control and that everything is healthier than it is. In fact, another point I mentioned, it really adds the administrative burden with the value. So people will spend time really collecting the data, reporting the data. That doesn't really improve decisions. That's the issue. And that's one point which we need to have in mind. And I would say finally, uh, it also weakens the decision making. So, in fact, strong metrics really help the teams decide. And that's really the ambition of the metrics we want to have in the ERP dashboards to decide what to adjust, where to do improvement. And the poor metrics don't do that. They may even crowd out informal signals like managers' feedback, resistance patterns, user behavior that would have surface problems earlier. So the practical takeaway, I would say, is don't just have metrics, but build metrics maturity. That means really metrics should be comprehensive, consistently tracked, and actively used to guide decisions.

Speaker 1 11:10

Claire, that was very compelling about the importance of meaningful metrics. I've got a slightly devilish question for you because we've all been there, and I think sometimes the war stories are very compelling. What's the worst decision you've seen recently based on based on a bad or a poor metric?

Speaker 11:34

Yeah. Very recently I was working with a customer who started to deploy the ERP in some of their pilot countries, and they had designed very generic training, and they wanted all the impacted populations to run through this very generic training package. Um, the attendance rate was high, and they had that in the dashboard following how many people attended. They passed the goal live and they realized when people were supposed to use the ERP system that they were not using properly because the training was not role-based. So they had in very urgently, I would say very quickly, to redesign specific training packages to address the different impacted populations, to do a specific training for the procurement people. And within the procurement people, even you know, only the buyers or the category managers and for the finance people, you know, maybe having dedicated training for the people doing treasury, doing finance control. So all of that has been designed after the fact and it was not thought before. And they realized really after GoLife, because they haven't followed, let's say, the ADKAR assessment, I would say, there was no people metrics that they had to do a lot of works afterwards and in a very panic situation, if I may say.

Speaker 1 13:01

Very compelling. So on a dashboard of call metrics, that would have flagged this green, training the app completed. Back to your earlier point about one of our people measures being proficiency. How well can our users, people impacted by the change, actually deploy the new capability? Very low. So we have to delay, spend lots more energy, time, resource cost, redesigning and retraining everyone. Very compelling or story, Claire. Thank you for sharing. Claire, if activity metrics don't predict success, what does or what do? What are the people and adoption measurements, metrics, indicators in your experience that would signal early that we're on track or we're not on track for success in our ERP?

Speaker 13:54

Thanks, Emma. As I mentioned, what predicts ERP success is not how many change activities have been completed, but it's whether people are actually moving towards adoption and effective use of the new system. So that's important because, as you know, in the precise ERP research, we have 92% of organizations that define ERP success metrics, but only 20% measure the people factors that drive decisions. And something I would like to add is there is also a further gap. We have 54% of implementation problems which stem from people factors, but only 34% of metrics track them. So when I worked on the ERP programs, they are, let's say, five people metrics I recommend tracking. And we talked a bit about that already. The first one is the ADKAR readiness. Are people aware of the change, willing to support, knowledgeable about it, about what to do, you know, able to perform in the new environment? I've actually seen a go-life postponed because readiness survey results showed people were simply not ready yet. So this helped a lot. A second metric is around adoption velocity. So how quickly are users starting to use the ERP after go life? Early adoption is often a strong indicator of long-term success, as you know. The third one, third metrics, is the engagement depth. So are people just logging in into the system? We've seen that it's easy to track, or are they really using the right transaction for their job role? You know, the right workflows, the end-to-end processes they are supposed to do and to work in their own job. The fourth one is user proficiency. We mentioned that already. Can they perform their work correctly, efficiently in the new system? And to me, this is far more meaningful than training completion, as we discussed, because it shows whether knowledge is becoming reliability. And finally, I would say the fifth one I recommend is around user satisfaction. Are users telling us that the ERP is helping them to do their jobs, or are they experiencing pain points that could slow down the adoption? So that's the five metrics I try to discuss, challenge the leaders to have inserted into their dashboard. And for me, the key is not to have dozens of metrics, but it's to have a small set of metrics that leaders actually use to make decisions. So ultimately, the best ERP dashboards I have seen for me should answer one simple question. Are people adopting and using the ERP in a way that delivers business results?

Speaker 1 16:56

I love that. We don't need baskets of 30, 40 indicators. We need five powerful, trackable metrics that give us confidence as leaders that we are on track for success or not, and therefore we have to course correct. Thank you. Most metrics that you and I will have been working with clients, Claire, are designed to report, to tell leadership, to tell sponsorship what's happening. But metrics that drive better outcomes we might categorize as metrics that steer. So for you, what's the difference? And how do you build metrics that steer the outcomes we're looking for?

Speaker 17:44

Yes, so as you mentioned, one of the biggest distinctions I make with the ERP leaders when I start working with them at the beginning of the program, you know, to define what success looks like, is really to insist on the difference between reporting metrics and steering metrics. And really the reporting metrics normally tells you what happened, as you mentioned, and I try to make them realize what it is. And the steering metrics really help you decide what to do next. And that's the big difference. For example, saying 95% of users completed the training is really a reporting metrics, as we discussed, but saying only half of our users, you know, complete critical transaction for their job without support is a steering metric because it immediately prompts to action. So what I also in the ERP research is that it suggests that the metrics maturity matters far more than the number of metrics you track. That's what I also mentioned a bit earlier. So to answer your question, Emma, how do I build with the leaders metrics that actually steer on their ERP program? First, I start with the decision, not the dashboard. So before defining a metrics, I'm challenging the leaders. And I said, if this metrics turns red, what action will you take? If nobody would do anything differently, then it's probably just a reporting metric. So that's something I raise awareness about and I challenge the leaders about. Second, what I recommend as well is to keep, as I mentioned, the list really small. And I generally recommend three to five critical metrics, the ones I've mentioned earlier, that the leaders can really realistically review and really act upon because we really want them to make actions on the on the outputs. The third, focus on the adoption and not just the delivery. So again, measure things like adoption velocity, active use of the critical modules of the ERP deployment, the proficiency people have, different impacted people have in their key transactions, you know, the ADKAR barrier points for the impacted population, collect the user feedback. These are really indicators for me that tells you whether the people are actually moving towards the future state or not moving towards the future state. So that's the third one I mentioned. There is a fourth one, which would be make the metrics specific and actionable. So instead of saying users are adopting the ERP, we need to define a target. And I challenge the leaders around that. So for example, we should say 90% of, let's say, the finance users can complete month ends closing independently within 30 days of Go Live. So very specific, you know, smart metrics in a way. And finally, every metric also needs an owner and a response plan. So someone should really be responsible for a metric, should know who needs to review it, you know, who needs to take the action and what is the thresholds to trigger the action. This is really important to have those in place. So the simple test I use with the leaders is a metric is a steering metric, let's say, if it answers the question, what should we do differently now? So if it only helps you report status upward, it may still be useful, yes, but it's unlikely to improve the ERP outcomes on its own.

Speaker 1 21:36

And if I am a leader listening, watching us today, who's partway through an implementation, and what you have just cautioned us about is resonating, and I'm reflecting that my dashboard contains a basket of reporting metrics in your View, can I retrofit part way through and adjust some of those, perhaps reducing the number to steering metrics?

Speaker 22:10

Yes. Uh, I mean, I would say it's never too late to act on metrics. Ideally, as we always recommend, we want to define success at the beginning of the ERP program, but it's quite challenging, even when we are we have meetings with leaders around the pro-side change triangle, the PCT, when we want to define the success metrics at the beginning. It takes time, it takes alignment. And for me, it's never too late to reassess the dashboard, to challenge the dashboard, and to maybe reduce, as you mentioned, look at what are the ones which doesn't allow us to take actions, which are just reporting and might not be that useful. And incorporate maybe one or two metrics which are human-centric. Maybe not the five ones because you will not have the time to assess all of them, but you select what would be the most relevant based on where you are at the deployment phase. So that's where I think the ADKAR potentially metrics would be one very powerful because it's really aligning with the ERP deployment milestones.

Speaker 1 23:22

Thank you. Thank you. Claire, you speak so eloquently and compellingly of the examples where you have worked with a range of organizations. Can you share with us an example of the right metric used, read at the right time that steered perhaps altered the course, the trajectory of an ERP deployment?

Speaker 23:52

I will even have two examples. Let's say, indeed, one of the recurring challenges I've seen across all ERP implementations is definitely measurement. And the issue is rarely a lack of KPIs. It's in fact often the opposite. We have tons, dozens of indicators that need to be followed, as we just discussed. And it becomes difficult to distinguish what is actually predicting success from what is simply reporting activities. So the two examples I would like to highlight are very different. The first one is around a large SAP deployment in the telecom industry. And we had each stream leader in charge of their own technical KPIs. So they had a lot of KPIs, tracking milestones, defects, inventory reduction, approval lead times, I name but a few. On the change management side, we were also monitoring training completion attendance, readiness assessment, and satisfaction score. So we had quite some human-centric measurement. Those metrics were useful, but looking back, I would say we were missing a bit more indicators that I would consider far more predictive today, like the ADKAR progress as such and the user proficiency, the adoption of the key SAP modules, and also the manager engagement. But what was interesting in this story is that the deployment was still really successful. And when I look back, I think that's because the change team paid a very close attention to one factor that wasn't always visible on the dashboard, but it was the sponsor engagement and the stakeholders' engagement. And they made sure the leaders were highly visible throughout the rollout. So they opened training sessions, they reinforced key messages, they listened to feedback, and they remained involved during the hyper care, for example. So there was this continuously engagement, gathering employee feedback, which was not properly tracked in the ERP dashboard as such. And that's for me an example of success, which was not tracked, but because adoption and sponsor engagement was there, they really paid attention to this adoption part, finally. So that's the first example. So even if they were not perfect, we were really paying attention to the real signals coming from the organization. The second example was more in the energy sector. And this is where I really saw the power of the right metrics used at the right moment, as you mentioned. In this case, the change management office had built a very focused people dashboard around four areas: stakeholder engagement, communications, learning, and adoption. And rather than tracking everything, they selected a small number of metrics that were aligned with the deployment phases for each of the four categories. Things like stakeholder engagement levels, ADKAR assessments, readiness survey before go live, and learning effectiveness measures. So that's what made this approach very effective. It was not the number of metrics, but it was the fact that they were used to make decisions. So that's the point that I wanted to mention. So the best metrics don't really tell you whether the project was successful, they really give you the opportunity to make it successful.

Speaker 1 27:36

Oh, I love that. And as early as possible. So that we can do the course correction, repeat what's working, drop what's not, put new ideas in. Absolutely. Our viewers today, imagine somebody now at the beginning of an ERP starting to build their measurement plan for an ERP rollout. What's one principle you would recommend they anchor on? And what's one mistake you would say avoid, avoid, avoid?

Speaker 28:10

So for the first part, I would say measure what will steer adoption decisions. We just discussed that, not what is easiest to report. As mentioned, and I would like also to emphasize again the process ERP research, it shows why this matters. In fact, people factors make up 55% of ERP implementation challenges, but receive only 34% of measurement attention. Meanwhile, process metrics get 52% of measurement focus, despite representing only 29% of challenges. So the measurement plan should deliberately rebalance toward the people side. Adoption, usage, proficiency, we mentioned that already. Readiness, stakeholder alignment, resistance signals. So a strong starting question for me, for the leaders, is if this metric changes, what decision will you make differently? This is how they need to define the right metrics. In other words, if the metrics does not push for action, then it should not be part of your steering dashboard. And for the avoid, let's say for the mistake part, I would say what we want to avoid is building a big dashboard with low-value metrics. So we want to keep it simple. We want to pick up three to five metrics, to tie each one to adoption and use, to define thresholds for action, to assign an owner, as I mentioned already, and to review them frequently, you know, during the steering committee, enough to adjust training, as you just mentioned, to adjust sponsorship actions, to adjust the communications to the right people, and to support and manage better resistance. So, in short, don't measure more, measure what you are willing to act on.

Speaker 1 30:08

Oh, thank you. Now, the part of this these podcasts where people who have been joining us regularly will know that I attempt to read my handwriting on the notes I've been taking throughout. My takeaways today, firstly, activity metrics might be easier, but they're mostly useless for predicting success. Secondly, that research point that gave me pause for thought when I first read it, and I think you as well, that bad metrics are worse than no metrics. That seven versus twenty-six percent figure we we both talked about. And thank you for clarifying because a bad metric can drive false confidence and therefore bad decision making. Thirdly, the metrics that actually predict ERP success are those people and adoption metrics, and they are the ones that are capable of giving us signals really early on that we are heading in the right direction or not. So that's the predictive element we talked about. Fourthly, the best metrics steer rather than simply enable us to report on activity. They change what leaders do and what leaders don't know. And finally, you've said this a good few times, and I love that your three to five, three to five, fewer, better metrics, more powerful, more useful, more predictive than a whole street of reported metrics that are particularly helping us. Final question, if I might just squeeze one in, Claire. If one of our viewers today is going to take just one thing away from what we'd be talking about today, what would be your one takeaway for them?

Speaker 32:21

I would say focus on the people-centric metrics. Adoption is the key.

Speaker 1 32:29

Adoption is the key. And hey, we have our beautiful ad car model that can help, is versatile, is simple, great utility, and can really help with that. Claire, thank you so very much for joining us today. I'm grateful for your time, your enthusiasm, your stories, and your expertise in what we need to measure to enable great ERP success. Thank you.

Speaker 32:57

Thanks a lot, Emma.

Speaker 1 32:58

I was really happy to join this podcast. Thank you for taking the time to listen to Built to Change, a podcast by Prosci. If you enjoyed this conversation, be sure to follow the show to stay up to date with our monthly releases.