Welcome to the Evidence-Based Management podcast – dedicated to those studying evidence based practice in the field of management.
This episode introduces the regular podcast contributors – those that have been writing, teaching, practicing and promoting evidence based management for the last 20 years. They are all authors of the online course and the book that accompanies it.
We hear about how they became interested in this practice, why they are passionate about it, and where we are today, some 20 years after the movement started.
If you are interested in the organisation ScienceForWork, mentioned by Rob Briner, you can find out more here: https://scienceforwork.com/
Host: Karen Plum
Find out more about the course here: https://cebma.org/resources-and-tools/course-modules/
00:00:00 Karen Plum
Welcome to this introductory episode of the Evidence-Based Management podcast. This episode will introduce you to the podcast regulars and to the subject of evidence-based management. How it first emerged, how it has developed, and where we are now in this discipline.
I'm Karen Plum, the host of the podcast. So who am I and what qualifies me to be hosting this podcast? Let's find out.
00:00:34 Karen Plum
So who am I? With a background originally in HR, I've been a management consultant for 23 years, working with lots of blue-chip organisations, charities, not for profits and local and national government. I've worked with organisations like the Royal Bank of Scotland, Prostate Cancer UK, Microsoft, Morgan Stanley, London & Partners and the UK’s Cabinet Office. So I have practical experience of implementing change and how decisions are made in organisations.
I've been involved in evidence-based practice for the last seven years, taking the results of the best scientific evidence on a topic and making it accessible for others to use.
But I've realized that there's much more to learn about evidence-based practice, so I'm doing the online course to improve my skills and understanding. So I'm on the same journey as you and I hope these podcasts will answer some of your questions.
Each episode focuses on one module of the course, and I talk to the authors about aspects I've found interesting or difficult; we debate the practical applications; and ask questions that I and other students have raised. I hope you find the episodes interesting and helpful on your evidence-based management journey.
Let me now introduce you to some of the podcast regulars. First up, we have Eric Barends, Managing Director of the Center for Evidence-Based Management, one of the authors of the course and the book that accompanies it.
00:02:02 Eric Barends
My name is Eric Barends, I'm the managing director of the Center for Evidence-Based Management. I consider myself a manager, not so much an academic. I do have my PhD and I teach a lot at universities and I publish academic stuff, but I consider myself first and foremost, a manager.
That's because I worked mostly in healthcare management for a largish part of my career, and that's also where I came across the idea of evidence-based - in that case it was evidence-based medicine. I was in charge of a large-scale change project where three hospitals had to work closely together, but it involved a lot of changes for the physicians and other nurses and staff. And the three directors of the hospital had to report to me.
It was basically the same change project in all these three hospitals, but every director had a strong opinion on how the change should be managed and they were all different. So I thought that can't be right. I mean how is it possible that we have three very experienced managers / directors here? - they all have different ideas of how the change should be managed. And in medicine there are some differences between doctors of course, but in general there should be some consensus - what would be the best treatment for a patient. So I discussed it with some medical specialists and they said, well, we have this thing called evidence-based medicine and I said, wow, that sounds great. What is that?
Well so it works like this. If you have three different ideas or three different opinions based on whatever - on how to treat patients, we will then have a look at the evidence from research in general, but also from experience, to see which opinion is supported by the strongest evidence and then we go for that judgment.
And I thought that's an interesting idea. So I started asking the directors, so what's actually the evidence base for your opinion, your judgment that the change should be managed in this way?
Actually, the only answer I got was, well, that's actually how I learned doing this based on experience, but this is how I feel it should be done. Or I read this popular magazine and there was something about change or oh this is how it was taught actually at my university. And to be honest you could already figure out where the directors were trained or had their education. You could say, oh, you went to the Business School in Rotterdam; and oh, you are clearly someone that was trained by a business school in London.
Because they all had their strong beliefs and approaches and models. So it depends on where you were trained, how you would manage to change, and that was a shock for me. And that's when I started digging into evidence-based medicine and at a certain point, I realized that a lot of the insights and principles are also applicable to management. So that's how I go into evidence-based management.
00:05:20 Karen Plum
Next up is Denise Rousseau. Another of the authors of the course and. The book. Denise is HJ Heinz University professor at Carnegie Mellon University.
00:05:40 Denise Rousseau
I'm Denise Rousseau and I'm an industrial psychologist by training and I've been one for, oh, 45 years and I think the important thing about me, (context of evidence-based management) is I grew up in my profession believing that scientific evidence was useful for practice and that was why we studied the employment relationship, change, basis of work commitment, job commitment and that of course it would be useful in practice.
But as a further back story, I'm a blue-collar kid whose father hated his job and part of the reason I chose my field was because I thought it might help make the work experience and the way organizations function better for people.
It's true when I made that decision it was the early 1970s and it was a Daydream Believer’s age, but I haven't lost the dream that we can provide useful insights to practice and that practice at all levels will benefit. But it wasn't until I became the President of the Academy of Management, which is a large association of management scholars and many Business School professors or members. And I started hearing from the membership complaints!
Their complaints were about the quality of our journals. And the most frequent comment I got from people who were in many thousand person organizations was, journals aren't useful to me because I look at them and there's nothing I can use for teaching. Because most management faculty, of course, are teachers, primarily. And I thought to myself first I felt bad, Oh my God. It's just not useful for teaching what. Does that mean? And then when I reflected more deeply, I realized. But could this mean they're not updating what they teach based on what we learn in research?
And that I mean it was a flash of insight, because when I consider what we know about textbooks in management and organizational behavior and in HR, they're still covering material that was falsified in my youth. When I was a doctoral student, we knew Hertzberg was not supported by evidence; we knew Maslow was not empirically based; and yet almost every textbook in my field has a little trip down memory lane through Maslow and Hertzberg.
And that worried me a lot. I realized that this was an opportunity to ask the question. Do we have something called evidence-based management? And the basic answer to that was not yet. But here are some ways we can proceed. And part of the ways we could proceed is by teaching people more about what the evidence says that's relevant for practice because we weren't covering it and also building skills so that people who are practitioners could continue to update as research changes and also as the questions that they have issues with surface and weren't covered in their original training.
00:08:50 Karen Plum
Finally, I'd like you to meet Rob Briner, who also explains how he became interested in evidence-based practice.
00:08:57 Rob Briner
My name is Rob Briner and I'm currently Professor of Organizational Psychology at Queen Mary, University of London and I was one of the co-founders around 10 years ago of the Center for Evidence-Based Management.
My disciplinary background, I suppose, is organizational psychology, IO psychology. But I’m also interested in HRM and management more generally, and other areas of psychology.
Actually for me I can identify quite precisely when I first got interested in it. When I was doing my PhD, which was probably around 30 years ago in a place in Sheffield University, now called the Institute of Work Psychology. I was doing a PhD there about work and well-being.
In the institute, there were also some clinical psychologists and I had a colleague and friend called Shirley Reynolds and I used to sometimes talk to her, work with her, and she'd also write stuff about work psychology like I was writing as well.
And talking to her about my frustrations about the way in which scientific evidence just didn't seem to be used much by academics. And also the way academics themselves didn't use science and research very well and she said should you could be interested in this thing, this new thing called evidence-based medicine.
And so I bought a little Sackett book and I looked at it and I read the bits I could understand of it and I talked to her and other colleagues and I actually thought, yeah, this makes lots of sense. Evidence-based practice is a way of trying to make more informed decisions using scientific and other evidence, but it's also quite a good way of thinking about how academics themselves also use evidence. And so yes, I've been into it for quite a long time.
00:10:31 Karen Plum
So now you've met the regular contributors and found out how they first became interested in evidence-based management. So now let's hear about the development of the online course and the book that accompanies it. How did that first come about?
00:10:44 Eric Barends
So we started out with the book, but after a few chapters we already started building the online learning modules so some chapters in the book came after the development of the modules and some of the modules came after. It was very interactive and the book is text with some examples, but the modules take it one step further. It's basically the same text, with more examples. But then you have the “learn by doing” and “did I get this” exercises and that makes it completely different.
The book is a reference guide that includes all tables and checklists and rating systems, so that should be on your desk as a manager or on your bookshelf so you can get it when you need it. The modules are to train the skills, to develop your skills, so that's the difference between the two.
00:11:41 Karen Plum
I was also keen to know what our experts think is the most challenging aspect of practicing evidence-based management. Here's Rob.
00:11:50 Rob Briner
I think the most challenging aspects of it is that it does mean you have to stop, think and probably often question or challenge your assumptions and those of your colleagues. And that is the hardest thing. I don't think actually the hardest thing is the technical aspects of it. I think everyone probably knows and can certainly learn how to gather evidence from multiple sources, to think about its quality, to try and apply it in making more informed decisions.
I think that's not the hard bit. I think the hard bit is because it (depends a bit where you work) but I think most of us, most organizations, many professions, I'm generalizing, I don’t really know, it's difficult to ask questions about why, why are we doing this?
Why are we doing it this way? Do we understand the problem, opportunities? Why are we leaping ahead with a solution or an intervention, we're not even quite sure what's going on yet? Can we just spend a bit more time on this? And that, that kind of approach is in general my sense is not very popular and the people that just get on with stuff and they seen to be getting on with stuff and seen to be productive in particular ways, they will just kind of rule.
Because the hardest thing is that sort of slowing down and asking questions and wondering. Ask a question about why we're doing things, trying to answer those questions. I think that's to me the hardest thing about it.
00:13:10 Karen Plum
I have to agree that asking questions is way more challenging than it seems. We'll go into this in more detail in future episodes, but when we ask questions in organisations, there are all sorts of political, power and status aspects which can be at stake.
Many people aren't used to being questioned, and it takes much less energy to just carry on doing what we've always done, or to follow the highest paid person's opinion - charmingly referred to as the HIPPO.
00:13:39 Eric Barends
I would argue that it's the “that's the way we do things here”, kind of mentality in organizations. In module 15, the final module it's all about building evidence-based capacity in your organization. It starts with developing your own evidence-based skills - that's what you're doing right now, by doing these modules.
But then you get back in your organization and then you have to apply this. Probably many of your colleagues, but certainly quite a lot of your executives or the board, are maybe not familiar with the whole idea of evidence-based management and they feel they make brilliant decisions and you now are trained and have learned, are able to see that actually, maybe not so much.
And then of course it all starts. So I think that is the biggest challenge. I think we are capable of training individuals and make them more evidence-based. But making a whole company or an organization more evidence-based, that is the biggest challenge.
00:14:45 Karen Plum
Given that there are many challenges in terms of implementing an evidence-based management approach, I wondered if our experts, who've been plugging away at this for the last 20 years, are still passionate about educating people in this field.
00:14:59 Denise Rousseau
I think I am just beginning after almost 20 years of working in evidence-based management I’m starting to understand what it is. I'm also starting to understand many of the reasons why it's hard for people, and I don't just mean people with MBA's or people without high school educations, it's hard for everybody. Because the level of systematic thinking is not what we as humans are wired to do, so we shift between automatic modes and more reflective modes and evidence-based practice is largely a reflective mode of approaching problems, though to make it succeed I think we have to make some of its principles more automatic.
00:15:48 Rob Briner
Yes as I say I've been doing for 20 years and I've gone up and down! I wax and wane. At the moment, yes, I am, but there's been periods when I've just, not given up, but I haven't felt very enthusiastic about it. Mostly because it seems, it feels quite hard to get a sense of progress.
Is anyone listening? Is it making a difference? Are managers making better, informed decisions as a result of any of this. I don't really think they are very much, but I guess I've just decided it's important to keep banging away at it - even if it's not always clear if it's having much effect.
I feel it's important to do it yes and not just for practitioners, but I think also I would say from my academic colleagues as well, who typically again are not that interested in or knowledgeable about evidence-based practice to say, you know, maybe you have a responsibility as an academic, as a researcher, to try and do more to make sure not just your research, but research produced by all of us in academia, gets more use in practice.
00:16:47 Eric Barends
What I think is the most interesting or for me the most inspiring, energizing part of evidence-based management is when I'm with managers in real life situations. A lot of people have miserable jobs because of bad decisions made by managers and leaders.
So I think we underestimate the impact of management and leadership in organizations. We dedicate a lot of time to work, so being in an environment where decisions are made in a more evidence-based, in a more rational, sensible way really will improve the life of so many people, not only the employees but also their family, their loved ones.
The impact goes beyond the individual employee, so it affects whole households, families, neighborhoods, etc. So I'm therefore very passionate about evidence-based management because I think we can do a better job and we must do a better job.
00:17:50 Karen Plum
I found this was a common thread. The desire to make things better for people in organisations, to try to avoid the actions and decisions made by managers having a huge detrimental impact on the people in their teams and organisations. As a change manager, I've often heard that people grow weary of change because they don't think it is implemented well.
I have a strong hunch that this may, at least in part, be because we are trying to fix the wrong problems, but there's no doubt that poor management wears people down. From the selection of managers (those who were good at their last job are promoted into management, as it's the only way to reward them); through their training or lack of it (how often have you read that people leave their jobs because of their relationship with their manager); through to the poor decisions they make, resulting in rework, duplication of effort, poor communication, or just the application of energy to the wrong activity?
I find it inspiring that Denise, Rob and Eric have dedicated a good part of their careers to this practice and are still largely passionate about evidence-based management, even if at times their passion does get tested! I guess we all go through those phases in our careers.
The other aspect I found valuable was the notion of kindness and meeting people where they are. It takes time to move away from established beliefs. Providing people with information or evidence isn't going to magically transform their view.
Denise's approach is to just talk through the basis of people's beliefs and to respectfully ease them towards that self-discovery.
00:19:30 Denise Rousseau
People will - most cases I will say 9 times out of 10 – soften their position. Then what we're really asking here is that people be more reflective on what are alternative explanations and possibilities. I mean the truth or the proof is in the action, in its results.
But the notion that you cannot act on an alternative you don't consider, but that any consideration of a different way of thinking about a problem or issue can change the decision made and the outcome attained. And that's what we're going for.
My bumper sticker would say the perfect is the enemy of the good. We're looking for the good.
00:20:12 Karen Plum
I think my bumper sticker would say you can't act on an alternative you didn't consider because I just love that.
Finally, I wanted to ask our experts where we are now. Denise gave a presidential address to the Academy of Management back in 2005, where she spoke about the need to close the research practice gap. So given that 16 years have now passed, I wondered whether she thought that gap was closing?
00:20:38 Denise Rousseau
Yes, there is progress because science, organizational research in particular, has moved to being periodically more cumulative - where there are summaries available of what the evidence says, so that if somebody writes a textbook on collective intelligence, or a textbook on conflict management or teamwork, they have a chapter on these - they don't have to cite all these separate studies that say different things. They can go to a meta-analysis or a systematic review and know where the science is clear, so it's not as fuzzy and mish mushy as it once was.
The second part is, the somewhat greater awareness - this is still on the pushing evidence into practice, that part. There is more attention in the educated professional management community to - what's the scientific evidence on this issue? That's an idea that's on the radar. That was not on the radar 16 years ago.
Is it on the radar enough? Not to my liking! But it's been taken up and so you'll find in executive classes people will say, but what's the evidence for that? So we see that kind of that desire to have more content, and indeed students now will challenge their professors when the professor seems to be long on experiential stories and not long on well, what's the principle in this marketing process or whatnot?
I think one of the things that I have learned is that scientific evidence isn't answers. It doesn't give us the procedural knowledge or the solution set that could be used. We don't have science on all the different kinds of issues a practitioner faces because that kind of problem has not been studied.
The research practice gap is, in part the absence of practitioner-oriented evidence, which I think is absolutely critical. Do I expect academics to provide that? Probably not, but I do expect practitioner scholars and managers who can think critically to be able to generate their own local evidence. One of the best kinds of local evidence a manager can generate is to do a pre and post study.
We put this program in, we measure before, we measure after, we try to understand why did we get what we did - so much changed and so much didn't? I think that that kind of reflection after gathering kind of controlled information about your actions, is very important to learning because we're missing the procedural knowledge of how to make something work well.
00:23:37 Karen Plum
Reflecting that many academics have little interest in practice, Rob explained that this can leave a vacuum for those that need more evidence upon which to base their decisions, if they struggle to understand how to make use of the research that's been done.
00:23:51 Rob Briner
So on the one hand, yeah, I think there’s a bit more awareness and maybe we should try and make this research more relevant and accessible. On the other hand, there's also a knowledge that it doesn't really matter very much. I wouldn't say it's particularly supported.
I wouldn't say it's particularly incentivized either, and I think my favorite example of this is the ScienceForWork people? I don’t know if you know, but what ScienceForWork does is it tries to summarize things like meta-analysis and systematic reviews in management and organizational psychology and HRM, and tries to sort of translate them in a way that's understandable and usable for practitioners.
They have graphics, they try and do quality control on it. They make sure that the way they describe their findings is accurate given, for example, what's been found in the meta analysis.
And the question I always ask is - you know it's quite interesting. Who did that? Who provided that service or still tries to provide that service? Is it a bunch of academics? No. It is it a publisher? No, it's not. Is it universities? No, it is not. Is it a professional body? No, it's not.
It was a group of volunteer ex-MSC students. And to me that almost tells you all you need to know.
00:25:08 Karen Plum
And from Eric's perspective.
00:25:10 Eric Barends
I notice that more and more universities are teaching evidence-based management, more business schools and the term evidence-based is being used more frequently also within organizations, but also within society at large. So it's not us fighting the war, it is a whole range of domains and disciplines fighting this war.
So the medicine people are into this. Evidence-based policymaking, policing. All the areas that's considered a true profession are now slowly moving into the domain of evidence-based decision making and that helps.
And we're having this conversation in the middle of the COVID pandemic, and you notice that the understanding of evidence or the idea of evidence, strong evidence, weak evidence is adopted in our daily language and more and more people use the term rather than proof, by the way, which is a good thing.
So I think it's moving, it's changing for the good but of course, when you walk into an organization and have a look at how decisions are being made, you probably will be shocked. And a lot of listeners will say well, you know maybe you should visit my organization! What’s happening there is completely irrational and anti-evidence. Yeah probably, probably, but as a whole I think we're making some progress.
00:26:42 Karen Plum
Clearly we are all on a journey, and as with most things that are worth doing, it won't necessarily be easy. There's so much to know about practicing evidence-based management. The course will teach us a lot, and we'll gain deeper insight and hone our skills when we practice. And as we share our expertise through the way we talk with others, the movement will extend further still.
That's all for this introductory episode. If you've enjoyed it, you can follow or like the show wherever you get your podcasts. For now, I'll leave you with this final thought from Denise Rousseau.
00:27:20 Denise Rousseau
Most important thing about taking an evidence approach is pausing and reflecting. Second most important thing is then in that pause, beginning to ask questions. What's the problem I'm trying to solve? What assumptions do I have about it? What's the evidence for that?