HealthBiz with David E. Williams

Getting the most out of expert interviews. A conversation with Karen Donovan

May 13, 2021 David Williams Season 1 Episode 34
HealthBiz with David E. Williams
Getting the most out of expert interviews. A conversation with Karen Donovan
Show Notes Transcript Chapter Markers

Investors and management consultants often use expert networks like GLG and Guidepoint to arrange interviews with subject matter experts. These interviews are pricey so it's important to get the most out of them. My colleague, Karen Donovan and I offer five tips to help you succeed.

Host David E. Williams is president of healthcare strategy consulting firm Health Business Group. Produced by Dafna Williams.

Investors and management consultants often use expert network companies like GLG and Guidepoint and some of their smaller competitors like Zintro and Techspert to identify and conduct. Interviews with experts in specific industries. These interviews can be very valuable, but it all depends on how well they're conducted and leveraged. In this episode of the Health Biz podcast. I'm bringing on my longtime colleague, Karen Donovan from Health Business Group, and we're going to provide five tips for how to get the most out of expert interviews and the expert network companies that arrange them. I'm your host, David Williams, President of Health Business Group. I hope you find these tips to be valuable. If your company needs support on healthcare strategy consulting, including interviews with experts, please contact me. dwilliams@healthbusinessgroup.com.

David Williams:

Well, Karen, it's nice to be speaking with you today. We're going to share some of, I won't say our secrets, but some of our hard won experience in the area of conducting expert interviews. Yes.

Karen Donovan:

As long as there are no skeletons in the closet.

David Williams:

Well, if they're in the closet, we'll let them, we'll let them remain there. So let's start with tip number one. tip number one is to make the expert network company do their jobs. Sounds like something Bill Belichick would say. What are we talking about here? So we usually, we have GLG, Zintro, Techspert. There's a few different ones that we use and, they've done their job. They've assembled their network of experts and they have somebody. So what do you mean, have them do their job?

Karen Donovan:

The interviews, cost a significant sum tosource, to anyone actually seeking one of these networks to help them. the cost can be anywhere from 750 to$2,000. They may get a third to a half of that. And so knowing that the networks are making a good amount of money, they expect to do some work. So we really want to make sure that they are meeting our needs, that they are understanding the requests and able to really work with us, to find the right folks to meet our needs.

David Williams:

That makes sense. And some of the things that we do there is giving them some specifics. We don't just say, Hey, we're interested in learning about, subject X, we'll say here's the job titles we're looking for. Here's some of the specific companies, and here's some topics that we want them to have knowledge of. So we would say, for example, if we're talking about, outsourced services in software and biostatistics and data management. And that's the one that we worked on recently and give them some titles and vendors that we're interested in. And some topics that we plan to cover.

Karen Donovan:

Exactly It was interesting because even when you give the titles and the kinds of companies, and you might say VP level at a large company, or at a smaller company, you might want more of a program manager and you give even these generalities, that's not necessarily a home run and a direct hit. We need to ask are you familiar with certain vendors actually giving some specific names and telling them the discussion topics that we hope to go through. Making sure that they have familiarity and these are things that they can build into a screener, both in searching their database of contacts, as well as as they do outreach to identify other folks to do interviews.

David Williams:

Karen, one of the things that you've often done is, using that screener to find that kind of information you want, but also using the screener itself a data gathering tool sort of as a mini survey.

Karen Donovan:

It is well, just two things. One is you might, they may screen. three, times the number of people that we actually interview, number one. First of all, it's always interesting to learn what people see about market trends, how their use of outsourcing, for example, might change over the next two or three years or what their mix is of internal versus external, or, you know what, there are different staffing models that they might use. Those are all examples of questions we might build into a screener and even just getting the answers to that sheds some insight in terms of key trends may want to again, incorporate and refine the discussion guide based on that. It also helps qualify that the person is at least knowledgeable and represents a viewpoint that could be particularly useful for the project.

David Williams:

A lot of these networks have, big networks of experts that they can go to. And they can usually rip off a few names, pretty quickly. We've also seen is that, the squeaky wheel gets the grease and they don't necessarily make their biggest effort the first time around if they can satisfy you with the first things there. So I think asking for frequent updates is useful and getting them to go beyond that easy press the button. What have you found some of your specific things that you do

Karen Donovan:

Sure.

David Williams:

ways? Of course.

Karen Donovan:

Absolutely. No. In some cases we're ready to go and need interviews tomorrow. And in other cases you might need, you're fine. If the interview started in a week or two from now, just depends on the time urgency of the project. I commonly will ask for daily updates. Not only does it make makes sure that we're on track because they're working hard. They have a team assigned to actually to do the recruiting, you want to make sure that the candidates that they're coming up with are appropriate and you could actually have more real-time feedback to make sure they're going down the right path, and you can also know early on whether their existing databases will be sufficient, or if they need to do what they call custom sourcing and actually doing outreach and trying to reach out to new people, not in their network to participate in a particular project need.

David Williams:

You know GLG has been the inventor of the industry and the big boy here. There's other ones that we use. And sometimes you figure out, use this one or use that one. But on the other hand, sometimes you use more than one at a time.

Karen Donovan:

Yeah, this really surprised me at first. This is a couple of years ago, but they expect that you might be using multiple networks at the same time. Where I always get a little concern. I have to say, even now I wouldn't necessarily want one contact to be contacted by five different firms. I mean, I think that's probably a little overkill may not actually happen, but with that in mind they do have different networks and they don't seem to mind. And so what I might do is give them different geographies or give them a different title of person. They tend to be very hungry, especially some of the smaller ones and they might have the volume of a particular job title and a hunger to make it happen. So I wouldn't it hold back from using multiple vendors on one project.

David Williams:

The other thing I think that we've seen is that they don't necessarily all have the same people in the networks because they use different approaches. Some have a preexisting network, others purport to use AI or others who you know, reaching out to people on a customized basis. And there's different geographies, especially US versus Europe versus Asia. So that has been a surprisingly productive approach. I would say.

Karen Donovan:

Yes. I agree.

David Williams:

tip number two is it's almost like the Scouts, be prepared, prepare yourself for the interview. And I think it's like anything in life, you're going to get out of something, what you put into it. That time that you have with the expert, it's usually one hour unless, you are going to be paying for more and you should really use it for things that you could only do in that hour. So you want to ask them about things that only they know. You don't want to waste time on things that you could, look up or see on the website. Too often. I do see people when they're doing the expert interview, take the lazy way and, ask them things that you could find out with a Google search.

Karen Donovan:

The other way of saying it is what we probably tell our kids when they were younger, do your homework. Show up and you you bring in your A game, all the different little slogans. But yes, I think the more you put into it, the more you can get out of it, we're most commonly researching the companies beforehand, knowing. what products, what geographies, what operations might be, how big the company is just so that you can ask smart questions and not ask the questions, like you said, that you could have found out otherwise. It also, certainly can explain certain practices of, again, whether it's anything from the use of outsourcing to their overall volume of needs,

David Williams:

LinkedIn as usual is your friend here, can see that person's background. So you might hear about what company they're working for now, but you can also see, where have they worked in the past? What has their level of experience been and how recent is the information that they're likely to have? Sometimes you'll have somebody answer a screener one way that may conflict with what it says on LinkedIn. That could be helpful to, to double-check at a time.

Karen Donovan:

No. Well, it's always good to know. First of all, do you have mutual connections? Always a nice way, first of all, to break the ice as well, but also give some credibility and makes them connect different data points. You might want to know where else they worked. Especially you might say, all right, I understand you work at company A and you used to work a company B. How did company B do it similarly or differently than company A and be able to not just get the data for one company, but rather compare and contrast other companies in an industry, or even talk about the industry as a whole and have different practices, may vary across company type. And then, depending on how much information is possibly available, may ask them what their scope of responsibilities are, whether it's global or country specific or region. Whether it's business unit or corporate, this can really help guide the conversation. In terms of, if you're asking about spending for example. Or volume of outsourcing is another example.

David Williams:

Many people who are true experts will have presented at conferences, written articles, quoted professionally, and it makes sense to see what they've said in the past. It's reasonable to use that as a starting point that you can then use in the interview to probe further, or they may give you some ideas that you hadn't thought about before. A lot of this also then goes into preparing a good interview guide, and I know you're a master of that, Karen, to putting together the the interview guide. Any tips there for our listeners?.

Karen Donovan:

Sure. I mean, there, there are several, but some, probably the come jump jumps in my mind is we always like to do what we call the informed type, hypothesis driven guide. So we do our homework. We understand what experts are saying trends are and or what we anticipate the trends to be. And we use a combination of everything from quantitative as well as non quantitative types of questions, structured and unstructured, but we want to make sure that we're able to compare across multiple interviews. So those are particularly the times where it helps to have some standardized questions such as a willingness to recommend, is probably a common one, or level of satisfaction or. percent of again, I'll use of outsourcing as another example. So there are opportunities to not be too structured though, because you really want to let them talk and make sure that they're able to share their views of the industry and other intangibles. And one last one that I can

David Williams:

okay

Karen Donovan:

of is you never know exactly how verbose someone's going to be. And sometimes you're under the gun to keep them to a certain timeframe. Don't be concerned about asking them to, okay. You know, in the interest of time, I'd like to move on to the next question. You have to manage the time appropriately, but you also, at the same time, want to have extra questions in your back pocket. Perhaps if they're less talkative or it gets to the point quickly. You want to make sure that you have some questions in your back pocket so that you're making the fullest use of your time.

David Williams:

Absolutely. The third tip Karen, is to start the interview off on the right foot. These are sort of like funny little relationships cause you're paying big money for it. It's going to last an hour.

Karen Donovan:

It's like speed dating.

David Williams:

This is a little bit of a stilted relationship, because it's been, pre-screened and you're paying big money and it lasts an hour. Nonetheless you do want to think about it right? One way to do that, frankly, is to have somebody who's knowledgeable to conduct the interview, knowledgeable but open minded, somebody like you, Karen, you want to have these these interviews where you really get the most out of it and having somebody who's done their homework from tip number two is a good start.

Karen Donovan:

Now these interviews in some cases are like speed dating where you're really trying to get as much as you can out of an interview. That probably sounds a little risque but no, basically we want to, in many cases, an investor may want to do the interviews themselves. I feel like they need to check the box oftentimes will have, after the fact people will say: yes, we did the interviews because we know we needed to do the interviews, but in reality, they know that we could gotten more out of the interview simply because we have the subject matter expertise and have done the homework so we can get to the heart of the matter much more quickly and get more out of the interview than they might otherwise themselves.

David Williams:

Sometimes it's helpful to have a second person on the line, not necessarily to do, you know, as much of the interviewing. Sometimes you wanted to ask a probing or a follow-up question. Although other times that will take them off on the wrong. path, but definitely useful for the debrief afterwards. Cause you may hear something differently, you two may hear things differently and being able to reconcile that can be helpful. There's some people that like to do that instead of doing that, they like to record the interviews. That's never been something that we generally like to do, first of all, you'd want to make sure to ask for permission if you're doing it, but also recording, doesn't tell you everything. And I think better to better to rely on the actual live interaction. One of the things also in terms of getting off on the right foot is. It's okay to have a little bit of social chitchat. I think, you want to do a little bit for trust building, you don't necessarily need to go overboard. You're not necessarily looking for a new friend. There's less expensive and more appropriate ways to do that.

Karen Donovan:

I would just add one of the points just to elaborate for the audiences is people don't like to be recorded and they're much less likely to be as forthcoming. And so one of the benefits of having another person listen in is if someone may maybe a little more loose liped they may be able to allude to certain things that they might not otherwise if they are being recorded. But it's really great to have a second person online to be listening for those nuances and really picking up things that can be particularly salient.

David Williams:

So it's good to give a bit of context to the interview and provide some structure upfront and how you want the conversation to flow. If you don't do that, you know, people are experts and they can usually go and talk about all sorts of things that are interesting to them. That aren't necessarily interesting to us. Now, the context is good because it helps to guide that expert. It also establishes some credibility on your part, so they understand they're speaking to somebody who's already knowledgeable also, laying out your expected timeline or agenda for the interview. And let them know upfront, don't be insulted if we need to interrupt, and it's better to say that I'll say at the beginning, rather than cutting them off, at point at minute 35 and then wondering what's going on and feeling a little bit weird about it.

Karen Donovan:

The flow of the conversation goes much more smoothly if they know what to expect, just, you're saying these are the five or topics and how you want the interview to the, the content to cover in that discussion. And it helps them. too. They want to be, they want to please, they, they are being paid nicely and they want to make sure you're getting something out of it and they get referred or recommended for future projects as well.

David Williams:

I think a lot of times interviewers are not that well-prepared. And so the expert sort of feels the need to fill the space. And I'll say, based on our earlier tips, We go to a lot of trouble to be very well prepared. We know what we want to do with the interview. Now there's another piece that's like speed dating too, which is that you do want to figure out, they don't, despite all the good screening, it's not always a fit. If you can figure that out in the first 10 minutes, it's not a fit, hang up on them.

Karen Donovan:

Yes, no perfect example. Well, as we thought we had a great screener. We wanted to speak to CIOs and large pharma companies and talking about clinical trial technology. Of course, and we asked a couple of other questions in the screener and it just seemed okay. We know this was going to be a great person. Turned out in the first 10 minutes, we realized they were not familiar with at least three of the key technologies within that broader umbrella. This gets to a later point about definition of terms and what was included. But I would say if you do get in that situation, and you realize this person is not going to be helpful for these particular topics we want to go through, do cut them off. And the networks are usually very happy, to give a credit back for a bad match and happy to incorporate it. But it is maybe a bigger signal that you may need to incorporate and think more about adding additional questions to the screener.

David Williams:

So the final part here is something that would be true in the courtroom as well. I don't know if this is true for speed dating. Don't lead the witness.

Karen Donovan:

Yeah. I, my, my mind, the mental image there. So yes, clearly experts, there's always a confirmation bias. They want to tell you what they think you want to hear. So be careful about not just saying, we think this, or, or you may even suggest I might even suggest you present two opinions on diametrically opposite, some people would say:'the market's increasing', but other people would say'the market is decreasing' and this is why, what do you think? It, rather than just saying, okay, tell me, do you think the market's going to increase? It's just how you ask the question and making sure that you're not really leading the witness and do it, do a combination of unprompted answers and prompted choices as well.

David Williams:

Great. So tip number four is to get the most out of the expert. So now remember if you're up to this tip, you've already made the expert network company do their work. done good screening, you've prepared and you've kicked things off right with the experts. So now you're in the heart of the conversation and you want to make it work. And I think, we talked before about being well-prepared for the interview, having a good framework driving the conversation. But on the other hand,

Karen Donovan:

well, I would say how many times have you had a market research person call you on the phone? And they're reading off of a script and saying, multiple choice questions, A, B, C, or D, and then they, move onto the next one and they're going down their list. And, there's so much richness that you could be missing out on. So obviously again, combining open-ended and closed ended questions, let them talk, let them give their opinions. they have a wealth of knowledge.'That's why we've chosen them to be interviewed. So otherwise, if you don't give them that opportunity, you're going to miss out on some really valuable nuggets of information.

David Williams:

Sometimes there are some nuanced elements to the questions we're asking about a new market about, a company that has a strange past or position in the market. And you ask a question and they'll the expert may say they're a particularly good company or they, they were, people used to say good things about them and they may just leave it at that. Then you want to ask a followup question and part of it, sometimes you're in a conversation when you're the expert, or even as you, as we are as a consultant. And someone asks a question you're sort of supposed to know about it, but here, you're talking to an expert, you're paying for them. So if you don't understand something or if there's something more, ask that follow-up question, especially if it's something surprising or if they answered the question differently or they seem to wonder why are you asking this question?

Karen Donovan:

One good example of that is, oftentimes they'll say, okay, we've been doing it this way for X number of years. Yeah, maybe you might ask. Okay. Well, how did you do it before? What made the change? And sometimes it's something as innocuous as, okay, there's a change of management or there could be something really disruptive in the marketplace or new technology, new vendor, et cetera. So listening for those can really trigger a really fruitful discussion.

David Williams:

One of the things when you're doing expert interviews and comparing one to the other, and you may get different information. So one example would be work we did recently on adaptive clinical trials. And you'll ask somebody, what percent of trials are adaptive trials and you'll get, answers that range all over the place from 10% to 90%. And a lot of times this has to do with what's the definition. And so I think I'll just mention a few that we've done recently. I mentioned adaptive trials, but also wearable devices, real world evidence. Closed claims versus open claims. it's important to really define a term well, and sometimes what happens is people will use it, use a term like that, or, other one might be like virtual trial or decentralized trial. You think, okay, I guess I'm supposed to know what this means, but when you probe them, either there is a specific definition that they're talking about, or it may be.

Karen Donovan:

Yeah.

David Williams:

it actually, isn't very clearly defined. And part of what you can do is to, and part of the, part of the overall project is actually to make that definition.

Karen Donovan:

I have countless examples of that. So, yeah. So I think that's an excellent point.

David Williams:

Now these experts, tend to be experts in fairly narrow areas. And I think one thing that you do a good job on Karen is regardless of what your original structure and questions were, kind of honing in on their key areas of expertise.

Karen Donovan:

No. Every company is structured differently in terms of what their roles and responsibilities are. Another example is for example, if they are working in a global organization and they have counterparts in other parts of the organization, often, or they might work on a special task force or a special cross-functional group you need to hear from them. We do like to ask them upfront what is their scope of their role and their responsibilities, because you might find out that they have some other really important role that's closely adjacent and highly relevant. And you want to make sure you're listening for that and can probe appropriately.

David Williams:

Some of this is also recency sometimes you'll hear about a story and could be that they're talking about something relevant, but it could have been from, 15 years ago in in few industries, is that okay? But know, it's, sometimes we're working with something that is particularly time sensitive and something that was five or 10 years ago is actually, how can it be that valuable? So sometimes you do want to nicely ask about what they're talking about when it occurred.

Karen Donovan:

Well, that's true. Even if they've been at the same company all the time, but it also gets a little trickier if they've been at three separate companies over the past 10 years. And these are again good examples of. You use the opportunity to know their background and say, how did company A do it differently than company B or company C. And when they're giving examples, you ask, okay, well, which company was that? What timeframe was that? Because it certainly provides meaningful context.

David Williams:

So let's talk about what you do in the last, let's say quarter of the interview. So we already got ready for it, we broke the ice, we're doing the heart of it. And now the clock is ticking toward the top of the hour. How do you use that last quarter of the interview?

Karen Donovan:

It depends on how talkative they've been again. But if you have the extra time, we often, as many of our conversations we're asking about growth opportunities and where they see the market heading and the key headwinds and tailwinds and what we might often use that time for is, okay, we've talked about a lot of different trends. What do you see as the top two or three most important trends that are going to change the way the industry is functioning going forward. I might restate what I think I've heard them say the top two or three things are, I might ask them to think about it. Many times we're asking them questions that they haven't really thought about in the same type of way, and they're describing things for the first time. And so even as they hear themselves explain something and hear the kinds of questions, sometimes you can hear their opinions shifting. So you use that opportunity to say, okay, based on all of that, what has really resonated the most for you. And then the other bit that we do like to have something open-ended, what else would you like to add that you think will be useful on this topic? What did I not ask you? Or what does it, you know, someone, looking to make investment in the space, what would you like them to know and want them to prioritize or focus on? Those are really great ways to let them talk and listen, to share their, big picture thoughts.

David Williams:

Good. And I think that's related to your comment before about the market researcher calling and their, marching through a script and you're saying, well, gee, if they asked me about, you know, this or that, it might be better. we're trying to do that anyway, throughout the interview, but here explicitly to say, what else, based on what you've heard me asking, what should I be asking? What should I be doing? And then a lot of these sources, depending on if they're from academia or the nature of their expertise, they may have some

Karen Donovan:

sources

David Williams:

that they rely on. So they may point you to a particular journal, database, article, or another expert. And those can all be helpful as well.

Karen Donovan:

I would add to that. We like to say what conferences do you attend, for example, on these topics and what are the journals that you read? These are really great ways of getting the industry trends. We often will go on our own reach out to conference speakers and go to these conferences or pull information from these various associations, et cetera.

David Williams:

tip number five is to incorporate expert interviews into a broader gathering process. We focused here today, very much on the expert interviews themselves, and they can be useful as a standalone, but we usually use them in the context of a broader project. And in order to do that there's some basic housekeeping things to do. And then there's kind of a broader use that we make of these interviews.

Karen Donovan:

Yeah. So as you can appreciate, we're producing presentations and sharing synthesis of our findings. So having good notes is critical to be able to take what you've heard and be able to ultimately use that as raw material for the final outputs. So. But whatever. I do pride myself on taking incredibly good notes. You know, not just at a detail level and even some of those the cuff comments and making sure that you're capturing that for your own sake. But you know, we also want to, I do try to go back afterwards and say, okay, what were the key takeaways while they're still fresh, while the interview is still fresh in my head. That's definitely a best practice. And then we do share with others on the team the notes. Occasionally we record and again with the person's permission and really share that and be able to do some debrief with in some cases the clients even joining us for some of those calls. So those are all really helpful while, everything's still fresh the mind to make sure you've captured things right. And being able to leverage that and improve later iterations of interview guides and structure.

David Williams:

So I think once you've got the interviews, those are components of our overall work. Using those interviews to reconsider our hypotheses, we try to have hypothesis driven work plans. So we're testing those in the interviews, but we may also come up with new hypotheses reviewing these definitions sometimes, we talked, we're talking a little bit in retrospect about what we discovered that definitions weren't standardized. So being aware of that upfront is good, but then trying to reconcile after the fact to understand, what are the real definitions? What do we need to look at differently? Refining an interview guide is, as you mentioned, and then providing feedback back to those expert network companies about, what was the quality and suitability of the experts they introduced that helps them to do their job better and keeps them on their toes as well. And then, the expert interviews are not the only data source we're using. We're speaking with the clients we're speaking with other companies and customers and competitors, other experts and secondary data, and bringing this overall into a set of strategic hypotheses and findings and ultimately open issues as well. So Karen, it's been great talking with to you today about interviewing an expert about expert interviews, sort of a, has a meta feel to it. Hope that everyone has found this to be a useful conversation. If you want to continue the conversation, please contact us with the contact form on health business group.com. or connect with either one of us on LinkedIn. you. And thank you, Karen.

Karen Donovan:

Great. Thanks.

Intro
Tip #1: Make the expert networking companies do their jobs
Tip #2: Be prepared
Tip #3: Start the interview off on the right foot
Tip #4: Get the most out of the expert
Tip #5: Incorporate expert interviews into a broader data gathering process

Podcasts we love