Is it True that Most Interventions are not Effective? A Super Speedy Review

Join us for a super speedy exploration of evidence-based integrative medicine with Dr. Joshua Goldenberg. In a landscape where only a fraction of medical practices have top-tier evidence, we scrutinize healthcare implications with Dr. Ian Edes from Stanford. Dr. Adam Sadowski joins to dissect findings, debating evidence standards and urging for rigorous research. Tune in for a stimulating discussion on the pressing need for high-quality evidence in medicine.

Learn more and become a member at www.DrJournalClub.com

Check out our complete offerings of NANCEAC-approved Continuing Education Courses.

Transcript

Introducer: 0:02

Welcome to the Dr Journal Club podcast, the show that goes onto the hood of evidence-based integrative medicine. We review recent research articles, interview evidence-based medicine thought leaders and discuss the challenges and opportunities of integrating evidence-based and integrative medicine. Continue your learning after the show at www. drjournalclub. com.

0:31

Please bear in mind that this is for educational and entertainment purposes. Only Talk to your doctor before making any medical decisions, changes, etc. Everything we're talking about that's to teach you guys stuff and have fun. We are not your doctors. Also, we would love to answer your specific questions. On www.drjournalclub.com you can post questions and comments for specific videos, but go ahead and email us directly at josh at drjournalclub.com. That's josh at drjournalclubcom. Send us your listener questions and we will discuss it on our pod. Hello and welcome to Dr Journal Club. 

This is your host, Dr. Joshua Goldenberg. This is a super speedy video where we're going to go over a really excellent basic level video that Dr Adam Sadowski put together for us in my opinion, one of his best. So please go ahead and watch this video.

Dr. Sadowski: 1:25

He takes a really careful critical look at this paper and really asks the question. If perhaps the author has kind of oversold it, but here is the paper. It came out this year, in 2022, in the Journal of Clinical Epidemiology, which is a very good journal, and basically it's by Harlech et al, but that includes Dr Ian Edes at Stanford, who is sort of very famous in this space of evidence-based medicine and sort of highlighting the fact that most evidence out there is pretty piss poor, which I would probably agree with. So the problem, of course, is that the evidence-based medicine movement has really highlighted a bunch of examples where common practices, established practices of medicines, had either been just straight out wrong or maybe even dangerous once we looked at it with better evidence. One of the examples they give is recommending to put babies in their cribs belly down at first, and then realizing that, oh no, that was actually a terrible idea. Back is fast, et cetera. So the question was, though is that the exception or the rule?

Dr. Sadowski: 2:30

If we look at the overall evidence-based medicine today, what amount or what proportion of high quality evidence? Well, let's put it this way what percentage of interventions have high quality evidence that these things work in a way that's beneficial for patients and statistically significant? And so what they did was they looked at a random sample of all Cochrane reviews published between 08 and 21. So Cochrane reviews, of course, are generally considered the pinnacle evidence, synthesis of medicine interventions sort of period, and so what they wanted to do is okay, we're going to look at this for each of these different interventions, whether it's a drug or an herb, or acupuncture, or whatever it is, and we're going to say how many have high level evidence. So that's using grade, and we've got a zillion videos on grade if you want to learn more about that.

Dr. Sadowski: 3:21

So what proportion have high level evidence that this intervention actually works, is statistically significant and is beneficial? And what they found was that a remarkable only 5.6% found high quality evidence that interventions worked. Now what Dr Sudowski really underlines is this of course does not mean that 5.6% of interventions work. It just means that most interventions really don't have high level evidence. And Dr Sudowski goes into maybe that's too high a bar. Maybe I agree, maybe I disagree, but I think the take home that even Dr Sudowski would agree with is that the amount of high quality evidence that we have out there for effective interventions is just remarkably low. So what that means is we need better evidence, more evidence, to get to that high quality level, and I would be interested to see, for example, what percentage of high quality evidence is positive or negative, not just what percentage of all evidence. Has high quality research for something being effective or not. So there's a little bit of a nuanced issue there, but I think what's getting conflated are the fact that you have just very few studies have high level evidence at all, and then, of course, not all of those, and probably a small percentage of them, are really going to have meaningful impacts, I would guess. So go ahead and watch the full video by Dr Adam Sadowski. Again, a great job, a great video on his part, and we'll see you again later.

Dr. Goldenberg 4:54

If you enjoy this podcast, chances are that one of your colleagues and friends probably would as well. Please do us a favor and let them know about the podcast and, if you have a little bit of extra time, even just a few seconds, if you could rate us and review us on Apple Podcast or any other distributor, it would be greatly appreciated. It would mean a lot to us and help get the word out to other people that would really enjoy our content. Thank you, hey y'all. This is Josh. You know we talked about some really interesting stuff today.

Dr. Goldenberg 5:19

I think one of the things we're going to do that's relevant. There is a course we have on Dr Journal Club called the EBM Boot Camp. That's really meant for clinicians to sort of help them understand how to critically evaluate the literature, et cetera, et cetera Some of the things that we've been talking about today. Go ahead and check out the show notes link. We're going to link to it directly, I think it might be of interest. Don't forget to follow us on social and interact with us on social media at Dr Journal Club DR Journal Club on Twitter. We're on Facebook, we're on LinkedIn, et cetera, et cetera. So please reach out to us. We always love to talk to our fans and our listeners. If you have any specific questions you'd like to ask us about research, evidence, being a clinician, et cetera, don't hesitate to ask. And then, of course, if you have any topics that you'd like us to cover on the pod, please let us know as well.

Introducer: 6:09

Thank you for listening to the Dr Journal Club podcast, the show that goes under the hood of evidence-based integrative medicine. We review recent research articles, review evidence-based medicine thought leaders and discuss the challenges and opportunities of integrating evidence-based and integrative medicine. Be sure to visit wwwdrjournalclubcom to learn more.