As we wrap up our exploration, we delve into the challenging task of assessing homeopathy's efficacy, considering not only scientific aspects but also clinical evidence and patient outcomes. The debate spans from the rigors of scientific reasoning to the inclusivity of evidence-based evaluation. Join doctors Goldenberg and Davis as they dive into the controversial topic of homeopathy.
Jacobs J, Jonas WB, Jiménez-Pérez M, Crothers D. Homeopathy for childhood diarrhea: combined results and metaanalysis from three randomized, controlled clinical trials. Pediatr Infect Dis J. 2003 Mar;22(3):229-34. doi: 10.1097/01.inf.0000055096.25724.48. PMID: 12634583.
Learn more and become a member at www.DrJournalClub.com
Check out our complete offerings of NANCEAC-approved Continuing Education Courses.
As we wrap up our exploration, we delve into the challenging task of assessing homeopathy's efficacy, considering not only scientific aspects but also clinical evidence and patient outcomes. The debate spans from the rigors of scientific reasoning to the inclusivity of evidence-based evaluation. Join doctors Goldenberg and Davis as they dive into the controversial topic of homeopathy.
Jacobs J, Jonas WB, Jiménez-Pérez M, Crothers D. Homeopathy for childhood diarrhea: combined results and metaanalysis from three randomized, controlled clinical trials. Pediatr Infect Dis J. 2003 Mar;22(3):229-34. doi: 10.1097/01.inf.0000055096.25724.48. PMID: 12634583.
Learn more and become a member at www.DrJournalClub.com
Check out our complete offerings of NANCEAC-approved Continuing Education Courses.
Welcome to the Dr Journal Club podcast, the show that goes on to the hood of evidence-based integrative medicine. We review recent research articles, interview evidence-based medicine thought leaders and discuss the challenges and opportunities of integrating evidence-based and integrative medicine. Continue your learning after the show at www. drjournalclub. com.
Dr. Joshua Goldenberg:Please bear in mind that this is for educational and entertainment purposes only. Talk to your doctor before making any medical decisions, changes etc. Everything we're talking about that's to teach you guys stuff and have fun. We are not your doctors. Also, we would love to answer your specific questions. On wwwdrjournalclub. com you can post questions and comments for specific videos, but go ahead and email us directly at josh at drjournalclub. com. That's josh at drjournalclub. com. Send us your listener questions and we will discuss it on our pod. Hello and welcome to the Dr Journal Club podcast. Today we have a guest with us, Dr Mark Davis. Hello, sir, how are you doing?
Dr. Mark Davis:So good, so happy to be here.
Dr. Joshua Goldenberg:So I adore Mark. We go way back, and one of the things I love about you, sir, is I feel like you're always, like you're just always full of life, and every time I say hi, you're just like in a really great mood. And I mean, I don't know if it's a facade I don't think it is. It seems authentic. I don't know how someone could be continuously that full of life.
Dr. Mark Davis:I feel like I was like lucky. I was born happy and it just never wore off.
Dr. Joshua Goldenberg:Nice, nice, nice, nice, nice. Dear listener, Adam has absconded for the day. He claims he has family responsibilities. I have a feeling he's just going to go do a try workout or something. But we will assume the best about him and he will join us again next week. Or he's just really sick of talking about homeopathy, which could also be true. There could be more than one true thing. So this is probably our last homeopathy talk in the series. And, Mark you well, maybe tell us a little bit about yourself first, and then maybe you know why, why you're here, why we're, why we're chatting and all that jazz, and then we can, we can jump in.
Dr. Mark Davis:Okay, great, so. Yeah, so I am also a naturopathic doctor. I've been practicing since 2011 and and I am a long time listener first time guest on the show and I love listening to it. And when I heard you do start this meta wars on homeopathy, I felt like you were describing me as you described yourself yeah, the skeptic sitting in the back of the naturopathic classroom. We had four required courses in homeopathy, which is like a lot of courses and the whole time.
Dr. Mark Davis:So I grew up, a little bit of background. I grew up in a kind of a medical family. My mom is a nurse. She has three siblings who are doctors. Their mom and dad were a nurse and a doctor. Their dads mom and dad were a doctor and a nurse. And so I grew up. Our family was all in unconventional medicine.
Dr. Mark Davis:I'd never heard of homeopathy, didn't, didn't know it existed until I was in my late teens and, randomly, the suburb next to mine growing up is the largest concentration of this religion called Swedenborgism, I think, in the world, or maybe at least outside of Sweden, and there's a lot of famous Swedenborgist homeopaths. As I learned in school, did not know, interesting, yeah. So anyway, when I met those people they were like oh, somebody falls down and skins their knees, their grandmother gives them Arnica, and so homeopathy is just in the culture of that group of people, and so that's the first time I ever heard of it. And so then I got to school and I was like, oh, this is the thing that weird Swedenborg journalists do, like, okay, what's this? I was like.
Dr. Mark Davis:And then I heard about this idea that these are substances that at least in the most common potency, 30 C, are diluted beyond Avogadros number. I was like this is impossible. So if it's real, it blows everything up. I have to find out if it's real. So I did the same thing you did I read the full text of every randomized control trial I could find, and when Lint et al was published, I read it obsessively. And when Shang et al came out well, I guess actually both of those were published, I guess when I discovered them, I read them obsessively, because they were published before I started school.
Dr. Joshua Goldenberg:I think we met talking about Shang, didn't we?
Dr. Mark Davis:Absolutely. Yeah, I remember in best year our good mutual friends were like oh, you two have to know each other.
Dr. Joshua Goldenberg:The nerds in the back of the class talking about homeopathy meta-analyses. You two need to meet.
Dr. Mark Davis:Exactly yeah. And my dad is a physicist so in his world he's like oh yeah, statistical significance for, like particle physics at the large Hadron collider is like 0.0005 or below. So his world of significance is very different. And so I started school. I was like could this homeopathy thing be real? That would, it can't right. And so I took all the courses, I read all the papers, I wrote a song.
Dr. Joshua Goldenberg:You did? You wrote a homeopahty song?
Dr. Mark Davis:Can I sing it for you. Here's how it goes. , oh homey, homey, homey, homeopathy. They tell you what you get is much more than what you see. Does the science bear it out? I have to ask you when? Because because science does not have good answers in 2010. Sure, it's been controversial from the start, because medicines of science and an art, oh, but I wonder, is it the medicine for me? Well, homeopathy set me up.
Dr. Joshua Goldenberg:Oh my god, I'm so much. Thank you. Ah, did I not know you had a song.
Dr. Mark Davis:Yeah, I wrote it in 2010 because I was obsessed. I was like I have to know. And then someone was like, oh no, you know what you have to do. You have to take the Nash course, new England School of Homeopathy. You did the Nash course. Yeah, it was thousands of dollars I didn't have as a portion. I borrowed money from my parents to take this course.
Dr. Joshua Goldenberg:That's right, so it was just a nurse, my mom's a nurse, yeah.
Dr. Mark Davis:And they were like, okay, okay and so, but like four or five weekends, and I was like, oh, this is not going to convince me of anything. No, it's just what it does.
Dr. Mark Davis:So I never finished the course, and so, you know, I was there doing the same skeptical thing you were doing, and so when you started this series, I was like, oh my God, I've got to know. And so, yeah, I say you a couple of trials. Yeah, thanks for reviewing the Depression. One Turned out to be nothing. I actually hadn't read it. I felt like a king who had brilliant meta analysts to read a paper for me and explain that it was sort of nothing. I felt so lucky, and so I'm so excited to have this conversation, because this little meta analysis is one that I have brought up several times in conversations with friends and family who I talk about homeopathy, so I'm excited to read it.
Dr. Joshua Goldenberg:Awesome, awesome, awesome. Well, let's jump in. I want to set a little bit more of the stage on this one that I thought was a nice addition. So maybe we need your dad on the podcast to do the physics stuff, but essentially, like we started with the meta wars Linn paper really well done. Meta analysis, pro homeopathy shang paper really good meta analysis. Maybe it's still placebo, and the end of that was this you know these arguments about well. You know, maybe it was publication bias we talked about that maybe it was small study effects talked about that but one of the things that they ended with was you know what would be really, what would be more convincing is if we had, you know, consistent evidence for a particular condition. That's when we did the Depression, and then which you sent us, thank you. And then also, if we had, like, consistent evidence within a single lab, that would be you know, supportive evidence as well, and then you sent us this one as well, which is exactly exactly that, basically.
Dr. Mark Davis:Yeah, I think that was Adam's take home from Linn is there needs to be more reproducibility? A bunch of studies from, ideally, a bunch of different labs, but even one lab. And and then shang was published in 2005, but they only looked at the literature up until 2003. And so this meta analysis we're going to talk about was published in 2003. The individual clinical trials were all published earlier, and shang must have looked at them, I would assume. So yeah. But if you remember, they were only looking at the eight largest trials. Oh, yeah, yeah.
Dr. Joshua Goldenberg:And these weren't large enough. That's a very interesting approach.
Dr. Mark Davis:Yeah, their approach is very interesting. I actually want to, if you guys have it in you, to do one more episode. Okay, what do you got? I'd love to send you some some rebuttals that people wrote oh, we should totally do that. Like link wrote a response to shang Okay, yeah. And then there was another group that looked at the shang data and was like they didn't pre register what, why they picked the trial sizes. They did, and if you look at the seven largest trials using the methodology, you get significance. And if you look at the nine largest trials, you get significance.
Dr. Joshua Goldenberg:And it's kind of and they did eight.
Dr. Mark Davis:They did eight.
Dr. Joshua Goldenberg:Yeah, that's interesting. I had the same thought about the registration because the methods were so novel and unique. And there was, you know, there's some control by also doing it, for, you know, conventional medicine. But yeah, there's, there could be a lot of opportunity for p hacking there without registration. But we should. Maybe we should do the the lin paper. Maybe you'll join us again. Guest appearance, guest homeopathic appearance, great for the the lind rebuttal. Okay, cool, so then. Yeah, so then this was one of these ideas of can we like repeat this experiment multiple times, multiple occasions, and get, and what would we see? Do you want to, because you, you clearly know the stuff super, super well. Do you want to walk us through like the basics of the study design and then we can get into like critique and thoughts and stuff?
Dr. Mark Davis:Yeah, so the studies called homeopathy for childhood diarrhea combined results and meta analysis from three randomized controlled clinical trials published in pediatric infectious disease journal in 2003. Let's see, has an impact factor of 3.2 now, so that's not too bad. And authors are the primary investigators, jennifer Jacobs, who was also the primary investigator in each of the clinical trials. Second author is Wayne Jonas, who was he was the director of N cam, which is now NCCIH, for a bunch of years and which is the NIH, is complementary integrated medicine and arm.
Dr. Mark Davis:Yeah, and he teaches medicine, that he's a medical doctor.
Dr. Mark Davis:teaches at Georgetown, near me, and university uniformed services University of Health Sciences, high level guy very, very famous Yep high level guy, very famous guy, so maybe that's a little bit conflicted that she's the primary investigator on all three clinical trials and not not conflict of interest, but like more bias, potential for bias.
Dr. Mark Davis:So yeah, so she had done three small ish clinical trials in Nicaragua and also in Nepal, and in those trials they found kids suffering from infectious diarrhea and they gave them all standard of care, which is oral rehydration therapy, and they all had classical homeopathy intakes. And then they took their prescription to a pharmacist who gave them either the homeopathic remedy that was prescribed for them or an identical placebo. It was double blinded. Only the pharmacist knew who was getting what. And then they were assessed for a bunch of things. The main outcome was resolution of diarrhea, which I think was like two consecutive days of formed stools, two of only formed stools. Anyway, so the first pilot study homeopathy, safe and like I think it was not significant but trended towards significance. The second one, also in Nicaragua, was larger and significant, if I'm remembering correctly.
Dr. Joshua Goldenberg:Well, just shy there I think it was 0.07.
Dr. Mark Davis:Okay, 0.07. And then the third one in Nepal. That one did hit statistical significance.
Dr. Joshua Goldenberg:I think so, at least for the number of stools per day, I believe.
Dr. Mark Davis:Oh, that's right, Okay, so they were looking at. Oh, I see. So the second Nicaragua trial was statistically significant for duration of diarrhea but not for number of stools per day, right. And then Nepal won one same thing, actually 0.06 for duration of diarrhea and 0.03 for number of stools per day. So anyway, they're small trials. And she said let's combine these into a little meta analysis and see what we can find.
Dr. Joshua Goldenberg:Yeah, good, yeah, I think that's. That's excellent. Anything else that you want to add about the methods or you feel like? I mean, I think that's pretty good description.
Dr. Mark Davis:A few more details. I mean there were a total of almost 250 kids, so that's the size kind of like across all three studies, kind of respectable. Yes, so the actual the docs doing the homeopathic prescribing was a different set of practitioners at each study site and one thing so this was all using that 30 C potency which as far as we know.
Dr. Joshua Goldenberg:So over Avagadra's number. Okay, so if it's, if there's an effect, it's due to the homeopathic effects, not, we can't say it's due to you know the fact that there's herbal effects or something like that. At that, at that level.
Dr. Mark Davis:Not as far as we know, Okay cool. Not according to pure math.
Dr. Joshua Goldenberg:Right, and then they had a few other interesting safeguards in there. I thought, like one complaint you sometimes or one concern you sometimes hear is well, maybe it was adulterated with an active you know medication or something and that's why it worked. And so they did do a random sampling of the pellets and found no difference between them, so they did have some control for adulteration. Like you said, the dilution was was quite far down. So we're dealing with these homeopathic effects only in theory. Yeah, I think that I think that was good. You want to roll into the results and then we can try to critique it a bit. Okay, yeah.
Dr. Mark Davis:So 230 of the kids completed the full five day follow up period and looks like okay for those patients. There was an 18.5 reduction in the duration of diarrhea from 3.8 days in the placebo group to 3.1 days in the group receiving homeopathy, and the mean number of stools per day for each patient, oh, and that was p value of 0.008. So nice and statistically significant. And then the mean number of stools per day for each patient during the entire five day treatment period was 2.7 for the treatment group and 3.4 for the placebo group, p value of 0.004. So also nicely statistically significant. And then it gets into a bunch of stuff that is kind of over my head where it's talking about Kaplan-Meier plots and Cox regression analysis, and I was hoping, if that stuff seems relevant, you could comment on it.
Dr. Joshua Goldenberg:Yeah, I don't know that the, that the survival analysis is super, super relevant. I do think that if we keep just sort of like the basic meta analytic results I think are interesting, and then more, and then also just the setup, so let me well, okay, let's start with that. So why don't you start with your thoughts, if you want, and then I'll share my thoughts and we can, we can chat about it.
Dr. Mark Davis:Well, so if this is a treatment of any random thing and we're not calling it homeopathy, it's kind of a slam dunk. It looks like it works. The trial design looks good, the you know they're. They're. If you look at their conflicts of interest. At the end, Boyron, a homeopathy company, did give them a research grant to support the project, but these are respectable people in academic positions and so I don't think getting some money to do the trial is enough to make me really worry about a conflict of interest. They're not like on their boards or any kind of conflicts of interest like that. They're not paid by the companies. So, yes, so I mean, if it's anything but homeopathy, it's a nice trial design and nice statistically significant results. The clinical significance, I guess I would say, is not super amazing. You know the difference between 2.7 stools and 3.4 over a five-day period. Mm-hmm, mm-hmm.
Dr. Mark Davis:Or the difference between what was it? 3.8 days of diarrhea versus 3.1 days. That's not like a wow home run. That's really changing people's lives. But really, if this question is does homeopathy do anything? Looks like it does.
Dr. Joshua Goldenberg:Gotcha. Yeah, yeah, interesting. I think I agree with some of the things that you're saying, like I think the conflicts of interest are definitely there. I guess I have more concern about them than you do. So, yeah, so it looks like the funding came from a homeopathic company. Like you said, the author of the systematic review, the first author, is also the researcher in these studies. Yeah.
Dr. Joshua Goldenberg:So there's like definitely personal and professional and belief structure conflicts there, a potential conflict, or at least like affiliation to one side, I would guess. And so normally I would say I agree with you that you can have those influences and have a fine study, like if you kind of cross all your T's and dot all your I's, and I don't know that they did that. So I guess I'll take a little bit of a nuanced approach. So I agree, like I think the primary studies were from what I could tell reading about them in this. I didn't look at the original ones but they looked pretty solid, like the design is good. It was like quadruple blind. You had this one pharmacist in the US who was like actually knew what was what and everyone else had no clue. You had secondary people following up with the patients, like yes, the parents were reporting on their kids response, but the parents were blinded too, so it shouldn't really matter and it's really easy to blind homeopathy. So I feel like that blind really stuck. So I don't have any issues with the primary studies. I guess I have issues with this secondary analysis. So it's a meta-analysis but it's not a systematic review, which is kind of weird. Like normally you have a systematic review with a meta-analysis or you have a systematic review without a meta-analysis. Very rarely do you have a meta-analysis without a systematic review. And basically they're like hey, we did these three studies and there's a lot of similarities, so we think it would be useful to meta-analyze them. So we're going to meta-analyze them and that is that's like a totally legitimate position to have.
Dr. Joshua Goldenberg:But I guess my concern was when and there's like no, that I could see, I don't maybe you caught it like no registration, like they didn't register the design for this anywhere either. I'm speaking of that with the shang paper and I just and I didn't think the reporting was very good, like it wasn't clear how they did their systematic review I ended up plugging in the raw numbers into R and running it on my end and I got very similar results. So I don't feel like they're totally misleading. But I guess I guess my concern is if we take these numbers at face value from the primary studies and even if we meta-analyze them and get a very statistically significant and I mean you could even argue that clinically significant result, you know, shave off three quarters of a day, like maybe that's a big deal, maybe not. But if we're kind of interested in this is this homeopathy real thing argument then we'll take anything, I guess, for that point. So I guess my question is to the argument of this is all publication bias right? And I wonder. I'm concerned that we didn't do a systematic review on this clinical topic right, because it could be that these three studies indeed did find this, but there's another three studies in this population that didn't, and if they were meta-analyzed together they would be the whole thing would be negative. I think that was.
Dr. Joshua Goldenberg:My biggest concern is like I didn't. I don't know that there isn't other evidence out there that should not have been included in this. But I agree with you. You just do the straight meta-analysis. You do see this highly significant thing and just the thing about p-values real quick. So it's so interesting that you're saying that about your dad and physics.
Dr. Joshua Goldenberg:We have this like obsession with like 0.05. I just gave a class on this at Sonoran last week where I was like it's somewhat random and like if you actually talk to statisticians they're like why are doctors obsessed with this number? And they wrote like a whole book of responses about how doctors have just like totally taken it. I mean it's not unimportant, but it's like taking it too far. So it is like tyranny of a threshold type of thing. But I would say like it is totally legitimate to meta-analyze three studies that are like barely significant and have a significant. So that's like one of the major points of a meta-analysis is to borrow strength across multiple sites. So I have no issue with that whatsoever. But yeah, I guess I was just. My biggest concern was the conflicts of interest and the lack of a systematic search.
Dr. Mark Davis:Yeah, those are my concerns too. I mean sort of what I appreciate about it being a meta-analysis without a systematic review. Is that it's really apples to apples? Right, that's true. They're like look, we're one group, we did these same trials the same way.
Dr. Joshua Goldenberg:That's right.
Dr. Mark Davis:We can just plug and play with our numbers and there's no like fudging with different outcomes or different times of collecting data, that type of thing, and that was an issue, if I remember correctly. I was looking at Lin's response to Shang and they were like you're comparing such disparate things that it's hard to tell if you are really getting a result worth considering, and with this, you're really comparing the same thing every time.
Dr. Mark Davis:That's true, but could there be publication by? I mean, is there even some sort of bias in this group, because they are all doing something the same way? That is a total concern.
Dr. Joshua Goldenberg:Look, the thing is we don't do this for money. This is pro-bodo and, quite honestly, the mother ships kind of eeks it out every month or so. Right, so we do this because we care about this, we think it's important, we think that integrating evidence-based medicine and integrative medicine is essential and there just aren't other resources out there. The moment we find something that does it better, we'll probably drop it. We're busy folks, but right now this is what's out there. Unfortunately, that's it, and so we're going to keep on fighting that good fight. And if you believe in that, if you believe in intellectual honesty and the profession and integrative medicine and being an integrative provider and bringing that into the integrative space, please help us, and you can help us by becoming a member on Dr Journal Club. If you're in need of continuing education credits, take our Nanceac-approved courses. We have ethics courses, pharmacy courses, general courses, interactions with us on social media, listen to the podcast, rate our podcast, tell your friends and there are ways that you can help support the cause. Yeah, it's interesting. So this was a really interesting paper for a lot of reasons.
Dr. Joshua Goldenberg:I think I've come across one or two papers in 10 years where they do a meta-analysis without a systematic review and it's the same idea. It's like one clinic or one lab that's done a bunch of similar experiments and I think, if you think about the theory behind meta-analysis in general, in a situation like that you would do something called a fixed effects meta-analysis and you would take out heterogeneity because you would say that these are essentially the same experiment run three times. And I think, if we think of I haven't had to think about this in so long but if we think about this from the philosophy almost of meta-analysis, it's almost as if it's one single study that we're now reporting on. And so I wonder if a better way to think about this is here are three small studies, or you can also think about this as one large study which was statistically significant, and I think that is probably the better way to think about it, because it doesn't address at all the possibility of yeah, you can have individual statistically significant studies, but then that may be random chance, whatever it is, and you need to look across the literature. So, yeah, I wonder about that.
Dr. Joshua Goldenberg:And when I ran they didn't report their heterogeneity, I don't think. But when I ran it in R there was like zero. It wasn't zero, it was like 52%, I squared, which is pretty good, and basically like no, no, no, there was a 0% heterogeneity, so I squared. So essentially like it was three identical studies and it turned out that the random effects meta-analysis which you would normally run in a fixed effects, which you would run in a situation like this, were identical because there was like no heterogeneity to consider. So I really think it might just be best thought of as one large study in that way, with just like three sites done over time. Yep.
Dr. Mark Davis:Yep, that makes a lot of sense. Yeah, I have another. I don't know how much time we have. Yeah, yeah, we got time. Okay, go for it, hot takes. So I think really, in terms of the meta wars, this kind of doesn't really count as a meta, it just Right, because when we say meta, we mean systematic review with meta-analysis.
Dr. Joshua Goldenberg:Yeah, it's just cooler to say meta wars. Yeah, that's right, that's right.
Dr. Mark Davis:Yeah, yeah, yeah. And so, even though this is a meta-analysis and it's got more to look at than the depression one that you looked at last time, it's not a systematic review and meta-analysis to really battle with Wint at all and Shang at all.
Dr. Joshua Goldenberg:I think, that's fair.
Dr. Mark Davis:So that takes me back to thinking about kind of the philosophy of it, and you guys have kind of talked about Bayesian statistics a little bit on the podcast. Do you want to tell your listeners what Bayesian statistics is basically?
Dr. Joshua Goldenberg:Yeah, yeah, and this is I want to tie this back also to your dad in physics and using a different p-value, because I think this is like maybe the crux of what we're talking about, which is a really Is a hot take. So the way so the statistics, I know is frequent to statistics. I don't consider myself a Bayesian, and how. So there is formal Bayesian statistics.
Dr. Joshua Goldenberg:However, when most normal human beings talk about Bayesian approaches or Bayesian view, what we mean is that we need to consider the prior possibility, the prior likelihood. That then gets modified by a new experiment, and so normally with frequentists, we don't really do that. We basically assume there's like no previous knowledge and we do our experiment and then we come up with a conclusion and Bayesian statistics says no, no, no, that's dumb. You should consider the previous knowledge, which totally makes sense. And so the argument from a Bayesian perspective for this is if it seems so implausible, then you should have a much higher bar. Now the frequentists would say well, this is now an unfair bar type of thing that you're applying, and so that's sort of the debate we've had. Now can you tell me more about? I mean, maybe you know, maybe you don't why with physics is it so low? Is it just that we want to have? Well, yeah, tell me your thoughts on that.
Dr. Mark Davis:Well, I'm certainly not a physicist. So my basic understanding is you're working with so much data that you're going to have unlikely things happen all the time in, like particle physicists, physics. So you've got to have a really Really got to consider things significant only if they have a very, very, very, very unfrequently, or the odds of them happening by chance are very, very low, just because you're looking at billions and billions of particles. Okay, yeah, that makes sense. That's my basic understanding, but I think I use Bayesian or Bayesian statistics in that more lay sense you're talking about of. Let's just consider the priors. So what is your baseline thought? And then, how much do you want to modify that? And as I listened to you and Adam, I think, well, I don't know about you and Adam, but I can say for me, if you're coming to this with a baseline prior of, oh, something with no molecules in it working is so improbable, I would need a lot of evidence to show that it works. And in that case I'm like no, the evidence isn't there. The evidence is not there.
Dr. Joshua Goldenberg:I think that's fair.
Dr. Mark Davis:Extraordinary claims require extraordinary evidence. We haven't seen that level of extraordinary evidence. But if you're asking a bit of a different question, which is whatever is in these little white pills, does that work as well as or better than whatever is in these little placebo white pills? I think we don't need extraordinary evidence for that.
Dr. Joshua Goldenberg:Walk me through that again. Hold on Well.
Dr. Mark Davis:I'm just not convinced that there's nothing in there. Is there a memory of water and its shape different? Or is there some sort of physical force we don't understand? I don't claim to know that mechanism. Stuff is a little deeper than I can claim. So maybe there's molecules in there, maybe there's not. Maybe water has some sort of memory. I don't think so, but I don't have the hubris to think I know how everything works. We used aspirin for 80 years before we had any idea of what the mechanism of action was. But that didn't mean we needed a higher bar in a randomized, controlled trial of aspirin versus placebo. We just didn't know how it worked. And eventually we figured out how it worked. And so there's a world in which homeopathy doesn't exist or it doesn't do anything. It doesn't work. But there's a world, a potential universe in my mind, a theoretical universe, in which it does work. And in that universe a randomized, controlled trial of homeopathy pills versus placebo pills needs exactly the same level of evidence as aspirin pills versus placebo pills.
Dr. Joshua Goldenberg:I don't know if I understand the difference of what you're saying. So let me see You're saying so. If we want to say that there is like on a philosophical level, that there is something there, we need extraordinary evidence and we're not there yet. If we are not making a philosophical claim but saying I don't know should we pay, should ensure we pay for you know, this pellet or something like that, that would be enough. Is that like? What's the difference between those two scenarios? I don't know that. I'm following that.
Dr. Mark Davis:Or even just a clinical claim, like if we're trying to prove that pellets with no physical substance in them work. We haven't done that. That's too extraordinary of a claim. But I'm just not convinced that we know what's in there or we can make claims about a mechanism. I'm just like, whatever this homeopathy stuff is, does it work? And so I come to it with.
Dr. Mark Davis:You know, first I heard about it from my Swedenborgian neighbors and then I came to ND school, nature Pathic Medical School, and I heard really smart clinicians. Our homeopathy professor, will Taylor, one of the most thorough diagnosticians I've ever met, all he did was homeopathy. And hearing from these clinicians, oh yeah, I think this works. So I don't know what's in it. I don't know exactly how it works or doesn't work. I can't make claims about the mechanism of action, but I think it helps my patients. So that's the Bayesian in me came from that, from being like, oh, sounds like clinicians are saying it works. And that's the history of medicine, is the history of professional opinion. And so I feel like we're starting from the basic history of medicine. Maybe this herb works, maybe it doesn't. The person who taught me said it it does. Maybe they're right, maybe they're wrong and then assessing that with a randomized control trial, which we've done a lot of in homeopathy.
Dr. Joshua Goldenberg:That's kind of interesting. So, okay, so it's like your frame of reference. If it's like a philosophical frame of reference, that's one thing, if it's a clinician frame of reference, that's right. And, to be fair, okay, so this is exactly what you're saying. But I think it's finally clicking with me. So like, if you're a clinician and you have a patient in front of you and you want to know to give them a medication, let's say, let's say you want to give PPIs for some protein pump inhibitors or something like that, you might be confident enough saying, yeah, there's some randomized control trial evidence to support this. It seems reasonable. I think you can do it. It's not like you're digging deep on a philosophical level to know at.0001 level of p-value that PPIs actually work. You're comfortable enough in a clinical environment to say, yeah, that's enough evidence for me. Is that a fair approximation of what you were trying to say?
Dr. Mark Davis:those two groupings yeah, I mean, I guess I'm just saying that I am comfortable with homeopathy being assessed at the same level as pharmaceuticals and I don't think it needs a deeper level of scrutiny, unless you're insisting on bundling the claim that there's nothing in there with the claim that it works. And I don't care about the claim of whether or not there's something in there, I'm just curious as a clinician about whether it works.
Dr. Joshua Goldenberg:I hear what you're saying, I guess I just I wonder about that, just to push back a little bit, because the design of a randomized control trial has nothing to do with the mechanism, right, it's just if it works or not. Yeah, oh and that's maybe that's your point. And then you're saying that we should use the same rules, whereas if we're trying to make a different type of claim, that's when we would use different thresholds.
Dr. Mark Davis:Yeah, exactly Right, like I'm, like, for the 80 years we didn't know about how Aspen worked. We still assessed it with the same rules as everything else. And this thing, if it works, we don't know how it works, so we should assess with the same rules, unless we're trying to make this extraordinary claim that it's working through this method. That doesn't have to do with molecules. Like, I can imagine a world in which at some point we find out there are molecules involved in homeopathy. We just didn't know how to look at them, right.
Dr. Joshua Goldenberg:And it doesn't break our brain. It doesn't break physics as we know it to.
Dr. Joshua Goldenberg:Doesn't break physics To have a mechanism there.
Dr. Joshua Goldenberg:You know, yeah, I think that that's super, that is that is interesting. You know, I would say once I've definitely, I've definitely softened to it over the years, like I think you know I was like we were both that that student in the back of the room and I still think the chances that it's real are infinitesimal. But I think I don't know, the longer I've been in practice, like hopefully, the less hubris I have and I just feel like there's this recognition that there's so little. There are things that I don't know is more and more. And you know I think I was so I wrote this like seventh principle paper about how we should, you know, have a critical evaluation principle and that's where I think I still believe in that. But we came down really heavy handed against homeopathy in that paper and when I think about myself and who I was when I wrote that versus now, I think that I was reacting probably more to like the chip on my shoulder, want to prove this profession more than anything else, you know.
Dr. Joshua Goldenberg:I mean like I was aware of the literature, but I mean, I think, with like, with fresh eyes, as you're saying, like from a standard clinical perspective, like the research is there. It's just so hard to believe that it's real. So I don't know, is it as heavy handed should have been as heavy handed as I had? I don't know, or at least what was my motivation. I think it was more about, I don't know, getting us in a more conventional realm or something like that. I'm not sure.
Dr. Mark Davis:Yeah, you know, I think a lot of times naturopathic doctors, especially when we're early in practice, we're really afraid to be wrong or to look dumb and I think requiring more stringent evidence decreases your chance of a type one error. Right, it decreases your chance of seeing an effect that isn't there, but it increases your risk of a type two error, of missing an effect that is there. And so that's my. I also don't use homeopathy in my practice, I just don't. It wasn't convincing enough when I was a student for me to make the big effort to study it deeply, and there's so many other things that I use that I feel so confident work well that I use those things instead. But I also don't presume that I know.
Dr. Mark Davis:You know I don't want to miss effect. That's there, and I think really making sure you're so rigorous that you exclude you're willing to miss benefit is really important for medicines that maybe are toxic or super expensive or have a big societal burden or something. But for something as safe and cheap as homeopathy, like I'm, I'd be more worried to miss benefit. That is there. Then I would be worried to accidentally see benefit that is not there.
Dr. Joshua Goldenberg:That's interesting. Yeah, it's, it's so, you know. I think the crux of what we're talking about is, if we want to zoom out beyond homeopathy per se for a second, is like what is our sort of, what is our priors or what is our, what is our clinical framework, or what is the framework with which we are approaching a question? And and I think that makes sense, and you think about just informed practice anyway, you need to frame things within the risk of harm, the risk of benefit, the patient values and all that. And so we're constantly. We're constantly doing that and I also think that you know this study is a good example, like they did.
Dr. Joshua Goldenberg:I'll never forget. I don't know well, you were you. You went to NUNM but at at Bastyr we had, we had a homeopathy debate while I was a student and Nat Brignal went up against our like chief homeopathy chair I don't remember who it was and I think his, his big argument I thought it was going to revolve around like meta-analysis stuff. It didn't, and the the his big argument was you know, you're going to, you're going to there could be toxicity in there. But the bigger argument was you're going to miss other treatments, you should be giving other treatments and you're going to spend time on this bogus stuff and patients are going to get harmed. And I just remember the I thought you know, even though I was, like you know, a Brignal devotee I just remember the the other, the chair, kind of being like I'm so sorry that happened to you in clinical practice and that's a terrible, that should never have been practiced and that's a terrible homeopath. And you know, and and I that just really stuck with me is that you can you can still be a good clinician and keep an eye on things and have, you know, you know, understand when you're no longer comfortable with things progressing.
Dr. Joshua Goldenberg:And I think they did a good job with that in this study where you know, they basically said we're going to do homeopathy for a few days, we're going to send a I don't know if it was like a clinical personnel or social worker something out to the houses and make sure that the patient's okay.
Dr. Joshua Goldenberg:After a few days, we're going to do parasite testing and if, if they're not better, we're going to treat them, you know, and and we're going to check in on them, right. So it was like very intentional to not allow for that risk and I think, like you know, I don't know what what Dr Brugnall's experience was, but yeah, I could see and I had patients when I was at Bestia where they were like I refused to do conventional medicine and you know I just want to do that and that was very uncomfortable for me, right, cause it was like that's clearly their choice. But you know that, in my belief, structure like you may be harmed by this, right. So I don't know it's a I think the harm argument is real, but I don't know that it's required, right. So you can be a reasonable person prescribing out there what to me seems like out there stuff, as long as you're keeping an eye on risk and time and when do you bail and when are we being religious about this versus keeping the best eye on the patient, type of thing.
Dr. Mark Davis:Yeah, yeah, that'll make sense to me that you know. That being said, I still don't use it with patients, even though it's safe. And yeah, I guess, I think, I think, if it is real.
Dr. Joshua Goldenberg:I can't. I can't stomach it, I don't do it.
Dr. Mark Davis:You probably have to pay a lot of attention to it to get a good effect, like the classical homeopaths giving individualized remedies. In these three studies that were meta analyzed and I'm not I'm not that I only have so much time. There's only so much I can do, so don't give give it to patients. When patients come into me and they say, oh, I'm taking this remedy, I also don't tell them. Well, that's bullshit. You should stop, because we really don't know. It's 2023. And I think science still does not have good answers.
Dr. Joshua Goldenberg:Yeah, I think that's kind of where I'm at, like I have yet to prescribe a homeopathic and I never will probably. It's just not my jam and I don't feel confident enough in it. But yeah, but I think like as the years progress, I've got less and less judgmental when other clinicians have brought it up and things like that, and I think that's probably a fair statement.
Dr. Mark Davis:Yeah, I will say zooming out from is there an effect to? Is it a thing that clinicians could or should use? I mean, there is all the non specific effects that you've talked about, the like, the intake, the fact you're taking something. So I don't use it with patients. But you know, my son does competitive martial arts and I throw him some Arnica sometimes when he comes back all bruised. Is it just a placebo effect? I mean, yeah, maybe it is, but you know it's not going to hurt him and I happen to have it on my shelf.
Dr. Joshua Goldenberg:So yeah, yeah, yeah, no, I know my, my wife will use it with our, with our kids. And I'm just feel like muttering under my breath and she's like whatever. And then you know it's, you know, people get better regression, the mean or whatever, and she's just like well, see, here's the proof. And I'm like but yes, I see this reinforcement that you know we get from all medicines, of course and sure, but you can see how that would. That would absolutely play out.
Dr. Mark Davis:Yeah.
Dr. Joshua Goldenberg:Um, okay, cool man. What else, Anything else we should talk about here?
Dr. Mark Davis:I think that's, that's it with the study. So basically not systematic review and meta analysis, but sort of an interesting large ish for homeopathy randomized control trial.
Dr. Joshua Goldenberg:Yeah, yeah, and I liked your you know the to the earlier point of we need replication, which, like you said, was it Shang or Lind argued? I guess Lind argued. This is indeed that right, it's. It's a threefold study being replicated across the board and showing similar effects.
Dr. Mark Davis:So here's one other thing. Yeah, when I was pulling up all these Jennifer Jacobs papers to you know, print out this meta analyst. Are you prepped, nice? Oh, yeah, I read the study in advance. I found out. So this meta analysis was from 2003,. There was one more Jennifer Jacobs trial of homeopathy for diarrhea in 2006 in Honduras. Oh, and what they did was they took the five most commonly used remedies in these, all of these studies and they bundled them into one combination thing and they just gave just that combo to everybody with diarrhea.
Dr. Joshua Goldenberg:So no classical intake or anything like that.
Dr. Mark Davis:No classical intake, nothing like that, but still randomized and still oral rehydration therapy for everything, everybody, and it was like 200 or 250 kids and what they found was no difference from placebo.
Dr. Joshua Goldenberg:Really interesting. Didn't differentiate at all and how big was the study, how many people it was like 200 or 250 kids. Wow, interesting so what that means.
Dr. Mark Davis:I don't know. You know, I mean maybe that maybe it's a point towards her integrity that she isn't just trying to prove homeopathy works in every occasion, or?
Dr. Joshua Goldenberg:maybe, but that's like you could. You could take it either way, right, Cause like that's. So all these three studies was like 230 people and so like one more negative study to our point would negate the effect and say it was a publication bias, randomness thing or not publication bias, but just sort of random. Well, yeah, I guess publication bias, randomness. Or you could say, yeah, but the design was different and it wasn't true classical homeopathy and that's why it didn't work. And I mean that's just so.
Dr. Joshua Goldenberg:That's so interesting that really underlies this issue, that like you can argue both sides of this case like ad nauseam. And it's like every study you can come back with some. I mean, look at this. It's like, oh, you need replication studies. And I'm like, well, you know, if we look at three replication studies really just one big study and there's still publication bias and it's like, and then, yeah, so, and you're like, oh, one more 200 person study and it negates it all.
Dr. Joshua Goldenberg:Yeah, Is it really classical homeopathy? So, man, I you can see why I think the lay public and maybe not just lay public gets infuriated with scientists sometimes, because it's like you can really bring your own biases to the table and kind of argue both sides, which is so frustrating because I feel like the whole point of some of these standardized approaches is to have standardized ways of analyzing it, which, to the homeopaths point like standardized ways of doing meta analysis here is like is somewhat supportive. It's only when you use these unstandardized approaches where it's not and so it's like yeah, I don't know it's, it's, it's challenging.
Dr. Mark Davis:Okay. So if this wraps up, which it may, wraps up your homeopathy mini series, well, you have.
Dr. Joshua Goldenberg:But with no conclusion, but with no conclusion more confusion than we started with.
Dr. Mark Davis:So that's what I wanted to end on. If it's an end, Joshua Goldenberg, what do you think? Hmm. What, what's? What do you think all this means? Do you have a conclusion or just a big question mark?
Dr. Joshua Goldenberg:Oh, that's a good one. Um, I don't know, man. I feel like there is some pretty significant evidence that this is the thing you know like just standard clinical evidence, and I feel like to disregard that is like to play on such different level than I do with any other medical study, just seems disingenuous and like not really evidence based. And so I think like there's a difference. There's like my science-based mind and my evidence-based mind maybe, and my science-based mind is not pleased and convinced, but my evidence-based mind is like, well, this is probably sufficient for people to continue to use, even if I'm not comfortable doing it. I don't know, I'm kind of shocked to be here, but I think that's my, that's my honest truth. I'm going to lose, like all my EBM friends now. Thanks, man, thanks for outing me, but I don't know, I mean.
Dr. Mark Davis:I mean, I think I think what you're doing is you're you're remaining true to your pre-registered beliefs about the degree to which you believe in science. I admire it and I'm honestly. Maybe I'm just admiring it because I find myself in the exact same place where I'm like I still don't use homeopathy or like send my kids to a homeopathy or pediatrician, like you know, but I'll throw some Arnica at them and I think the science says it works, like what science we have seems to indicate there is something there.
Dr. Joshua Goldenberg:So are we at the we'll throw some Arnica at it. Stage of acceptance. Is that, is that where we are at?
Dr. Mark Davis:That should be a whole new phase of acceptance. The official throw some Arnica at it. The back of it.
Dr. Joshua Goldenberg:We started with back of the room mumbling to ourselves or louder, and now we're a throw some Arnica at it. What can? How can it hurt? Okay, fair enough, all right, sir. Well, that was a wonderful. I'm going to get an earful from Adam. He's going to be like you said, what on air? So maybe he has plausible deniability. Maybe that's what he thought was going to happen and he needed plausible deniability. That might be a thing he's very smart in that way, all right. Thanks for joining us.
Dr. Mark Davis:Thanks for having me.
Dr. Joshua Goldenberg:And thanks, as always, for being such a great listener and sending us articles to read and science to talk about, and we'll see you all next time listeners. If you enjoy this podcast, chances are that one of your colleagues and friends probably would as well. Please do us a favor and let them know about the podcast and if you have a little bit of extra time, even just a few seconds, if you could rate us and review us on Apple podcast or any other distributor, it would be greatly appreciated. It would mean a lot to us and help get the word out to other people that would really enjoy our content. Thank you.
Dr. Joshua Goldenberg:--tw- know we talked about some really interesting stuff today. I think one of the things we're going to do that's relevant. There is a course we have on Dr Journal Club called the EBM Boot Camp. That's really meant for clinicians to sort of help them understand how to critically evaluate the literature, et cetera, et cetera Some of the things that we've been talking about today. Go ahead and check out the show notes link. We're going to link to it directly. I think it might be of interest. Don't forget to follow us on social and interact with us on social media at Dr Journal Club, Dr Journal Club on Twitter, we're on Facebook, we're on LinkedIn, et cetera, et cetera. So please reach out to us. We always love to talk to our fans and our listeners. If you have any specific questions you'd like to ask us about research, evidence, being a clinician, et cetera, don't hesitate to ask. And then, of course, if you have any topics that you'd like us to cover on the pod, please let us know as well.
Introducer:Thank you for listening to the Dr Journal Club podcast, the show that goes under the hood of evidence-based integrative medicine. We review recent research articles, interview evidence-based medicine thought leaders and discuss the challenges and opportunities of integrating evidence-based and integrative medicine. Be sure to visit www. drjournalclub. com to learn more.