Embark on an insightful exploration of pediatric health with hosts Josh and Adam. In this episode, we delve into the nuanced impact of probiotics on strep throat prevention. Take a guided tour through the intricate landscape of K12 Streptococcus salivarius research, acquiring a nuanced understanding of scientific papers. From elucidating the concept of 'lantibiotics' to dissecting the intricacies of clinical trials, we unravel the complexities of evidence-based integrative medicine. Navigate the challenges associated with industry-funded research and delve into proposed solutions for fostering transparency in the scientific landscape. Engage with us at josh@drjournalclub.com to contribute to the discourse and be a part of reshaping our approach to pediatric health.
Learn more and become a member at www.DrJournalClub.com
Check out our complete offerings of NANCEAC-approved Continuing Education Courses.
Embark on an insightful exploration of pediatric health with hosts Josh and Adam. In this episode, we delve into the nuanced impact of probiotics on strep throat prevention. Take a guided tour through the intricate landscape of K12 Streptococcus salivarius research, acquiring a nuanced understanding of scientific papers. From elucidating the concept of 'lantibiotics' to dissecting the intricacies of clinical trials, we unravel the complexities of evidence-based integrative medicine. Navigate the challenges associated with industry-funded research and delve into proposed solutions for fostering transparency in the scientific landscape. Engage with us at josh@drjournalclub.com to contribute to the discourse and be a part of reshaping our approach to pediatric health.
Learn more and become a member at www.DrJournalClub.com
Check out our complete offerings of NANCEAC-approved Continuing Education Courses.
Welcome to the Dr. Journal Club podcast, the show that goes on to the hood of evidence-based integrative medicine. We review recent research articles, interview evidence-based medicine thought leaders and discuss the challenges and opportunities of integrating evidence-based and integrative medicine. Continue your learning after the show at www. drjournalclub. com.
Josh:Please bear in mind that this is for educational and entertainment purposes. Only Talk to your doctor before making any medical decisions, changes etc. Everything we're talking about that's to teach you guys stuff and have fun. We are not your doctors. Also, we would love to answer your specific questions. On www. drjournalclub. com you can post questions and comments for specific videos, but go ahead and email us directly at josh@ drjournalclub. com. That's josh@ drjournalclub. com. Send us your listener questions and we will discuss it on our pod. Hello and welcome to the Dr. Journal Club podcast. We are on a short schedule today, so there will be very little chit chat. We are going to jump right into the paper or close there too. Adam, how are you doing sir? Just quick chit chat, super fast.
Adam:I'm great. I am very much prepared for this paper. That is not true.
Josh:I happen to know Adam was pulling it up on his computer screen not 30 seconds ago, so we will be gentle on him. But I think it's a pretty straightforward paper and also I kind of like when we have done a couple of these where we have done them off the cuff, and I kind of feel like it models what we look for to the audience. So that's good, I like it. Okay, but I'm prepared, so at least one of us is prepared, so I think we will be all right. So I will just have to carry the podcast today, whatever. What else is there?
Adam:As per usual.
Josh:Yeah, all right. So this paper today which, Michele, I will send you the link this time and here is the challenge, Josh, can I get it to her before she's like, "Josh, we need to publish, Can you send it to me too? So I'm going to get it to you today. This was sent by a patient who has some, you know, young kids and, as I can attest, because we just went through this this season, little kids like to get strep throat and they like to get it again and again and again, and they like to give it to everyone in the family, and then everyone has to stay home from work and home from school, and then you have to get rid of the toothbrushes. And then did I miss a day? Do we need to get new toothbrushes, replace it again?
Josh:So, gone through all of this, so I totally get it and she was like, hey, what about probiotics to prevent this? And I was like, oh, that's interesting. I'm kind of curious. I do know, you know, have you seen anything? And she sent me a couple of articles I have the best patients, by the way, they send me articles and this one looked interesting on its title and I was like, yeah, cool, we'll do it on the podcast because I'm curious myself and for clinic Not that I do a lot of peeds and we'll talk about it. So that's how we got this paper today. That's our preamble, and also me stalling for Adam to read through.
Adam:Oh, I'm supposed to be reading.
Josh:Oh, man, you guys, you're in a treat. You're in for a treat today, dear listener. All right, so, Adam, where shall we jump in? And so, all joking aside, not giving you a hard time, I really do think that, like modeling how we freshly look at papers is useful to the audience. So let's go through that. We've done this once or twice before. What's the first thing? You would walk me through your thoughts as you look at it with fresh eyes, and then I can weigh in if there's anything I saw different.
Adam:Well, this is strep throat. Well, specifically, this is actually strep tococcul, viral pharyngeal tonsillitis in children. So we don't need to read the introduction because it's a pretty common occurrence. So let's save some time and just skip the introduction. Basically, just jump straight to the methods. Okay.
Josh:Hold on, hold on, before you do that. I know you don't like the intro, I sometimes do. There's a few interesting things that I saw. One is this phrase lantibiotics. Look at that, second line of the introduction. Have you ever seen that before?
Adam:I have not.
Josh:I'm like am I just a total idiot and miss that day in natural pathogen medical school? I have never seen that.
Adam:It's possible.
Josh:It's possible. Yeah, but I was like, first one thought it was a typo. I was like, did they put a one in front of antibiotics and it didn't get checked? And then it was again and they've never defined it. Anyway, I googled it. It's a type of antibiotic peptide, specific made by different bacteria. I thought that was interesting and also a little presumptive, I must say, of the authors to assume that we all know what the heck that is without defining it, and I think they would have been just fine saying antibiotics. I think they're flexing a little bit, but anyway. So that was one thing. And then the other thing was just that they, and that is a good background to do it set up the previous evidence for this strain. So this is using the K12 streptococcus salivarius strain as like a lozenge to prevent streptococcus, so strep throat and viral fringiotonsulitis in kids. And yeah, I think that's all that I have to say about the intro. I'm ready to jump in now.
Adam:All right, cool. So yeah, like you said, it looked like it was a the K12 probiotic strain and it looks like it was actually formulated in Europe and looks like there's some stringent guidelines with which that they actually have to adhere to as it pertains to Italian law, and apparently this supplement needed, excuse me, could not have had less than one billion colony forming units of this probiotic strain.
Josh:Yeah, so it's K12 bliss technology, so I think so. I actually have a probiotic lozenge I used just for like dental health and I checked it out and it does actually have this strain, so it looks like you can get it in the US and yeah, so it looks like for this study. One billion CFU is the dose that they used.
Adam:Okay, and they also said it was a multi-center, open, non-randomized controlled clinical trial. So I mean, kind of off the bat, we've got some bias right there.
Josh:Yeah.
Adam:It's open labeled, meaning the providers or the people who are involved in the study know what they're getting and the participants know what they're getting. It was non-randomized, so that you know always can cause issues because you're not essentially equating for sort of prognosticating factors. So, and yeah, go ahead.
Josh:No, go ahead, oh I was going to say, yeah, it sounded great until we started reading the methods.
Adam:Yeah, Right, yeah, there you go.
Josh:So yeah, so right off the bat, so open, so not blinded and not randomized, which I thought was kind of interesting.
Adam:And it was done in 61 pediatric individuals, so a pretty small trial. And they said it was multi-center. Sometimes I feel like people will say multi-center but really it's just like two clinic locations. Was this truly like multi-center of, like several pediatric clinics that spread across the Milan, you know, metropolitan area, or was this really just two clinics?
Josh:That's a good question. I don't think they actually defined how many clinics they had. That's a good question.
Adam:And they were. Of those pediatric patients, there were 32 boys and 29 girls.
Josh:Let's jump into inclusion criteria. I'll tell us more about the population.
Adam:Yeah, so everyone who was enrolled had to be between the ages of 3 to 13 years of age. They had to have no less than 3 episodes in the same quarter as that of the study of the previous year.
Josh:Yeah, so a little oddly worded, but basically these are kiddos that get recurrent fringiotonsulitis, and so much so that the season previous they needed to have at least 3 times to be eligible. So this is a very specific population.
Adam:And they also had to have you know strep infection confirmed by a rapid swab for either group A or group B strep and had no other sort of infectious disease at the time of diagnosis?
Josh:So they have a history of the stuff, but they couldn't have it coming in, which makes sense. And yeah, and they did objective lab testing, which is pretty standard. These sort of rapid swabs I remember those, try giving that. So ironically, my two year old handled it like a champ. I almost gagged all over the provider and then my six year old like literally ran across the room and it was quite a challenge. But yeah, they are not fun for kids. Yeah, they are not fun for kids to get swabbed, or grownups for that matter. But yeah, but you've got some objective data about. Was it indeed strep throat in particular? Cool Alrighty.
Adam:And then also importantly, the exclusion criteria. Who was also not allowed into the study was basically anyone who just otherwise wasn't really just a typical healthy kid who gets strep infections. So they excluded people who had immunocompromizations kind of status, had undergone a tonsillectomy, so a removal of it, or removal of the adenoids, anyone with rheumatic disorders, asthma or allergies. So really this is just like your typical healthy kid who just gets strep throat infections.
Josh:Yeah, any concerns with the inclusion and exclusion criteria from your side?
Adam:Not really.
Josh:Yeah me, neither. I like that. I think it would be a population you would think about prophylactic stuff for. Yeah, just a little bit of a heads up, like you said, on the non-randomized and open. But the population looks legit to me. All right Onward.
Adam:All right, and then basically they were separated into one to one, so one kiddo would get the probiotic and then another one would just go into the control group where they didn't receive any treatment, which is kind of a bit frustrating, because it doesn't look like they're even being like compared to a placebo. It looks like maybe just like a weight listed group. They're not getting anything and yeah.
Josh:So, like the method, issues are like beginning to emerge right, like so, first of all, they do one to one, but they don't randomize, which is interesting and also this is like kind of they didn't.
Adam:So is this more of like a prevention study? Yes, Okay.
Josh:Yeah prophylaxis.
Adam:Yeah, okay, okay, okay, because the next sentence was that the product was to be administered for 90 consecutive days. Yes, and so I didn't know if this was like treatment four or for prevention, okay, cool.
Josh:For prevention. Yeah, yeah. So no, no to your point. No placebo. Basically it's just the kiddos either get the probiotic or not, and it's apparently one plus one enrollment. I don't know what that means. It says an enrollment key, but it specifically says it's non-randomized. So I'm just very curious about why they would not randomize, but also how that allocation is actually happening. And of course, the concern with non-randomization is that they're being selected in a specific way, like oh, you're going to get this, you're not going to get that, et cetera, and that the populations are different, like healthier in one group versus the other. So, yeah, so a little bit of a lot of questions there. But, yeah, awesome, all right, you're doing great. Keep them going. Rockstar status.
Adam:And then they let the probiotic dissolve underneath their mouth right after brushing their teeth before going to bed, and we're told not to chew or swallow it whole and could not drink or basically eat anything afterwards.
Josh:Yeah, so you want it. It's for the microbiome of the mouth. It's not like a probiotic you would take for the gut. This is the I usually call it lozenge. I think they called it a dissolvable tab, but essentially you want it to coat the mouth and the oral microbiome. That's what we're after With the idea of and again, this was in the introduction, which is kind of cool. So usually you're right. I feel like there isn't a lot in the intro, but apparently there are two substances salivaricins, I don't know if I pronounced that right that are antimicrobial that are made by this particular strain, and that's the proposed mechanism is that this probiotic will take up, residence, release these antiviral lentibiotics and that's why strep won't grow. Essentially, that's the proposed mechanism.
Adam:Great. And then basically they told them just to take the product as otherwise instructed to, but if they were to have any sort of symptoms of strep throat, to come back and then, if they had a positive test, would get a treatment of augmentin for 10 days, and then we're actually told to restart to the probiotic after completing treatment until basically the end of the 90 days.
Josh:OK, so I got a red flag here. How about you?
Adam:I thought that was kind of unusual.
Josh:Yeah, yeah. And but think too about how they came into clinic. Like, did they test everybody every month or so to see if they had strep? When did they test? Who got tested and when did they test?
Adam:Basically it's kind of like subjective, like if you think your kid has strep throat, bring them in.
Josh:Yeah, and it's not blinded, right.
Adam:Yeah, and it's not blinded.
Josh:And it's not blinded.
Adam:Yeah.
Josh:Yeah. So now you've got this potential placebo effect where parents who know they're not getting anything or parents that perhaps are getting some of this play this out.
Adam:Yeah, you're likely to go to go be seen and get treatment, as opposed to someone who's getting some sort of an intervention, even if that intervention actually isn't doing anything.
Josh:Well, so maybe right. So that's one possibility. Or if you're getting the intervention, maybe you don't come in because you're like well, I don't know. You said you're a little bit unwell but you're taking something and is probably covering it. I don't think you're getting infection because you're taking medicine, right, and so I'm just thinking about how this could skew the results. So if you are coming in less often because you're taking the probiotic, then you're going to undercount strep and make it look better. On the flip side, if you are coming in less often because you are not getting it, then that would basically undercount those cases.
Josh:So one thing that we should think about is it would be really great to see a table of the percentage of people from each group that came in for an assessment. If it was balanced, I would feel better about that. But then also, you're just like well is it? Are they not coming in because of the placebo effect? Are they not coming in because it's actually working and you can't tell that? And that's why you need a placebo, you need to blind it, and or you need to be going to their door and swabbing everybody all the time. That's the other thing. But because the outcome, which is, the number of positive swabs depends upon the number of people that show up to get swabbed, and because that number of people that show up to get swabbed can depend upon your knowledge of whether or not you got the probiotic, it sets us up for dramatic potential bias. So that was a really big red flag to me and underlines why you need placebos to begin with for this sort of business.
Adam:Yeah, so I mean, one thing I wouldn't discredit people for would be to just stop at this point, and there's just too much risk of bias going on, that really the results are going to be taken away.
Josh:You're already like throw it out, throw it out, yeah.
Adam:I mean, I'm kind of already there of like this let's throw this out, but we'll entertain our fans.
Josh:It'll be a good learning experience. So I think this was actually turned out to be a really good learning experience, so let's keep on going.
Adam:Skip the part where they kind of defined how they diagnosed viral infection and just get to the objective and the results because of time, but basically what they did as like, their primary outcome was the efficacy of the probiotic supplement in preventing infection during the study period, as well as reducing viral infection during the same time period. They also looked at the onset of side effects or toxicity while the product was being administered. And then for secondary objectives which I still thought were clinically relevant, was basically the frequency of antibiotic use, frequency or reliance on antipyretics, so basically using Tylenol to reduce the fever. This one is actually really important. Working days lost by parents so having to call out of work or go to work later or something like that because they have to stay home with their kid and then absence from school for children.
Josh:Yep. So I agree, I think the outcomes were excellent. I think the way they defined the infections were good, perfect, ok, great. And then I don't think there's anything we really need to cover in the statistical analysis. The only thing I would add from a methods perspective I did not see any mention of a registration of a protocol.
Adam:Yeah, I didn't see that either.
Josh:So that's going to be another red flag, right, because people can always tweak how they decide to analyze stuff after the fact. Look, the thing is we don't do this for money. This is pro-bono and, quite honestly, the motherships kind of eeks it out every month or so, right? So we do this because we care about this, we think it's important, we think that integrating evidence-based medicine and integrative medicine is essential, and there just aren't other resources out there. The moment we find something that does it better, we'll probably drop it. We're busy folks, but right now this is what's out there. Unfortunately, that's it, and so we're going to keep on fighting that good fight.
Josh:And if you believe in that, if you believe in intellectual honesty and the profession and integrative medicine and being an integrative provider and bringing that into the integrative space, please help us, and you can help us by becoming a member on Dr Journal Club. If you're in need of continuing education credits, take our NANSIAC-approved courses. We have ethics courses, pharmacy courses, general courses, interactions, that's on social media, listen to the podcast, rate our podcast, tell your friends. These are all ways that you can sort of help support the cause. Okay, so that's all of our method stuff. Let's jump into the results.
Adam:Right, okay, so when we look at table I think it was, yeah, table one looking at the baseline characteristics, there were no statistically significant differences between the two groups, but they didn't really look at many different characteristics. Both groups had 30 participants. The group that received the probiotic had 19 males versus 13. Age was essentially the same between the 2 groups and then the episodes of pharyngeal tonsular infection was similar between the 2 groups, with both groups having about three I guess 3 per quarter.
Josh:Yeah, in the previous year. So this is the type of table you would look at to see if randomization worked. Of course they didn't randomize, so their counter argument could be yeah, we didn't randomize, but these groups looked the same. But, to your point, they only really report on 3 variables. Sex, age and previous episodes per cycle, the only one that I would. Well, I mean, I guess theoretically sex and age can be predictive, but episodes per child would probably be important. But yeah, there weren't a lot of variables here, and so again, we don't know if these groups are equally balanced and the fear, of course, is, without randomization they may not be All right, keep on going.
Adam:And also another reason why it's important to have a trial registry is because in that registry they could have outlined what those baseline characteristics were going to be, and so they may have had more and just decided not to report on them.
Josh:Yeah, exactly exactly right, exactly right, and we have no idea because they didn't register their protocol. Exactly right, okay, cool, what else? Anything else? All right, keep on going. What's the next thing that comes to mind?
Adam:I'm trying to look at table three and make sense of that, just because that's the next one.
Josh:Yeah, so okay. So for the sake of time, let me jump in here and kind of go through the results and then we can talk about it. So let's look at table 2 first. So table 2 is going to be reports, the number of treated, you know, infections the previous year, which is about 90 for both, and then it talks about the year with the preventative probiotic. So you have 90 basic infections, treated infections the previous year. But in the in this year when you take the probiotic, if you take the probiotic, only three people whereas last time it was 90 only three of them got treated for an infection, whereas before in the untreated group got treated very similar to that 90. So you have this dramatic reduction. So very little change if you for the untreated group from one year to another, which would make sense, and you have this dramatic reduction, a 96.79% reduction in treated infections in the treated group. So that looks like a dramatic effect size of a very, very effective intervention. But but yeah, go ahead.
Adam:But we have a kind of circles back to what we were talking about before with why we're concerned about not having a placebo.
Josh:Yeah, exactly right. So when I I actually and I know you like to not read the abstract on purpose, which I think is smart I read the abstract first on this one, and the first thing that put up a red flag for me actually was the effect size. Yeah it was way too good. Yeah it was way too good and I was looking for issues from the get go because of that. And I mean 97% reduction really. I mean maybe, but really. And then you have all these methodological issues and you're like huh.
Josh:And then table three, just for sake of time. This is the viral for angiotensilitis and we see a similar story in the previous year. About 25 people in both groups had viral infections and then in the year where there was treated, the people that didn't get treated, just about the same number 24 as opposed to 28. But if you got the probiotic, only five people got viral infections compared to 25. The year before. So again, another dramatic improvement in 80% reduction. Very, very good. Is it too good to be true? Is is the question.
Adam:Well, the answer is yes.
Josh:All right. So let's let's kind of close up here. There are a few other things I wanted to touch on and then let's chat a bit. So we talked about the actual effect and then here's here's some lines that were just more red flags for me, so for the listener, as a learning example. First thing that was a red flag to me was reading the abstract and seeing the effect size just looked too good to be true and I could tell from Adam that there were red flags with non randomization, no placebo group, and then we had this sort of interesting enrollment piece, etc. Etc. We didn't have a registry, so all these sort of red flags. Here was another red flag for me. Here's a line the treatment. Let's see it was well tolerated and without any side effects worth mentioning. It's just the strangest.
Adam:It's just perfect. It's the perfect treatment. It cures everything, Josh.
Josh:Here's everything, but no side effects worth mentioning, don't you think that's kind of an interesting language like? It didn't sound very much like a regular paper to me and I was also like, well, who decides? What's worth mentioning is that to find a priori we don't have a registry, so that was already kind of a red flag. Compliance was very good. And then they go through and they talk about the benefits from a loss of school and a loss of work perspective, which obviously makes a lot of sense. It's going to be dramatic Because you have so much fewer, so many fewer infections. There is less days lost of work. One other cool thing they did Again just going fast here, skipping over some details is they looked at like a financial analysis like how much it costs to take the probiotic for everybody for three months and then how much was spent on antibiotics and on antipyretics etc. And it turned out overall, when you do that it's obviously more expensive to take the probiotic. But then they say about well, yeah, but to save this many school days and this many work days and it's really just like three, was it 30 cents a day to prevent it? You know sort of this, this interesting kind of calculation about the benefit. So they do some financial analysis there as well, sort of back of napkin analysis to, so we don't have to then go through the discussion and it's just sort of more and more waxing poetic about how amazing it is.
Josh:I have a note here on the side reads like an infomercial oh, here we go, here's a great one. As each family spent 30 euros more than those who did not resort to the K-12 strain, it can be stated that each saved day of work cost about two euro per family. Hiring a babysitter to avoid losing a working day will certainly cost much more than that. It was just like it just did not sound like a proper research paper. Maybe English wasn't their first language and these are translation things, but it just again lots of, lots of red flags. It read to me like a infomercial. Now, with everything we've said, what's your guess about the affiliation of the authors?
Adam:Well, actually it was gonna. I just looked at that. I was gonna say that and that a big probiotic was involved in this and the lead author is the main formulator of the tested product and involved in the scientific council of the company that tested the product.
Josh:So this has no, no, even worse that trait it's trading the tested product, aka the sellers of the product. He's on the scientific council of the company that owns it.
Adam:This just has the red flags all over it, yeah.
Josh:Yeah, it does and does, and so this is a classic exam. I'm gonna use this as a teaching example for now on. This is a classic example of like I don't know I need to come up with a good phrase. It's like when industry Not trying.
Josh:Yeah, it's like industry wants to say that there's like research behind their product and it's peer reviewed and all that, and then it's basically an infomercial for the product and I just feel like it's science washing. It's like science washing essentially, or peer review washing or something. So, yeah, it is a classic example of industry funded research. Now, that, of course, does not mean that this stuff does not work. I would be willing to bet that in real life it doesn't work as well as advertised, but the mechanism makes sense. It may work very well.
Josh:This is a massive effect size. So even with all this bias, it could be that there is a very reasonable beneficial effect. All of this to say we have no idea if that's true or not because of the methods, and that's why methods are important and that's why we have to be cautious when we. If I don't know where the where my patient found this, probably, well, I don't know, I'll have to ask her but a lot of these companies. I just went the other day to recommend a supplement and went to the website and they're like science. Click here, of course, click there, click bait for Josh, and it lists all these peer reviewed studies. I'm like, ooh, there's some studies behind this, but you always have to ask which studies are they gonna list on their website and who funded those studies Anyway. So it's a good. I thought it was a good teaching example.
Adam:Do you think that industry should start using like reverse psychology and it'd be like our results are at best? How does that? How does the advertisement?
Josh:Yeah, I mean it's so. So I talked a lot about this. I gave this talk at CRN about this too. It's like if industry so let's assume the best about industry, just for sake of argument, let's steel man them right. So it's like you want you believe in your product, you want to show that in a rigorous way A lot of scientists involved in these supplement companies, and so you want to do research and show the world that your stuff works as well as you know it works. And so the question is how do you do that in a way where we're not laughing about it later Like we are right now. Right, we're not like, oh man, we can't believe a word of this. Is there a way to do that?
Josh:And there were a few recommendations that I had in the talk. I'm curious what your thoughts are. One, of course, is just to write a check and walk away, and you see that in the big New England Journal of Medicine, studies like on the drug manufacturers, and the other is to at least have a registry. I think that saves so much of these issues. Right To have an a priori protocol with reasonable outcomes and then published ahead of time, and then we can all compare it and a lot of this P hacking, a lot of the outcome manipulation that we see I think is gonna show up, and the journals have to stop publishing papers that don't have protocols at all, let alone a priori protocol. I don't know how this is still a thing.
Adam:I prefer to cut the check and walk away with the caveat that they're publishing results no matter what and not getting their hands like all up in the manuscript. Yes, my concern with the former option is that oftentimes you'll see like protocol deviations and then justifying why they did a protocol deviation. So it's like oh yeah, we did do a protocol registry, but because of XYZ we actually had to do this other thing instead, and so then it kind of does become a little bit murky, especially if you see it happening a lot. So I would rather just accompany you like yeah, we just donated $400 million to this project.
Josh:Yeah.
Adam:And then we just walked away and whatever it is, it is.
Josh:So I used to feel that way and I still feel mostly that way. Some people came up to me after that conference I was talking about who were in industry. They're industry scientists, and they were like they told the story about how they had done that, written it. They really believed in their product. They wrote a check and walked away. They gave it to reputable scientists and when they saw the paper the same time everyone else did they were appalled because, according to this person who was talking to me, the methods just stunk and it made zero sense based on the mechanism of how they thought this thing worked, and they were very, very frustrated and the result was negative.
Josh:And so I can see an argument where, like, if you know your product really, really well, like you should at least have a voice at the table in methods development, maybe not a voting voice, but at least being able to say like hey, here's our chief scientist, they know this product better than anyone else in the world. Like, let's at least let them have a presentation about why they think this method is a bad idea, et cetera. So yeah, anyway, I think that made sense to me when I heard that story and I could totally see that happening. We see that in integrative medicine, where non integrative researchers design these studies that make like zero external validity sense.
Adam:Yeah, I mean, I think what they could do then is just like hire on or not necessarily hire, but like consult an independent methodologist who can. So you know how, like sometimes in meta analysis, they'll have like two reviewers and a third for any sort of disagreement. Yeah, you could kind of be the same thing from a method standpoint, of like have two methodologists and a third, independent, to resolve any sort of differences, where one could be the methodologist from the research team, one could be from industry and then one who's not related to either.
Josh:That's an interesting idea. Have the methods be independently drafted that's cool. I've never heard of that and I've never thought about that, but I think that's really interesting Because, yeah, if you have independent methodologists and they come up with totally different ways of analyzing it, yeah, that's very cool. Need idea and just more ways of? Yeah.
Adam:And not necessarily coming up with something different, but being like yeah, I know that the industry methodologists' recommendations make more sense because of these reasons, or oh, I see what you're saying. Or no, we can't do that, I see. Yeah, so they're necessarily have to make new methods, but kind of serve as an arbiter, if you will.
Josh:That's a neat idea, okay. So basically you have an independent panel design the study. Then you have the industry group saying this is a terrible idea and this is why. But then, instead of the other group getting defensive and saying, well, they're just saying that because they want it to look good, you have an independent person looking at it and saying, no, industry's right about this. That is really cool. That's even cooler than what I thought you were proposing. Neat and more realistic too. I don't know why that's not done. Maybe that is done and I just never heard about it.
Adam:And you could even like blind them to it. So this way you could sort of blind the authors on the paper and you could blind where the funding is coming from. So this way they don't know, yeah, and there's no influence.
Josh:I love that, really, really love that. Cool man. Well, great, I think we got through it in time with some time to spare. Thank you, dear listeners, for listening. Thank you, dear patient, for recommending the paper. I thought it was a really good learning example for all of us and I think it worked out really well, Adam, that you didn't read ahead of time, because I think it was really cool for the audience to see and I could see in your eyes and then the audience could hear in your voice, like as the red flags were accumulating, and then by the end you were like oh yeah, I am not surprised at all what the affiliation here is with the authorship and I think that is exactly. You know, my thesis on how people should read papers is like look for those red flags and just it doesn't mean it's bad, it just means look deeper, look deeper, maybe a second, something's off here. You could tell you're like wait, that doesn't smell right. And then it would just kind of build and build and build and build. So that was awesome, well done.
Adam:Yeah, are you someone who sits through a movie, no matter like how bad it is, or do you have the? Do you have the wherewithal to just stop and be like this is terrible?
Josh:Oh no, I definitely stop. Yeah, I do have to finish books and I have to finish podcasts. That is a problem. That's wasted a huge amount of time in my life years and I cannot move on until it's checked off the list. But for some reason, movies I don't have that problem. How about you?
Adam:I can stop all three of them and just move on with my life, and I think that people, when they read papers, need to do the same thing of like no, yeah, these are awful methods, I'm done.
Josh:You said that, two lines into the method section I think that was when about when you made that like, oh, we could just stop here. We can't believe a word. That comes next and yeah, fair enough, and you were right. So awesome, man. Well, thank you for another great pod and, dear listeners, thank you for listening.
Josh:Oh, one thing I've been wanting to say that we haven't said yet is a lot. So we have a lot of institutions that are members of Dr Journal Club. So if you are part of a university, faculty students, et cetera, or even large clinics, reach out to them and ask them about we call it institution level subscriptions to Dr Journal Club. Lots of universities and state associations, naturopathics, state associations, chiropractic associations are members, and that basically provides free access to all of their membership. So if you are a member of a university or an institution or a clinic and you don't feel like you can subscribe as an individual, remember that institutional subscription is an option as well. That is my disclosure and connection. I am owner of Dr Journal Club, so I want to be transparent about that. But also it's an awesome product and you should come and listen. All righty, take care everybody, and we'll see you next time.
Josh:If you enjoy this podcast, chances are that one of your colleagues and friends probably would as well. Please do us a favor and let them know about the podcast and, if you have a little bit of extra time, even just a few seconds, if you could rate us and review us on Apple Podcast or any other distributor, it would be greatly appreciated. It would mean a lot to us and help get the word out to other people that would really enjoy our content. Thank you, hey y'all. This is Josh. We talked about some really interesting stuff today. I think one of the things we're going to do that's relevant. There is a course we have on Dr Journal Club called the EBM Boot Camp. That's really meant for clinicians to sort of help them understand how to critically evaluate the literature, et cetera, et cetera Some of the things that we've been talking about today.
Josh:Go ahead and check out the show notes link. We're going to link to it directly. I think it might be of interest. Don't forget to follow us on social and interact with us on social media at Dr Journal Club, DR Journal Club on Twitter. We're on Facebook. We're on LinkedIn, et cetera, et cetera. So please reach out to us. We always love to talk to our fans and our listeners. If you have any specific questions you'd like to ask us about research, evidence, being a clinician, et cetera, don't hesitate to ask. And then, of course, if you have any topics that you'd like us to cover on the pod, please let us know as well.
Introducer:Thank you for listening to the Dr Journal Club podcast, the show that goes under the hood of evidence-based integrative medicine. We review recent research articles, interview evidence-based medicine thought leaders and discuss the challenges and opportunities of integrating evidence-based and integrative medicine. Be sure to visit www. drjournalclub. com to learn more.