In this edition of the Dr Journal Club podcast, hosts Josh and Adam delve into the captivating realm of Ashwagandha, an adaptogenic herb renowned for its stress-alleviating properties. They explore a systematic review and meta-analysis of randomized controlled trials investigating Ashwagandha's efficacy in managing anxiety and stress. Despite promising findings suggesting significant effects, the hosts meticulously scrutinize the study's methodology using the Amstar 2 checklist, identifying deficiencies such as the lack of a registered protocol and unclear reporting on funding sources. However, they underscore the importance of considering the results while acknowledging the low evidence level and study limitations. They advocate for critical evaluation of systematic reviews and encourage listeners to utilize tools like Amstar 2 to assess the quality of evidence in research studies.
Learn more and become a member at www.DrJournalClub.com
Check out our complete offerings of NANCEAC-approved Continuing Education Courses.
In this edition of the Dr Journal Club podcast, hosts Josh and Adam delve into the captivating realm of Ashwagandha, an adaptogenic herb renowned for its stress-alleviating properties. They explore a systematic review and meta-analysis of randomized controlled trials investigating Ashwagandha's efficacy in managing anxiety and stress. Despite promising findings suggesting significant effects, the hosts meticulously scrutinize the study's methodology using the Amstar 2 checklist, identifying deficiencies such as the lack of a registered protocol and unclear reporting on funding sources. However, they underscore the importance of considering the results while acknowledging the low evidence level and study limitations. They advocate for critical evaluation of systematic reviews and encourage listeners to utilize tools like Amstar 2 to assess the quality of evidence in research studies.
Learn more and become a member at www.DrJournalClub.com
Check out our complete offerings of NANCEAC-approved Continuing Education Courses.
Welcome to the Dr Journal Club podcast, the show that goes onto the hood of evidence-based integrative medicine. We review recent research articles, interview evidence-based medicine thought leaders and discuss the challenges and opportunities of integrating evidence-based and integrative medicine. Continue your learning after the show at www. d rjournalclub. com.
Josh:Please bear in mind that this is for educational and entertainment purposes only. Talk to your doctor before making any medical decisions, changes, etc. Everything we're talking about that's to teach you guys stuff and have fun. We are not your doctors. Also, we would love to answer your specific questions. On www. d rjournalclub. com you can post questions and comments for specific videos, but go ahead and email us directly at josh at drjournalclub. com. That's josh at drjournalclub. com. Send us your listener questions and we will discuss it on our pod.
Adam:And welcome to the Dr Journal Club, where the questions are made up and the research doesn't matter.
Josh:You caught us in the middle of a deep conversation about different ways to make coffee, but I like that intro.
Adam:Why don't you make a French press? Why are you meal prepping your Keurig like a freak.
Josh:First of all, it's not a Keurig. As I explained to you, it's an espresso, which is another level of like Hoity-Toity. I don't know, I hadn't actually thought about that. Oh, I know why, because it's going to be cold, so well. Oh so, like you say, like have a hot water maker, make a pot of French press and then just yeah, that's actually a really good idea. I should probably just do that. That would have been a lot cheaper.
Adam:Yeah, like the French press I have is like metal, so like the glass ones suck. Yeah, they're classic, but the metal ones are nice.
Josh:I miss French press. I go through phases, like I did mochas for a while. You know those things they do in England.
Adam:I haven't done.
Josh:Yeah, they're like these little. There's water on the bottom, you put it on the stove top and it like percolates.
Adam:Oh, the mocha pot, yeah, yeah, yeah, yeah.
Josh:Yeah, yeah, yeah, yeah, yeah so.
Adam:My mocha pot broke, though I was pissed.
Josh:I know I love those things, but they don't always last forever. Anyway, what are we talking about today?
Adam:You know what you should get? You should get like a little sand, like convection oven, so you can make your own Turkish coffee.
Josh:Oh, so speaking of Turkish coffee, okay, last thing, I could talk about coffee all day long. My neighbors are Muslim and they basically live on Turkish coffee. They're from the Middle East and it's like we went over there a couple months ago and they served us Turkish coffee and I haven't had Turkish coffee since I lived in Israel when I was younger and I forgot how much I love it. They call it boats over there, which means mud, because that's literally what you're drinking the bottom anyway, and I just love the. I love the I don't know. I love the culture of it, I love the history of it, I love the ritual of having these little things, and so I ended up buying some. So I got them to help me find proper Turkish coffee, because I think it's like cinnamon that's mixed with it or something like that. There's a spice.
Adam:Not cinnamon, it is Cardamom, cardamom, okay.
Josh:And so they approved, and then I bought the whole apparatus and I did it for a little while. I really liked it. It is a bit of a pain to prepare it, but yeah, it's cool, because you have to wait for it to spoil, just so, and then you have to take off the top and anyway, now we're really wasting time. Okay, don't go listener. We're going to talk about research here in a second. So this is my third attempt to bring us back to research. Let the records show, all right. So what are we talking about there? Oh, ashwagandha, we're talking about Ashwagandha. Yeah, all right. Yeah, what do you got for me? Is it my paper? Yeah, you recommended it. You've been like every time. I've been like, let's do this, let's do this. You're like well, what about the Ashwagandha paper, josh? And now we're doing your Ashwagandha paper.
Adam:Well, that's because you want to keep doing these weird methods that don't matter.
Josh:Everyone that listens to this podcast is a methods fan. I just want to say that and we should have a poll. People should call in and be like yeah, no, I'm a methods guy. I don't know what this guy's talking about. All right, Ashwagandha back focus. I need to drink my coffee. All right, you start us off.
Adam:Can you spell Ashwagandha without having to Google it?
Josh:No, and why should I? This is an AI world. We don't need to spell anymore.
Adam:This is fair. I can't spell it either, but anyway. So this was a cool paper. At least I thought it was going to be cool. It was not. It was a systematic review.
Josh:Walk us through the context.
Adam:Yeah, it was a systematic review and meta analysis of randomized controlled trials looking at Ashwagandha supplementation for the management of anxiety and stress. And for those who don't know, ashwagandha is an herb and it's considered to be part of a class of herbs called adaptogens. And adaptogens are kind of these, they're pretty cool and sort of like they're non-scientific mechanism of action in that basically, if you're feeling low they'll help sort of like elevate you, and if you're feeling a bit stimulated they'll help kind of calm you down. So they adapt to your environment in that sense, and so there is actually quite a bit of evidence for Ashwagandha and that class of herbs. So another one would be like rhodiola for a number of things, but particularly what we're looking at today is the management of. I kind of focus on the anxiety. I don't really care so much about the stress, but I guess we can talk about both today if we want. Yeah, but that's what these authors did, was they kind of looked at randomized control trials and tried to summarize the evidence for them.
Josh:Yeah, and I think to your point. Like the reason they looked at stress is traditionally it's used as this adaptogen and so, like you would like clinically, like I would recommend adaptogens for people that come in and they're just like stressed out, like they may not like meet anxiety criteria but you know, they're just kind of overwhelmed with life and they need something to level them out. That's how I think about it. And yeah, I was taught the same way. I was always kind of curious, like clinically, like I've always found them very helpful personally and with patients. And then, but yeah, like the herbal explanation, I was always like I wonder if that's a real thing, like the sort of like leveling you out thing. What does that look like biochemically? And I don't know, but anyway, so yeah, I was, I was kind of excited about it. Why don't we go through?
Adam:The method.
Josh:The methods dive into the methods. Yeah, there's not a huge amount to go over in the methods. Well, we should do like pros and cons of methods in a little bit, but maybe first what I'm thinking we do is first just like talk about the results of what they found, and then we can sort of critique and go back to methods and see what we, what our take homes are.
Adam:Yeah, sort of sort of like long story short. They found that Ashwagandha, when pooled together, was beneficial for both stress and anxiety, and they did find that there was a dose response relationship where the more Ashwagandha that was was taken, the greater the improvement. However, the majority of the dosing was around like 300 to 600 milligrams, and then there was one trial that looked at 12,000 milligrams, and so when you actually look at figure four, it's just a straight lying down where there there the dose response is essentially linear, in that the more Ashwagandha, the greater the reduction in anxiety. I don't know if they reported about the safety of that.
Josh:Right, it seemed like a crazy amount.
Adam:However, it's not a dose that I am familiar with.
Josh:No, me, neither. And yeah to your point, I think that entire dose response relationship was driven by that single data point and I don't know that I necessarily would would trust it. But, interestingly, like you've got a bunch of these studies that use similar dose. So yeah, so they found, just just so people have a sense of the evidence base to support this. Like again, these are just, these are randomized control trials only.
Josh:And they found eight randomized control trials on anxiety with about 555 patients total and standardized mean difference of 1.55. So, like we talked about, that's usually usually considered like a large effect, statistically significant. And for stress, seven randomized controlled trials, also a large effects, standardized mean difference of 1.75, also statistically significant. So quite a relatively large number of studies that were identified. And again, these are randomized trials. I thought that was kind of cool, the grade level. So they did do a grade assessment, which is how we measure the confidence we can have in their estimate. So they're estimating this large effect on stress and anxiety. How confident can we be in that? They rate it as low, low certainty of evidence and they ranked down for inconsistency. That's because their heterogeneity marker was.
Adam:It was high. Yeah, it was like super high, I think 92% or something like that for anxiety and 83% for stress.
Josh:Yeah, and that's the I square. And so, just for the listener, you want like a 0%, like that would be awesome if you had 0%, but certainly under 50%.
Adam:But that also makes sense when you look at like the different types of trials that they looked at. They kind of used all. There wasn't only anxiety and stress. They looked at kind of a diverse population.
Josh:That's right, not just healthy people, all sorts of different clinical conditions, you know actually. So, speaking of they did some really neat things, like they did a heterogeneity explanation, like they looked at or exploration, they looked at different ways of dividing up the studies to see if it explained the heterogeneity which is what you're supposed to do, and they were actually able to explain like a decent amount of the heterogeneity I think to your point, like they got when they just bulked it by. Was it healthy people versus disease populations? They had like really clean, like okay, these are the apples, these are the oranges, type of thing. So I felt like they probably could have they don't necessarily, didn't necessarily need to rank down for heterogeneity, cause a lot of that, a heterogeneity they could explain.
Adam:And it also made sense if you think about it too, like higher doses in older people of trials that were of larger sample size and in people who had psychological disorders were seem to be greater responders than younger, healthier people of smaller trial size with lower dosing. Like it kind of just makes sense.
Josh:Yeah, it did, but I thought it was pretty intuitive. Look, the thing is we don't do this for money. This is pro-bodo and, quite honestly, the mother ships kind of eeks it out every month or so, right. So we do this because we care about this, we think it's important, we think that integrating evidence-based medicine and integrative medicine is essential and there just aren't other resources out there. The moment we find something that does it better, we'll probably drop it. We're busy folks, but right now this is what's out there. Unfortunately, that's it, and so we're going to keep on fighting that good fight.
Josh:And if you believe in that, if you believe in intellectual honesty and the profession and integrative medicine and being an integrative provider and bringing that into the integrative space, please help us, and you can help us by becoming a member on Dr Journal Club. If you're in need of continuing education credits, take our Nanceac approved courses. We have ethics courses, pharmacy courses, general courses, interactions, that's on social media, listen to the podcast, rate our podcast, tell your friends. These are all ways that you can sort of help support the cause.
Josh:So, basically, what you have is a systematic review of randomized control trials for Ashwagandha, low level evidence on large effect sizes for that and before we so, on its face, I thought it was a pretty good study. And then I like that they did grade, I like that they did heterogeneity exploration. So they were speaking to my heart. And then someone asked me in our evidence synthesis lab about how you evaluate systematic reviews. So I started talking about Amstar 2, which is this checklist for quality and systematic reviews. It's like, oh, let me just apply the Amstar 2 since I'm prepping for this pod, and it was like terrible.
Josh:Yeah, it had like. Well, maybe I'll give a quick. Are you familiar with Amstar 2?
Adam:Well, we talked about it with the last podcast episode with the blood pressure.
Josh:Oh yeah, that's right.
Adam:Yeah.
Josh:Yeah. So they had done their own assessments of the quality of the systematic review. So there's the quote unquote quality of the evidence within a systematic review, but then there's the quality of how the systematic review itself was conducted. That's what Amstar 2 is, and Amstar is basically all these brilliant Cochrane methodologists who got together and we're like OK, what are the most important domains to think about for systematic reviews? There's, I think there's like 13 of them and seven of which they say are critical.
Josh:So you could have a few non-critical domains and get a red mark and that's fine, you could still trust the results. But if you have even one in these critical domains, basically they view it as a critically flawed study and you need to be extraordinarily cautious. And of the seven critical domains, they got a red mark, at least when I did it on 1, 2, 3. So critically flawed turns out. And just for the listeners, like what these essential things are that you should be looking for when you're evaluating a systematic review, probably the most important was you pointed out in the green room was having a a priori registered protocol. And so they claim they had a protocol but it was not registered. I don't think at all, let alone before they started looking at results.
Adam:Yeah.
Adam:But, I think, I feel like that also doesn't really change sort of the takeaway from this study, because they're kind of already saying that the certainty of the evidence is low to begin with. It doesn't necessarily mean it's bad, it's just that, you know, can we really trust these results? Not really. We need more evidence that may kind of sway things towards the null or be consistent with these findings Overall, you know we only have a handful of trials that are kind of small. We have only 300 in the intervention group and 250 in the comparison or so in the placebo group. For anxiety specifically, the risk of bias was low.
Adam:However, you know we saw pretty consistently the in the imprecision in the results and you know it's really just a heterogeneous, heterogeneous population. There was really only two trials that actually looked at, you know, individuals with with an anxiety disorder. One trial looked at 500 milligrams per day, another one looked at 12,000 milligrams per day. All the other trials looked at people with insomnia, healthy adults, schizophrenia or schizoaffective, bipolar, and really what they did was, because of the standardized mean difference, they just kind of looked at the, the questionnaires that they used and kind of pulled the results from that. So I mean, I think you know, even without you looking at the amstard you can. You still have to kind of take this with with a grain of salt, and I don't. I don't think that the amstard really changes much for this, in this one in particular.
Josh:Yeah, it's interesting. It's like I was kind of thinking about that a bit when we were going over last the last one we did on blood pressure, you know, because you're looking at both the grade results, which is the confidence in the evidence, and then the results of the quality of the study. That's sort of coming up with that quality of evidence and so, yeah, so where do you like, where do you weigh? And to your point, Okay, let me like think through what you're saying. If you're like, well, the evidence level is already low, we're already going to have a lot of skepticism about this. And so, yeah, I mean that's an interesting point. So in this case, let me think this through for a bit. Yeah, they didn't register their protocol. What's the major fear with that? That they change their outcome measures or something like this and they're cherry picking. But, to your point, they're doing standardized.
Adam:Do you also difference? Yeah, do you also give them a little bit of credit for, like directly saying, we did not register this?
Josh:No, because I bet that was a peer review comment. That was like you need to say if this is registered or not, because that's part of like that's that's part of like the standard reporting. You have to say what your registration is. Or if it was not registered and I don't know, that was my read between the lines is they were forced to do that for publication because that's the to meet criteria. You know, that's kind of how it goes, like Prisma, sr, whatever is. I think that's part of the standard reporting. But so okay, so maybe. But yeah, so it's not like they're cherry picking outcomes, because they're basically pooling all outcomes about stress and anxiety. Stress and anxiety makes sense for an adaptogen to be looking at, so that doesn't seem too suspicious to me. Yeah, so I don't know. And then I didn't love their search. I thought their search was a little bit weak but they found like nine randomized controlled trials. That's a lot of evidence. Like is it possible there's a bunch of other studies out there that they didn't find?
Josh:I don't know, I kind of doubt it. Maybe, maybe, so, yeah, so I don't know. It's interesting to kind of think about and that's a good point because we really should not be thinking about these quality tools as checklist, right, Like we need to be thinking through how they would impact our interpretation, and so, yes, let me look at that. So the protocol maybe we give them a pass I mean we don't give them a pass but how that influences our thoughts about the evidence. Maybe it doesn't change it. Literature search we said maybe that doesn't change it. What else did I flag them for? They didn't talk about funding of the studies, or at least I didn't see that. That's sort of the new thing in Amstar 2 to really call that out. And we've talked about how that's pretty important. Yeah, and I can see how funding would be an issue because they said that they had no conflicts of interest to declare, so perhaps that's.
Josh:No, not them. Not them Of the primary studies they didn't report on.
Josh:Oh, right yeah yeah, yeah, so that's like two levels. So like, do the authors of the System and Review have conflicts? So they claim no, but then did they report on of these nine studies which had financial funding issues that need to be aware of? But in theory that should have been taken into account for the risk of bias assessment? Right, I mean, maybe, maybe not, so anyway, interesting. But I think your greater point is super valid, which is the evidence is already low. We're already not sure about this effect size. It is a large effect but it might shift tomorrow if someone actually publishes a large new randomized controlled trial with results that differ. So yeah, I think those are the main take-homes that I had for this study. So a little bit of a short summary, pretty straightforward for me. Anything else you wanted to touch on on this one?
Adam:I think we commented about this earlier when we were talking about this paper. I thought it was kind of interesting that on their PICO PICO stands for Population Intervention, comparison and Outcome their comparison was placebo or no intervention, and that really matters, because no intervention is kind of an intervention in a way. It's basically like our weightless control, which is not the same as placebo. But on their exclusion criteria they excluded trials without any placebo groups. So I'm just not sure if this was like a language barrier issue, because this was a publication that came out of Iran and so I don't know if this was sort of they just needed some help in translating to English manuscript or what, but I kind of thought that was a little bit contradictory. It's like are you, is your comparison placebo? Is it either, or Was it this an issue because it was not registered? So it was kind of just interesting to see.
Josh:Yeah, I agree, I saw that too, and my comment next to it was sloppy reporting or sloppy language, and it might have been a language barrier thing. For sure, because, you're right, that did contradict itself. And when you look at the table of the studies, all the controls are placebo. So I don't know if that's because that was actually the inclusion criteria or it just happened to all the placebo controlled and not active control. It's hard to say.
Adam:Right.
Josh:But while we're looking at that, just to describe some of these studies a little bit further yeah, so, like dosing, like you said, between mostly between 300 milligrams a day and 600 milligrams a day, 600 seems to be one of the more common dosing regimen, which is interesting.
Josh:And yeah, and one thing I would say too is just about Amstar, for a second is I was, as I was going over it again, to present on it. It's a really great learning tool, so, and it's designed for clinicians that don't necessarily have like methodological experience and it's designed to be done in like 15 minutes or less. So if you're a listener and you're primarily the clinician and you come across a systematic review that you're interested in and you wonder how much you know how well it is, how well conducted it is and how well done it is, grab the Amstar to checklist and the questions guide you through what to look for and, again, a little bit of practice. You're done in 15 minutes and you have a good learning tool and assessment. For you know, faith, we get that a lot right. People want to know is this a good study? And you know we can look at it. But I think it's even better if, like you can learn these instruments that are really not that difficult to, and they're designed not to be difficult to do.
Adam:Yeah, I would agree.
Josh:Cool, all right, anything else?
Adam:No sir.
Josh:That's it, Okay. So short one today. Interesting, straightforward, systematic reviews, some quibbles, but for the most part probably doesn't change our take home, which is interesting large effects, surprising number of studies, but low level evidence overall. All right, dear listener, thanks for checking in and we will see you next time. If you enjoy this podcast, chances are that one of your colleagues and friends probably would as well. Please do us a favor and let them know about the podcast and, if you have a little bit of extra time, even just a few seconds, if you could rate us and review us on Apple podcast or any other distributor, it would be greatly appreciated. It would mean a lot to us and help get the word out to other people that would really enjoy our content. Thank you, hey y'all. This is Josh.
Josh:You know we talked about some really interesting stuff today. I think one of the things we're going to do that's relevant. There is a course we have on Dr Journal Club called the EBM Bootcamp. That's really meant for clinicians to sort of help them understand how to critically evaluate the literature, et cetera, et cetera Some of the things that we've been talking about today. Go ahead and check out the show notes link. We're going to link to it directly. I think it might be of interest. Don't forget to follow us on social and interact with us on social media at Dr Journal Club DR Journal Club on Twitter, we're on Facebook, we're on LinkedIn, et cetera, et cetera. So please reach out to us. We always love to talk to our fans and our listeners. If you have any specific questions you'd like to ask us about research, evidence, being a clinician, et cetera, don't hesitate to ask. And then, of course, if you have any topics that you'd like us to cover on the pod, please let us know as well.
Introducer:Thank you for listening to the Dr Journal Club podcast, the show that goes under the hood of evidence-based integrative medicine. We review recent research articles, interview evidence-based medicine thought leaders and discuss the challenges and opportunities of integrating evidence-based and integrative medicine. Be sure to visit www. drjournalclub. com to learn more.