A Step-by-Step Guide to Reviewing Evidence in Aesthetics

On today's episode, we'll go back to basics – how to read and digest a research article. We'll cover the hierarchy of evidence and the need-to-knows for when you are reading and critiquing research.

Episode participants:
Radiance Wellness | @radiance.well
Megan Ucich | @the.skin.practitioner
Aayesha Patel | @aaestheticsbyaayesha
Sarah Pertschuk | @beautyinjectorsarah

Episode show notes / blog: Link

Disclaimers & disclosures: Link

Transcript

0:00

We would get into the bowels of the study and then the sample size would be like 11 rabbits. Like wait, there's not even on humans. Yeah. So, okay. Welcome to AP class, your virtual journal club for all things aesthetic medicine. Created by injectors, for injectors this is where we take a clear evidence-first look at the research shaping modern aesthetics. Each episode we'll unpack a study. What's solid? What's questionable? And how the evidence informs natural, refined results in real practice. Class is now in session. On today's episode, we'll go back to basics, how to read and digest a research article. We'll cover the hierarchy of evidence and the need-to-knows of when you are reading and critiquing research. Okay, welcome to the first ever episode of AP class. Yay. We finally are making this happen. This has been a long journey to get us here, but we're doing it today. So, for those of you that don't know, my name is Megan Ucich. I'm Aayesha Patel. I'm Sarah Pertschuk. This podcast is by injectors for injectors. And the goal of AP class is to bring more evidence-based medicine into practice. We know this is something that's been kind of lagging in the field for a long time. It's gotten a lot better, but I think for so many of us who are talking about it, NP school, a um PA school was a long time ago for all of us. So the last time we really had to sit down and analyze and critique a research article was a while ago. Yeah. So, our hope in this is to every episode give you a new study that we think is applicable to aesthetic based medicine. We will break it down a little bit, go over the evidence, discuss if we think that it's applicable to practice, and hopefully this just helps other providers that want to be more evidence-based in their own practice feel like they have a resource available to them. Yeah, I agree. I think the goal of this is to kind of wash out all that noise that's out there. Um Sarah was talking about how there's like so many quick fixes and trends and hopefully this, you know, platform will bring more evidence. We can talk about our experiences and our insights in practice to just make aesthetic medicine more trusted and safe for patients and providers. Totally. There aren't a ton of studies out there in aesthetic medicine to begin with. So, we get a lot of our information from Instagram, Patreon, YouTube, wherever. We look for resources and we take the the word that those clinicians or providers are giving as the word. This is truth. But even though these providers are incredible providers and have a lot of experience, we still need to look back at where the medicine started and what these what the research actually says. Totally. I feel like we're in a weird space where, and I've talked about this before, but like in our profession, for some reason, we seem to be on social media so much. It's like where our patients consume information and where we as providers consume information. and it's very murky. And so, yeah, science maybe isn't as sexy and talking about research is isn't as fun, but I think we can make it digestible and make it more relatable to people um and try to take us back to where we should be, which is the practice of medicine. Yeah, I think it's time we reel it back because I remember when I first started as a new injector and when I was getting trained, it was watch YouTube videos or educational videos on Instagram and I was like, how am I practicing by just watching videos and like where is the base of our knowledge coming from? Um, so I think it's time we take it back and actually look into our research more. Totally. And while this AP class is for providers, it's also for patients alike because they want to know that our information is sound. That we've gotten it from research papers, the basis of science, not just from a YouTube video or something that we saw on Tik Tok. While there's still a place for that. Absolutely. We still we want to make sure that science and education are the number one as well as safety. Yeah, absolutely. It helps to legitimize our profession. Again, I think that sometimes people think that this is like a beauty treatment and they don't really understand the amount of education and experience that we have and why that really matters. So, I think this will be good for consumers to see too the nuances behind everything we do and how much thought goes into the services that we offer here and how we provide those services. It's not just a willy-nilly, let's try this out. And we still use Instagram. Like we have our own like Radiance Wellness group chat that's like my brain hurts but this feels important. And we'll put our little like important. Yeah. But then we'll like try to dig into it a little bit deeper. So I think there's a place for both. But for sure going back to science, going back to evidence is most important. So we're going to kick this off with just a crash course in what is research again? Because even for us when we were preparing for this, it was like a crash course for us too. Like okay, what are all of the different research studies? How do we know if something's statistically significant? So like that's so important because this is the other part. There's lots of different research out there, but that doesn't necessarily mean that it's good research. So statistically significant or statistically significant, does it actually relate to practice? Can it be repeated over again? So we're going to go through that a little bit. The next episode will dive into more research, but this will be available for people to always kind of lean back on as a resource if they're trying to read more articles and understand how should I break this down and be looking at it. Cool. Let's take it away. I feel like you guys had a great way of starting it. So, I'll let you guys. I think before like diving into a bunch of papers, you kind of want to frame a question like what are we looking for and why are we looking for it? So I read a lot about the PICO framework which stands for like the population intervention comparison and outcome. So like what population are we looking for? Is it a range of you know age? Um is it the specific age um intervention? What are we looking in treatment wise for aesthetics? Is it fillers, lasers, neurotoxins? There's so many things. And what are we comparing it to? is it comparing it to a different product, a placebo, and what was the outcome of it all? So, break down that question. And once you have your question, you don't necessarily want to spend a ton of time reviewing a lengthy paper because they do get lengthy and complicated, especially when you're going through the methods and the results. So, a good way to kind of check the cliff notes for reading a paper, is look at the conclusion and the discussion. If the conclusion and the discussion answer your question, you also read the abstract and the introduction, but you want to read those four things first because if it doesn't answer your question, you really don't want to take the time to read through all the minutia. Totally. Yeah. Yeah. The abstract kind of gives you a little brief summary if you feel like, okay, this is helpful. This is what I want to read a little bit more about. And then you can get kind of deep dive into it. I'm so nerdy, but I have gone down so many rabbit holes in the introduction because the introduction is always like a literature review, right? Like what's actually known about the topic and they'll often be like citations in there like they'll have one piece of information like I don't know, I'm just saying something random like um Botox for melasma has been shown to have like this sort of effect and then like there's a little you know asterisk that says like two or whatever the citation is. I'll like go to that research article and then like I go to the next research article and read their introduction and you can find yourself in a rabbit hole real quick real quick just reading the introduction. Now we know what you're doing. I know I'm such a nerd but it's interesting. I find it really really helpful. Yeah. And I think a lot of people get overwhelmed with research in general because it's like okay I can read the abstract or introduction but how do I dissect the information? Which was honestly overwhelming when we first started looking into this. But I think there's a few things that you kind of have to keep in mind um when you evaluate and take apart all this research. So like the design and what kind of study is it cuz obviously just you know experience versus actual metal analysis is going to outweigh that. Um, totally and there's just, you know, is it bias? Is there what are the results? How are they measuring the results? How big are the samples? There's a lot of things to consider. Um, so that's a place to start. Yeah, there's also not a lot of papers for aesthetic medicine. So when you're trying to figure out how to compile the research and get these answers, I think one of the best ways that they recommend is to look for meta analysis papers that scan all of the big search engines like PubMed and bring all of the papers that have the characteristics that you're looking for and give you those papers so that you can refine your question but also have the knowledge of all the papers combined to what's important. So meta analysis is definitely a great way to start looking for your answers. I think it's nice now if I don't know if you've noticed this, but when you pull up a research study, they'll have a pyramid on the top right corner and it's kind of the hierarchy of evidence and it will tell you if it's at the top of the pyramid or at the bottom to kind of say to you in terms of quality of evidence, this is where it lands. So at the top of that it's systematic review the meta analysis which is the summaries of high quality studies then you have randomized control trials under that then you have cohort studies case control cross-sectional expert opinion kind of at the lowest level. So that's a really nice snapshot to snapshot to look at when you want to quickly look at an article and say hey um what's the quality of the evidence here before diving into it a little bit more too. So I think that's super helpful. I think what's also important to know like in our field compared to other pharmaceutical companies in other parts of medicine is we don't have big companies funding our studies. We don't have the NIH that's giving us multi-million dollar grants to put on studies about filler. They don't care. These aren't saving lives per se. We might defer. Um but yeah, so we don't have the same funding. So, we really do rely on our big pharmaceutical companies like Allergan and Galderma, Merz, all of these other ones to kind of put on the bigger studies for us. And sometimes I've heard people say like, "Oo, be careful. There's bias because that was funded by these big pharmaceutical companies." I definitely think it's important to have a discerning eye for that. But someone brought up a good point the other day was that those companies are under the scope of the FDA when they're putting out those large studies. So actually a lot of times the quality of their evidence is really really good because the FDA also has to look at that um with an unbiased opinion and decide is this good enough quality of evidence that I'm going to give an approval on something. Whereas the three of us could go and put together to put together a research study and say that it's not funded by someone or we're not biased but what's the quality of our study, right? Like we don't we don't put together research studies. So how many people are in it? probably just the people that we know, which is really not enough for a good study. Yeah. So, I think knowing all of this stuff is important and I think that's the the art of reading research studies is just to have a discerning eye to be aware of all of those things, but not necessarily it's not black or white, right? Like just because it's biased or funded by a big pharmaceutical company does not mean that it's not quality evidence and vice versa. It's just to know it. You should always note who puts on the study, determine if there could be bias, but just keep that in the back of your mind when you're reading these things. Yeah. Yeah. I think even when we look at the results and the outcome, you kind of have to take a step back and look at the big picture, like how are you going to apply this clinically? Just because you read one paper that says, "Okay, this treats melasma or whatnot, we're going to go and do it on all of our patients." It's just you got to look at it with a grain of salt. Yeah, absolutely. A lot of times we Aayesha and I read a lot of papers in just trying to figure out what we wanted to talk about during our next AP classes and we would get into the bowels of the study and then the sample size would be like 11 rabbits. Like wait there's not even on humans. So okay. Yeah. So there is a lot where you get in there and you find out they have never looked at Botox for melasma since we're on that topic necessarily on a lot of humans. Some of the studies we read were just on rabbit ears but we did find some bigger studies but yeah and I think that's important to note too like even in the big FDA approved studies for aesthetics they're not large sample sizes. We're not talking thousands. Sometimes we're not even talking about hundreds of participants. Like there aren't a lot. And so that is something to take into consideration. Sample size definitely matters for sure. And then are they human or are they rabbit definitely makes a difference as well. I think one thing also that I read was interesting is you have to also look at how long the follow-up was especially for aesthetics. Yes. And I think we see that in practice because we'll get patients that ask us like, well, how long does this treatment last and sometimes we have to say, well, the study ended after two years, so we're going to say that, right? Exactly. So, that also takes into consideration. Yeah. Okay. Now that we've kind of gone over the basics of the research study, why it's important, we're going to go through, so for a listener who wants to go into practice tomorrow, grab a research study, and they want to just quickly be able to analyze it to decide, is this study important? Is it applicable to my practice? We're going to kind of go over a little bit of that. So, you've grabbed your research study. I think the most easy thing to look at is for that little pyramid at the top. It's a quick cheat sheet to tell you in the hierarchy of evidence where does this article land. So the top of that being a meta analysis or a systematic review, right? So that's taking all of the best studies that are out there. It's analyzing and synthesizing all of those. The second level down would be your randomized control trial. So those are your experimental studies. And then below that, it continues down. So cohort studies and then you have your expert opinions, etc. So that's always great to look at just so that you know before you're getting into the study where does this land in terms of hierarchy of evidence. Then we're looking over the abstract. We've kind of gone into that to decide if it's a study we want to get into or not. We look at the purpose. We look at the introduction. The introduction is really nice because that really lays out the framework of what do we know about the question at hand. What is the evidence that's available to us? And that should all kind of get you down by the time you finish that introduction. It should get you to the gaps. So what we know and then what do we not know? And that should kind of the author should should be getting you towards why is this study important? So the introduction happens after that we're getting into the aims and objectives the research question at hand. So a little bit about that is the hypothesis identified is it clearly stated? Yeah. So I think with the aims and objectives it's basically the results that we're looking for before we even read the full research. Um, so is this intervention going to take us to the results? Um, and that's the objective that we're looking for. And then it kind of leads into more detailed of the study. So the sample size and the relevance. So for example, when we're looking in aesthetics, if the study is only looking at a certain, you know, if are they an old group of people, are they males, females, and does it relate to our practice and the population and demographics that we're seeing? Or is it just like Sarah said, rabbits and their ears and whatnot? So, is it relevant to us? Are there actual humans in the study, in the sample that they're using? How large is the sample or how small is the sample. Also like ethnic considerations as well. All of those things are so important. And so I think even if it's a wonderful study, if you read the sample size and it's not relevant to who your your clientele are, take it with a grain of salt as well because maybe you know the results of that study don't necessarily work well with your clientele. So, like you were saying, like if it's a group of 80-year-old men in a study and our clientele is 40 to 60 year old women, like we need to take that with a grain of salt, too. I feel like for the most part, research studies do a pretty good job of trying to get a good quality sample size that they know is representative of the people trying to use it. Yeah. You want to be attentive to the exclusion criteria, too, because if you have a big group of first-time Botox patients in the study, but there's five or six people in there that have had Botox multiple times in their life, then those people can't be in the study. So you have to make sure that the exclusion criteria is attained to and that it's a good cohort. Yeah. Yeah. And then I think like you said Aayesha the sample size and it's a little bit tricky in aesthetics because we don't have huge sample sizes but like if it's a sample size of five or 10 that's not great you know like at least 50 100 obviously thousands would be amazing but um it's just something to take into consideration. I think it is interesting though when you do see those studies with only 10 people if they answer the question that you're looking for it I think it is a good stepping stone to see how they did the study maybe give you more ideas so that you can look for a larger study or even refine your question totally definitely um okay so then we're moving down and then we have kind of the methodology or the methods of the study so was the research design clearly identified. Did the tools that they use to gather the information like the surveys or the questionnaires, are those valid and reliable? That's something that I find is interesting because and I haven't done enough to dive into this, but Allergan and Galderma, they both have surveys that they use when they're thinking about patient satisfaction. And they use that survey a lot to determine at 3 months, at 6 months, at one year, how satisfied were patients. What I would like to know, and maybe someone can find this, are they using the same survey tool? Because they should. Yeah. Right. Like that should be a reliable and valid tool that all aesthetic research consistent across aesthetics. Exactly. Yeah, cuz it's definitely that way in med in like internal medicine or medical studies. Yeah, for sure. Like having specific tools that are valid and reliable that most people will pull from because, yeah, they have worldwide tools that are appreciated by everybody. Yeah. So that is a good question. If Galderma and Allergan are listening, I'm sure they will eventually better be using the same valid tool and we would like to know what it is. Yes. Well, we should I mean, we know what it is, but yeah. Well, I think the gap in aesthetic medicine is that our results can be very subjective. Um, so we don't have a defined way to measure something that's going to give us consistent values because you know what I think if I'm seeing a result to your eye, you may not be seeing that same result, so, very subjective and I think that's where we have that gap. There's also what clinicians are looking to accomplish and what patient response is. So if a patient is very happy with the results but the clinician didn't really deliver what they were thinking of delivering, what's the difference there? Patient satisfaction is one thing, but from a professional standpoint, what did we deliver and is that what we're looking for? Yeah, for sure. Um, and then next would be data analysis. So, this was the hardest part and for a while like we were going through this too and we still are, but obviously statistical significance is one of the most important things we're looking at. So, that's that p value. Is it statistically significant? So that matters. P is probability. So how likely was were these findings just by chance or is it something that is probably going to happen over and over again if you repeat it? So P less than 0.01 being the most clinically statistically significant value. Um anything below I think we said P equaling less than 0.05. So anything important more than 0.05 is going to be less statistically significant and could be due to chance. Just Yeah. Leave it up to chance a little bit more. Yeah. And then confidence intervals, we talked about effect size we talked about. Yeah. Confidence interval is basically that range of value that they give you in that result section. That basically says that the true population value is likely to fall into this range. And then it also follows up with the confidence level. So once you have that range, if they were likely to repeat the study over and over again, it's likely to fall into that percentage of times into that range, if that makes sense. Yeah. Yeah. And then effect size too. We were talking about that like you can have something that's statistically significant. So maybe it's P less than 0.01, but your effect size is a two or it's super small. So you have something that's statistically significant, but really isn't meaningful because the effect size is so small. So yeah. Um, and I think, and I could be wrong, but that your sample size has an effect on your effect size. I think going back to the confidence interval, the range is also important. So, if it's like a narrower range, you're going to have more credibility versus a wider range is going to have more variability and less credible. And I think from the results, you want to look at applicability. Is this applicable in a clinic? Yeah, that's the biggest thing and that's the biggest issue that we come across in medicine in general is how do you take evidence-based research and disseminate disseminate it into practice? Yeah. It may look great on paper, but can you apply it to our patients? Right. Right. Um and that's the hardest part, right? All of it. And we can't help you with that. Yeah. No cry. Um, and then last but not least, so we have the discussion, right? So that kind of goes over again what was the introduction, what were the gaps that were missing, what did this study tell us more about, and then were there limitations? And that parts super important for a research study to say, hey, these were our limitations or this is where more research is needed. Um, if it's missing that, there's a couple more red flags if they're not being honest and forefront with that, too. Yeah, it can also lead you to what are our future questions for this topic? What can we study next? You can also look at those limitations and say,"Oh, is there a study that fulfills this limitation of this study? Maybe go on to read that to get more information about the question that you're asking." Yeah, for sure. Yeah. Love it. Great job, guys. So, this was hopefully helpful to those that want to learn a little bit more about research. Obviously, this is the least exciting one, but an important one to kind of go over how to read research. Our hope is if you have um studies that you want us to look into a little bit more, let us know, we can read them and then talk about them at a future episode. Thank you so much for joining us. We are excited and thanks for coming to AP Class. So excited. Thank you. You've been listening to AP Class, your virtual journal club for all things aesthetic medicine. If you found today's episode helpful, like and subscribe, share it with a colleague, and join us next time as we break down more evidence that helps elevate your practice. Until then, class dismissed.