🎓 Why Systematic Reviews Are the Safest First Paper

In this live, I’ll break down why systematic reviews are often the safest first paper — especially if you’re early-career, overwhelmed, or worried about wasting time.

This isn’t a how-to.
It’s about making the right research decision early.

As ever we'll cover your research questions and review your work.
👇 Submit your video questions for feedback here:
https://forms.gle/gp9cceQfWrXXWcXb6

Transcript

SPEAKER_01 0:01

Systematic reviews are the best first paper to do as an early stage researcher. And in this week's Fast Track Live, I'm gonna make the case to you for why that is. You'll remember if you joined us last week that we delved into whether the PhD system's broken, and you'll remember the core idea I kept coming back to. That yes, the PhD system is failing broadly at the moment, because the conditions required to support the inherent apprenticeship model of it have slowly eroded over time. And that's left researchers to many not get the support they need, feel lost, and forced to figure it out for themselves. And so today I want to go over why systematic reviews are a specific decision that you can take early in your career that dramatically reduce that risk. So I recommend it as a first series paper, especially for PhD students or clinicians coming into research. And today I'm I'm not gonna go through how to do these. Thus, for another day, I've got a full playlist on my channel that uh you can check out and it takes you step by step through a systematic review challenge to take you start to finish in 90 days or less. Um, so we're not gonna do any PICO walkthroughs or Prisma boxes, although at the end of the session, as ever, I'm gonna take the questions that you sent this week. And if you would like to participate in these sessions and get my personal feedback on your research and these workshops, uh click the link to submit your video question. Whoops, and that's the wrong link. Yeah, just click this QR code and submit us your video question. I'll cover it. And of course, we've got, if you want to interact more closely, we've got our private members' workshops where obviously we can go into much more detail supported by our step-by-step courses. So, yes, coming back to this, um, I want to explain why strategically doing a systematic review review first is unusually safe and sound as a starting point. And it's one that I've used personally with my PhD students over the years and my time as a professor at Harvard, Cambridge, and Oxford. And many of those researchers who started on that very trajectory now find themselves as senior, uh tenured, permanent faculty at many top-tier institutions uh in North America and Europe. So uh let's dive in. Uh, let me start by saying systematic reviews are, even though they're safe and sound, they're by no means easy. Some people think that you can just get a bunch of papers and create a boring list of them. Uh, that doesn't work. They require real thinking, real work, real discipline, and importantly, they require a higher level of analysis, of synthesis. So, just to kind of clarify what a systematic review is, um many of you are familiar with the literature review and where you go and you find papers on your topic and you summarize them. Except that's not really what a literature review is supposed to do. It's supposed to be a strategic argument for your next paper, um, or what you want to do uh in the paper that you've slotted the literature review into. Systematic reviews are like these literature reviews on steroids, because what you're gonna do is you're gonna treat these articles like pieces of data. And you're going to be able to answer questions that others can't even ask by looking across the entire field. You'll spot patterns, gaps, you'll see where the field is moving. Um, and that's where the real magic and power of systematic reviews comes in. They can be done in a qualitative way, they can be done in a quantitative way. When it's done quantitatively, that's sometimes called meta-analysis. But what's important is that they're lower variants than other types of research papers. Imagine if you're doing interviews and you can't get, you can't recruit enough people to interview, or quantitative data and your analysis just isn't working out, or experiments and your experiments are are failing. So, as I'll go through uh in the reasons why I recommend these so strongly, um, is that they dramatically reduce the numbers of ways things can go wrong early on. And they're also often benchmarks or reference pieces in your field, relatively highly cited for what they are. Um, and that's those are just a couple of the ingredients of what makes them so powerful as a first paper. Um so let me go through the the first reason that I'm I'm such a big advocate on this channel of systematic reviews. The first reason is they're step-by-step. And I think one of the hardest parts of doing research for researchers is not knowing what to do next. There isn't the same built-in structure as when you did coursework before. You're kind of told, like, well, come up with a topic, find a research question, read the literature, and you really are pushed off the deep end into the pool, where what you've done before is you've aimed to get grades and high marks. But that doesn't, and there is a right or more right answer in many of those cases. But at this level of doing research, you're stepping into the unknown. And so what's great about the step-by-step nature of the systematic review is there is a structure, there's a built-in roadmap to it. It's one of the few research projects where you can see the path very clearly before you even start. And it follows a very predictable arc of um steps from going through how you define a research question using a PICO model, how you go about searching databases so that you don't end up getting confused in Google Scholar or others. You're searching actual databases uh like Web of Science, PubMed, others. Um you learn, you have boundaries so you know what to extract, what's in, what's out. Um, and there's a formula for doing the analysis for actually learning how to do that synthesis and learn how to see and say things more uh with data. And then the writing is also quite formulaic in that the across fields, the introduction, the methods, the results, the discussion, they all have kind of the same core ingredients. And so that lack of structure that should be provided by a true apprenticeship model in the PhD, um, just it creates a softer way, a softer entry into doing research because the structure is built in for you. That dramatically reduces anxiety, um, dramatically reduces the potential to drift as a researcher. I'm not saying it's easy, um, but it's it's a whole lot more navigable than other types of projects, especially if you're if if you're in the situation where you're on your own and you felt left to figure it out. Say hi to some people joining us again. Hey, hey, Amy, uh good to see you again here. Hey, Amy Chris, and uh, we have Abu Vakar. Can it be suitable for PhD and masters? Absolutely, 100%. I can if you go to my Google Scholar profile and you look back in the years when I I mean it's very easy to find when I was a professor at Oxford, you'll see several of my PhD students who went on to get permanent positions at King's College London, at um uh at Oxford, even uh London School of Economics, uh York, I mean, I I could I could go on several in North America. Uh that's exactly what they did. And we've had many master's students, I've even had undergraduates publish these systems, even high school students uh publish these reviews. So I if you're a complete beginner, no research experience at all, you've tried to publish and failed multiple times, gotten rejected, uh, had false starts, this is a great way to build your confidence back. The second reason, so the step-by-step nature is there. But the second reason I make the case for these is that is foundational. Um it sets the foundation for everything that comes next. And and and that's because the systematic review will force you to read everything on your topic. And when you do that, you get confidence. But there's something more that happens that's built in this process. It prevents some of the kind of rookie errors that can happen. It will prevent you from duplicating research that's already been done or answering a question that the field's already moved past. Um it's going to help you identify where the field is moving and where the sweet spots are for what I love, which is low-hanging fruit. It'll help you find those high-impact papers that the field's pointing to that haven't been done yet. And so it's gonna help you see where the evidence is thin and saturated, uh, what's contested, what's still live, and you're not gonna feel like you're swimming against the current because you'll be able to zoom out of your field and see where that current is moving. So um, you know, somebody put it to me that systematic review is kind of like uh Intel. It's like intellectual reconnaissance. So you're you're really mapping your field to figure out where to deploy your resources first. So think about systematic review as your intellectual recon on your journey. Number three is well, you're gonna have to do a lit review anyway. That's often what researchers are asked as the very first thing, right? A prof or supervisor will say, well, go do a lit review. They want to know that you know your field, that you're not coming up with something out of thin air that has some basis or logic to it. What they're really asking for when they say go do a lit review is they want you to find a good research question, they want you to justify a gap. It's just unfortunate they don't always say it in those terms. But what makes a systematic review better is that it's unusually publishable and so highly cited. So if you're gonna do a liter review anyway, do one that's more publishable. And the reason why it's publishable, people sometimes get confused about. So they often see narrative-based literature reviews published by top people in the field, and then think, oh, I can do that. But what they miss is that a lot of those narrative reviews are invited and they don't go through the same peer review process, or they might be in a special issue, or it might be an editor of the journal who publishes it because, well, they're the editor of the journal and they have a bully pulpit. They they can do that. Um what's different about a systematic review is it's reproducible, and that you're going to do everything in that step-by-step fashion. Uh, remember, point number one of why to do them, so that others, you know, we've got Amy Chris here in Abu Bookar. Amy does her review and sets out her methods in a cookbook-like way to where Abu Bookar could take those same steps, come up with the same papers to analyze, follow the same methods, and come up with the same conclusions. That reproducibility is at the heart of science. Um, I've got another training on how that reproducibility is under threat and why systematic reviews are one of the best uh remedies we have against. Um, but that reproducibility is what makes them so much more highly publishable. And to just kind of emphasize their importance in clinical medicine, they form the basis of most guidelines. So when you go get treated by a doctor for something, odds are over 90% of the time, the evidence that they are deploying to treat you has been crystallized and codified into guidelines from systematic reviews. And so when life is on the line, it's life or death, and evidence has to be of the highest quality, systematic reviews are used. So you are literally in producing a systematic review, producing the highest possible quality piece of science that's out there. It is at the top of the evidence hierarchy and pyramid. Now, some some people may not agree with me about that, but um that's just, I mean, you see the impact of evidence in how it's taken up and how it's used. And it's so strong because you you are drawing on the entire field to make your conclusions. I would that they and that's why they're getting picked up more and more in psychology, in economics, and social sciences, having sprouted out from clinical medicine, also in engineering, um, computer science, management, education across the board. Uh, because in a world with too much information, too many studies, and noise, systematic reviews have a critical function to distill, orient, and set an agenda. Okay. Um let me get to the last point that I think is deeper and more transformational as a researcher. Every researcher goes through a journey from where they start, maybe start feeling confident because they've done well in their classes and grades. And they're like, well, I did great in my master's, I did great in my undergrad, I'm gonna do great in my PhD. Um, and then by the end of this journey, the other side, they are confident as a researcher. They don't just uh, you know, they they don't just read research like a native, they actually do it and produce knowledge. I'm gonna do another training later on that PhD transformation arc, the very predictable stages it has, where you get stuck, and what kind of mental, uh cognitive, and emotional shifts you go through along the way. Um, but one of the things that that the systematic review does is it ingrains very good habits. It kind of forces you into a process of good research, of detailing your steps, dotting your I's, crossing your T's, asking precise defined questions, separating signal from noise to to learn how to synthesize instead of just regurgitate. And and so the way to think about it is um that that this is like the best kind of training because uh what what I'll find with researchers I work with, they go on to do quantitative work, and the level of rigor and detail they've done to define how they've included chosen studies for their review and analyze them is the same rigor and detail they need in their quant work. It's the same with qualitative work. Whatever field you're in, this gives you a kind of safer space to develop, hone, and refine those skills without being overwhelmed by the additional challenges that can come from implementing technically challenging analyses down the road. So you master really core skills about writing a full paper start to finish, and the first paper is always the hardest paper to do. Almost everyone I work with, what if they get past that first paper, um they they they've made that transformation journey, they've become a researcher. And the second paper, they always say, that's so much easier, so much faster. I I battled with this first paper, but then somehow two, three, four papers a year, it it was unthinkable just months ago, but once they got that first paper, so like everything clicked and suddenly it opened the floodgates. So the fourth reason here is that it builds confidence, it trains the right core skills of research and prepares you better for anything you do next. Uh, you'll just find little things like you read faster, you know how to forensically go through papers instead of just reading them start to finish or absorbing fast information. You quickly spot weak arguments, it shifts you in a mode of critique rather than just absorbing information. You get to understand what a contribution looks like. All these are skills that, again, we should be training through our apprenticeship model as a PhD, but with that model breaking down, this is a built-in practical system that will force you to learn those skills. Um, and that's why that's why I love it. Uh, I think it's a great foundation, and uh I am kind of an evangelist for these systematic reviews. It doesn't mean it doesn't mean that these are the right choice for everybody. It doesn't mean that everybody should do one. Um I I would just argue here that they they reduce the risk that you face of your confidence collapsing, your timeline slipping, of a paper not getting published, of you going down a dead end or a blind alley. Um so that's why I recommend them. So, guys, what's been your experience doing these systematic reviews? I'd love to hear from you before I turn to questions that you submitted this week. And we have some good ones, including a couple on systematic reviews. And uh, if you have anything pertaining to your systematic review, um do let me know. And I love this comment that Ahmed says. Uh Ahmad comes here and says it says, I chose a topic, did a huge systematic analysis, and my super advisor says you can write it, but it won't work for your PhD. Um, but yes, I found an amazing new topic just because of this systematic review. Uh, that's fantastic. Exactly. It's one of the best people get stuck trying to find winning topics, and systematic review is if you haven't found some winning topics through your systematic review, uh, something's gone wrong along the way in your review. Um, somewhere, uh, somewhere you have failed. So that's great to hear. I don't know what happened, Ahmad, uh, on choosing the topic and doing the systematic analysis that um won't work for the PhD. I'm not sure why that would happen. It it it is worth checking. I mean, you've got to do a lit review anyway. Uh, so though it's it depends. Sometimes some departments will allow the systematic if you say you have to do three research papers, they sometimes want the lit review and then three research papers. So you can do a systematic review, but that's not going to substitute for those three research papers. In other departments, um, especially management or others, the systematic review does count as one of those research papers, and there's even people who do their whole whole dissertations just on the back of systematic reviews. We have uh Dr. Jim Reddy says here, as someone who advised doctoral students in business, I use a systematic review approach to help them identify a reasonable gap in knowledge they fill via their doctoral study, mini SR. Yeah, I mean, I think, look, it takes, we've estimated it takes, and and people laugh at this, but it's really true. Go check out my systematic review playlist. You'll see that how step by step this is. Um, that's 100% free. Check it out on my channel, just go live and scroll back. Actually, there's even a playlist that makes it easy for you. Is it as good as our our full course? No, but uh our full course, you get observational learning, you see people doing the steps, you have worksheets, but that course is is is designed to have feedback built into it. So um anyway, uh I'm not here to sell you anything. Um if it's a great fit, check us out, see if it's a good fit for you. But the systematic review approach to identify a reasonable, reasonable gap is really really ideal. It's gonna again force you, it's gonna avoid a common trap of doing low-value research or duplicating something or just a very minor derivative value add project. So, yeah, Dr. Jim Reddy, I I completely agree with you, and uh I'm pleased that you found this approach helpful for for your students, and you'll see top-tier journals in business are increasingly publishing these. Um, we've got F1T9G4J. Where'd you guys come up with these names? Uh, what can one do if the systematic review isn't giving us the expected answers? Well, that's good. I mean, counterintuitive findings are some of the best findings in science, and this will exactly that. That's why you want to do a systematic review. You want to actively prove yourself wrong. It's a whole idea of falsification in science. We have an idea, we test it, we were wrong, we refine our theory. Um, so yeah, um that's that's that's a valuable signal. I I would listen to that and think what that means for you going forward. Um, and when you say it's not giving the I mean again, the whole purpose of science is you don't know the answer before you start. Uh you might have a pretty good hunch. But uh yeah, if you can give me some more detail, I can maybe help you navigate that. But um, I remember when I was my my very first research, I worked in a uh biochemistry lab and I was pipetting stuff. And let me tell you, I I just I don't know, maybe a wandering mind. I was terrible at pipetting, I just couldn't sit still and I thought it was monkey work, but um I kept getting results that weren't what we expected. And at first I blamed myself, and finally, when we were really able to prove that, yeah, that was real, um, it led to a really nice paper that that then became a quite important paper. And this was in some some cancer research down the road. I don't want to bore you with that. But um yeah, those those counter-I mean, it's sometimes harder to bring prove a finding that's that's unexpected that doesn't go with field norms, but those can be some of the most powerful findings. So I'd encourage you to just probe that a little more deeply. Um and oh hey Jeff, good to see you, Jeff. Glad to have you with us as always. I find few SLRs on AI related topics. Um, I I I disagree with you on that. There's tons. Um, but it depends what you're connecting it to. So it's kind of like COVID. Anything you could intersect with COVID was getting lots of citations just blowing up. AI, unlike COVID, thankfully perhaps, is is here to stay. And so we're gonna see, I mean, we're just seeing a flood of AI-related SLRs on everything you can imagine, um, educational performance, on business strategy, on every health domain you can think of, because it's transforming every aspect of life, much like the internet did to perhaps an even greater degree. So, um, but yes, Jeff, I love what you're saying here. The superior Jeff Jeff goes on to say um the superior learning from an SLR gives me ideas. To improve a later more publishable empirical study. 100%. 100%. And this is kind of the hidden, unseen value of the systematic review. You can feel well, the worst is when I see people do a kind of half-baked lit review, they develop a study, and then they get deeper into it, they get to peer review or somewhere, and the reviewer's like, well, you didn't cite my study that already did what you wanted to do, or you did it in a way that was inadequate. And all it makes me sad because that could have completely been avoided had they done the search in a more systematic way and left no stone uncovered. It's also why I don't really love when people try to use some of the AI search engines because they're they're only capturing a fragment of the literature right now. That may improve in the future, but they only have access broadly to open access stuff. Um so I'm pretty much old school in this. I still prefer Google Scholar. Um, so Amy Chris adds, please, what tools can make my work easier and how can I get journal uh to publish health-related research? Well, um, well, so what tools? Well, the the I mean the tech triad that we recommend right now is Zotero. The reference manager is indispensable. Um, we recommend Grammarly to uh tweak and touch up your grammar, and some very specific use cases of LLMs. And uh I I've shifted recently from ChatGPT to Gemini for that, uh, because of the ability of Notebook LM to keep projects together in a way that's slightly better than ChatGPT's project capacity. Um but uh yes, um those those that's it. So when I see people often getting huge tech stacks, to me, it's a sign of avoiding the hard work. It gives them a feeling of making progress, but they're often not going anywhere. And again, they're just dodging the hard things that they have to do. In your mind, when things get hard, it's really a debt at avoiding the thing that's hard. Um, we're kind of designed as humans to avoid discomfort and suffering and run away from it. Uh and so when you're doing research, you're stretching yourself, you're you're forming new mental connections and synapses. That's a that is is anything but a pleasant process. It's it's like a deep mental stretch. Um, so um, okay, F1T9G4J. I'm currently doing a systematic review on causes and consequences of late marriage, and can't seem to find enough appropriate literature on it. Yeah, so I'm glad you asked this. The way you want to define your, I mean, we we always go through two tests with our systematic reviews as you take them forward. One is a duplication test, so we want to make sure you're not you have some real value, you can add over and above existing systematic reviews. The second test you need to do is a feasibility test. And we do this very early on, and you do need to do a quick and dirty search and make sure there's enough literature uh on your topic uh before diving into it, because you you can't synthesize if there's only two or three papers in your field. So what does that mean? You need to go one level up. And okay, well I'm good, Professor Suckler. How do I go one level up? Well, systematic review, you're gonna use a model called a pico uh to define your topic. And those pico become like a control knob. So you can dial out. Um the pico stands for P is population, I as an intervention or exposure, C as a comparison group, O as an outcome. And you can dial those knobs out. So say if your outcome is late marriage, you could dial that out and maybe say, okay, if you mean late marriage is over 60, you could maybe dial that out a little bit to say over 50, right? You'll capture more studies over 40 and keep dialing, kind of zoom in, zoom out with that knob. Um, when you're looking at the causes and consequences, well, um another thing may so may be the way you've operationalized it. So um I I would be surprised though if there's not enough studies on this. Um so my intuition here, again, not knowing this, just pure intuition, is my suspicion is that there's a second problem. Um so you would have probably passed that feasibility gate if you were going through our course. Uh, but I would suspect that you've got some type two errors, that you're probably missing some studies. Because if you just it depends how you've operationalized your keywords. If you're just looking for causes and consequences, you're probably not going to find the literature that maybe looks like looks at things like divorce, dissolution, uh stress, depression, um, other consequences of marriage. So go look at other systematics. So we use an approach, look at other systematic reviews on marriage and look at what terms they looked for when looking for the causes and consequences. But um, yeah, I uh I think it's helpful already having a sense too, um, F1T9G4J of what you have in mind that you're you're really interested in and you'd like to show here. Because sometimes when people are looking at late marriage, maybe they're interested in kind of life satisfaction, quality of life, or uh relationship breakdown or financial stress or something else. Um, so it it would be helpful to know in the background what what you're most interested in and even just a broad, rough sense of where you're trying to go with your research, just to make sure that this review is aligned with that. But um, hope that helps. I I would need more data to more fully um navigate that with you uh and see kind of where and be able to pinpoint precisely where the bottleneck is. Um but my suspicion is it's gonna be how you've operationalized your keywords uh in this in this case. Um here Abdullah says, Is it a good idea to add SLR in an empirical study in a finance paper? That'll make it too proud. I I I wouldn't try to combine, I see this happen. Sometimes they're like, oh, I'll do two or three studies in one. And I don't recommend that when you're just starting out. Um you could do it later. I mean, sometimes you see big nature science papers, and it looks like they've got 10 experiments, like 10 studies packed into one. Um, do that later. I don't do it now. Are there some people who swing for the fences and hit big with that? Yes. But um often they might have close supervisor input supervision, they might be part of a bigger machine making that happen. Right now, go for low-hanging fruit, get some quick early wins. There's a phenomenon of a Matthew effect in science where the rich get richer and early wins lead to greater long-term success. So you gotta make a quick name for yourself out there to succeed on the job market. And don't hate me for saying it, it's just the nature of the game. Um, publisher parish still is really true. I don't see that going away anytime soon. So coming back to it, Abdullah, uh, yes, I recommend uh that you um keep keep one paper having one big defining method, and the SLR is gonna weaken that. For your empirical study, just do a traditional literature review short. Um, that's just gonna derive your research questions, derive your gap, maybe your hypotheses, um, to to glide into your methods. Um these are really great questions, everyone. Um uh and uh you know when you ask these questions, there's other people out there who might have the same questions, but just be afraid to ask them. So you're you're helping the entire community when you bring these up. With that, I'm gonna go through some of the questions that we had this week. So uh let me turn to the first. We have um we have Ahmed who writes. I'll try to pop this in the chat so you guys can see it. It's a little bit big. So um let's see how much will come up here. But he says uh he's a PhD student in accounting. Um he's published several papers in Arabic and some in English, but is less proficient in the latter. And graduated with masters, with honors and the masters, and uh currently in the okay, cool, currently in the third year of the PhD. Okay. And says, let me pop this in the chat again. Currently says, uh, I've completed the first chapters of my dissertation, and I'm considering publishing a narrative review of my dissertation research paper in a reputable journal. This would be for my own academic advancement and to fulfill my PhD requirements, as most of my PhD publications have been local or in journals that are not considered reputable enough to generate sufficient citations. Okay, yeah, so you want to aim for some higher quartile journals. Totally get it. Um and so uh I'm not sure entirely what your your question is, but in terms of publishing a narrative review uh for your dissertation, like I said, that is tends to be actually a lot harder to publish than a systematic review. Um, because it doesn't have that same reproducibility. Um so it's not impossible, but I do get a lot of researchers come to me and say, Oh, well, I've published my master's thesis. Can I just turn it into a publication? Sometimes yes, sometimes no. Um, if you've done the right calibration steps, that there's a gap, there's a real value add, yes, but in yours, I would just say the secret to you getting into higher impact journals is going to be ultimately based on the strength of your research question, the importance of the research question, the debate in the field, and how valuable that gap is in the literature review. So you really need to differentiate and make the case where well, why do we need a narrative review on your topic? So if you can do that, even if the execution of the review is not that good, you'll you'll still have a good shot. I I've seen I was working with one researcher who got a revise and resubmit, and I'll tell you, I mean, the paper, um she's watching. I mean, I told her the same thing in a fierce but loving way. The paper was terrible. But the question was fantastic. And just on the basis of the loan uh of the question of the gap, she got a revise and resubmit, and we've got a very good success rate of revise and resubmits because once you're in there, you've kind of constrained the space of what you got to do. And the reviewers almost are telling you, hey, here's what you got to do to clear the bar, and you just gotta go do that. Um, and do it effectively. It's not always easy, take some hard work, but um, yeah, so roundabout way of saying, Ahmed, uh I'd encourage you to do some of those diagnostic tests to establish the value of your paper before burning time and energy trying to publish a narrative review that could just continuously get rejected. Um, so yeah, if you want to share with us uh how you've defined the gap for yourself and the value out of your paper by calibrating it to a paper that's very close to yours, uh send that to our next workshop and I'd be happy to help you with that. And if you're confused about identifying the gap or clarifying that value added, um we've got a really good live session on the gap and also some videos on how to find the gap that will get you clarity on that. I won't have time to go into all of that here. Um, we've got a next submission here um who says, Oh, okay, this this I like this one. Um some of you might be able to relate to this. I'm relatively quick in writing papers. If I have the results, I write and submit within one month. But I procrastinate revisions and I prioritize new writing projects instead. How can I overcome my dislike of revisions? Ah, this is this is good. I'm glad you asked this. There's a lot to unpack here. So, one thing you said that I I really want to call out once you have the results clear, it's easy to write. And when people are telling me they get stuck writing, it's often because they don't have clarity about the results. So I love that. And you're naturally following a good, healthy writing flow and intuitively seeing something that's that's very important and in a big failure mode for a lot of researchers. They're writing to figure things out, not writing after they figured it out. The second, you procrastinate revisions. So, what's going on here, implicit in the revision process, is you're getting feedback. You're getting critique. Now, this is part of the PhD transformation arc I was referring to before. In the past, that critique would be in the form of a grade. And you might take that personally. You might have gone and talked to your friends, oh, I got an A, I got a B, I got a C, I got this or that. And it feels quite personal. That can happen with revisions or rejection. It feels like a knife in the gut. You're taking it personally. And the challenge and part of this PhD transformation is to be able to absorb and integrate and grow from that feedback. It's almost like you ever see those like you ever play those video games as a kid where like you shoot like the bad guy and the bad guy just gobbles it up and gets bigger? You kinda need to be able to do that. Uh and and not let it harm you. So, uh, feedback is a gift. And it will sting when you get it, but it it's gonna make your work stronger. So this was a big shift I had in my journey where in the beginning I almost rejected critique and revision because I was taking it personally. And now I go as far as I possibly can to integrate it because they're a signal that something's not clear and needs to be improved. So I I don't know if these revisions are coming from peer review or from a supervisor. But again, what did I say earlier in the session today? When things are uncomfortable, our mind tricks us and finds ways to avoid doing that hard thing. And look, it's bad enough. Uh, we're creative people often going into research. We've got some of the worst shiny object syndrome. And and you often get a burst of energy, the excitement of a new idea. You almost get this gut pull to do it. And and some of you can really uh go hours with like wind in your sails from from this new idea. Um, but finishing that last 10%, that's tough. That is tough to do. It can be boring, it can be tedious, um and it's indispensable. And think of it like a skill. You gotta practice it, you'll get better at it, and eventually it will become mechanical. And so um, yeah, I would encourage when you for all of you, whenever you have these feelings as a researcher, there's a part of this game that's technical, and there's a part that's mental. So when you find these feelings, I would treat it like an invitation to go deeper. And if I were gonna go deeper, just one more thing on procrastination. Procrastination is often perfectionism in disguise. And revisions are you are being exposed. Your last round got exposed as having imperfections, and trying to clear that bar again and subject yourself to having those imperfections called out can be very confronting and manifest as procrastination. So, look, I don't know personally, but if I were gonna go go deeper, I would really encourage you to probe that. And uh that's why also on the inside of our programs we we have an entire mindset track, and we have mindset coaches uh who are experts better than me in uh in working with and helping you cultivate these skills. It's just like professional sports, there's the physical side, but a lot of the top athletes they have mental coaches as well. Um, that's how you get to peak performance, and that's not just about academics, it's anything you want to do in life. Um, so that I really thanks for sharing that question. Really fantastic question. Um, and gave me the opportunity to chat with you about something I'm really passionate about. Um, okay, uh Dataraji 2. Yeah, Dadaraji in our group. Um, feedback is a gift. You know, Dadaraji, I actually I think you were the first person who put that so nicely. Feedback is a gift, and now I say it all the time. So uh thanks to you, Dadaraji, uh, for that. Um but yeah, Dadaraji, um maybe share in the chat with us if that resonated with you and how you responded first to feedback. Because I think a lot of people uh recognize that, and that shift to seeing feedback as a gift is is part of that journey I was talking about before, where you you just start out, you might have early confidence, um, then you get disoriented, then you go through some overcompensation, um, then then well, I don't want to steal too much thunder. This is coming up for another day, but uh that that's that's part of the arc. Um okay. Um we have Fatima. Um Fatima asks, I want to ask, what's your playlist on systematic reviews? But I'm a little confused on how to get the proper search strings if I can't find any pre-existing SLRs. So you don't need a pre-existing SLR on exactly your topic. You're gonna do this for like imagine your keywords, imagine you have three keywords. I'll just make something up. Let's say you have poverty, HIV, and treatment. Well, you don't need to find a systematic review on, I don't know, poverty and HIV. Go find a systematic review on poverty and figure out how everybody else searched for poverty. Go do a systematic review, look at systematic reviews on HIV. So kind of split apart your keywords and search for search strategies that were already established and validated for each of those keywords. And this might actually help our buddy whose name I can't stop saying now, F1T9G4J. Uh, this might help you on uh what you're trying to do as well. Um so Fatima, I hope that helps. My suspicion would be not knowing what you've done, that you might have tried to find just a pre-existing review on your topic, rather than treating this as, well, I actually just want to get the most robust search string, so I want to figure out how other people have operationalized these different terms. Um and and guys, if you're just starting out, what's going on here is when you're searching for literature, uh, if somebody uh says, you know, ethnic minority, um, other people might use different terms like African American, you might need to define as uh Hispanic or or something else like that to capture all the different permutations of the words, and you can try to intuit them, but why not just stand on the shoulders of giants and build on what's already been done? So if anyone ever critiques you in the future, you can say, Well, you have a problem with the way I did my search, you're gonna have to have a problem with those great papers that got published over there too. And you cite them. So hope that makes hope that makes sense, guys. Uh, I see people try to use LLMs for this purpose and they really don't work. This just don't recommend using LLMs at all to find keyword variants. I I just find it's a mess. Um, it's not the right algorithm. Um, okay, and Ahmed says feedback is life-saving. Yeah, yeah, 100%. I mean, I I benefited from a lot of mentors, a lot of input along the way. There's no way I could have gotten to where where I am without it. And it's also why I'm so passionate uh about this channel and our mission at Fast Track uh of creating research literacy and democratizing these implicit parts of research that were passed down from mentor to mentee, uh, creating a system to make this accessible to everybody. Because I genuinely believe that research is not about being a genius, it's not about how smart you are, it's about it's a craft, it's about being shown how to do it. And if I had my way, I would infect everybody in the world with the joy of discovery, the ability to sift truth through noise. Because for me, in this world of cacophony, of ideology, where hate can run rampant, uh I believe truth and research literacy is one of our best defenses and safeguards of democracy. You're getting me ranting. So uh somehow I got from feedback to safeguarding democracy. Well, that's a Friday for you. Guys, uh, I've got one more, a couple more sets to go through here. Uh Beata says, um uh okay, Beata is uh in the middle of systematic review, so I'm gonna pop in to the chat again so everybody can follow along. She's interested in identifying studies via other methods. So what Beata is referring to here is um in a systematic review, you've got, unlike a literature review, you've got to really step-by-step explain how you chose the articles you did. And there's two branches. So there's uh one branch where you identify studies from databases like Web of Science, PubMed, you put in keywords, you search for them, they come out. Others is where you go into things like Google Scholar that are not as reproducible to find articles. And maybe you have to go into gray literature, unpublished literature. In that other branch, um, bad as asking, can your nearest neighbor SLR be including the citation search approach? So one method for finding other articles that goes in that other branch that wasn't in your initial database search is to search citations, something called snowball searching or citation searching. And so the articles you've included from, say, Web of Science and PubMed, you look through all those citations and you see if there's anything that looks interesting, and then you pop that up because you want to detail what you've done in a transparent way, and you apply your criteria to see if they make the cut, if they're in or out, they they meet the inclusion exclusion criteria to be inside your study. Um, so yes, definitely. I mean the answer is yes. So 100%. That works well. And I reckon I do encourage you if you are going to do citation searching and there are systematic reviews on your topic. Yes, go ahead and do that. Now, if you set up your search strategy correctly, you should have already captured all the studies that are in there. So if you find yourself capturing a lot of studies that are in there, it's a sign there's probably something off with your search strategy in the first place. And this has to do with something more technical. I don't want to get into here about type one and type two errors, but Beata, I know you've taken our course, so you know what I'm talking about there. Um so if you're including a lot of studies from this nearest neighbor SLR, you probably have type two errors and something's gone off. Um but yeah, thank thanks for asking that. And and Beata has a follow-up question here. Let me pull that out. Um if uh she asks for new studies, should you pull duplicates out? Um yeah, well, if if you're citation searching and you pull the same study, there's not really a duplicate process because you're cherry picking at this stage and you're not going to cherry-pick duplicates. So you don't have a duplicate stage at that part of the flow. You're only getting duplicates because you're searching multiple databases that overlap. But here, the only reason you're even pulling it into the right-hand branch is because it wasn't already on your left-hand branch of Prisma. Okay, guys. I I if you have any questions about that, I if you're just beginning, don't worry. If that it sounds confusing, that's getting into a little bit of the weeds of the mechanics on how to do a systematic review. Um Usama Salim asks a very good question. How many articles should we have to read for a good systematic review? Well, um there's two aspects to this question. One is how many do you have to read? Uh and like pick up and read cover to cover. And the answer is uh you're not gonna like this answer. The answer is zero because you're not doing a deep read of these articles. You are going to forensically pull things out of them. And so if you're reading an article to cover to cover, um you've missed the the right way to read articles at this stage. And one of the good things a systematic review does is um it goes through, it forces you into a certain reading model to forensically pull data out of articles. I I can't tell you the last time I've read an article cover to cover, probably when I was doing a peer review. It's very rare you need to do this. I've got a great training on my channel about the triple pass reading method, uh, a method we've adapted from other reading methods out there. And basically there's a kind of a top-level pass where you're gonna look like, do I need to read this? Does this fit? Um, my purpose for reading right now. And then there's another pass that you're reading with a purpose to forensically pull things out, and then the most detailed passes you're gonna read to try to reconstruct what the paper did and found for yourself. And that that's very rare. I'd say less than less than 5%, certainly, of the time you should be in that that third, very deep, getting your hands dirty with the paper pass. Most of the time, even systematic review, you're just pulling out key information as it pertains to answering the main question or scope of your systematic review. Now, there's another layer. So I said there were two aspects. So one is about how you actually read how many articles you have to read. The second, I think, is going on here is how many articles you think might need to be included in your review. And to give you a good way to think about this, is that if you include a final set of articles, if you include hundreds in your final review, you're not gonna have a whole lot of space to summarize them. You're gonna end up having to do more quantitative approaches to summarize and map out that literature. Whereas if you have, say, imagine you only have six, that would be the very bottom end of what you could get away with, you'll have space to really get deep with those articles. So there's no hard and fast rule. Uh it depends a bit on your field and it depends on what you're trying to do. But I encourage people to do their first systematic review uh to follow something called swim, which is systematic review without meta-analysis. I imagine you can have a systematic review that's qualitative and a systematic review that's quantitative. I recommend the qualitative one, and to do that in the most effective way with more of a narrative synthesis, um, I find 20 to 40 articles is a really good sweet spot. Um again, can be more, can be less. These are not hard and fast rules. And I think this is all sometimes frustrating for people. There's there's this judgment involved. There's no right answer. You have assumptions, and you have to defend those assumptions. So, Uzama, let me know. I hope that answers uh your your question. Um, but uh again, rule of thumb, if you're below five, you something's gone wrong, you probably have too few articles. Um okay. And I've got one more here. I can see uh Beada just came in. And again, Beata, I think this is uh okay. I'll put let me pull this up first before I answer it so you guys can see it. She says if Silit.com is used, should it be reported in the identification of studies via other methods branched in Prisma? Should studies be found using the same search string that was used, for example, in Web of Science or not necessarily? Um Well, there's a problem here. So Silit is again gonna be a partial uh repository. Uh I mean they're all partial, but why I I I think behind that this quest is telling me, Beata, are you missing some articles? Are you not finding enough articles for your review? You don't need to do this. So my my sense here is just don't do this stuff. Like you've got to give me some compelling reason to do something unorthodox and that there's some payoff to doing that. Where if you search Web of Science and a few of the other major databases, you're fine. You don't need to do this right-hand branch of Prisma at all. Um, so this tells me we might need to revisit something or think about something in your search strategy if if you're not capturing somehow papers that answer your research question. So I really want to get to what's deeper and what's motivating you to try try to do these alternative methods. I will say very rarely, very, very rarely, do we go through uh at least the researchers who I work with and use that branch of Prisma to start cherry-picking articles? Just it's usually not necessary. Um there are exceptions, but I would say in less than 5% of the reviews that we publish, do we do that? Um, Jeff, good question. For top computer science journals, if SLR papers, uh if SLRs include a paper's code and results that are replicable, can you recalculate the paper's outcome statistics that match the outcome metric type used by the other papers? Um people have a harder time replicating studies than you might think. It's one of the things that um was assigned to me when I was a graduate student. Try to replicate this paper, and uh he'd be amazed how hard it is to do. Um so I I yeah, so yes, um uh you should be able to, but standards of reporting are not always sufficiently transparent enough for their reproducibility, which is again another reason why I love SLRs because you guys will be at the avant-garde of reproducibility, because you you will achieve a high standard of replicability with your SLR that I hope you will then later carry on and apply to every other type of study that you do. Um, so yeah, that thanks for that question, Jeff. Um guys, we have had a good session. Have I convinced you that a systematic review might be a good thing to do? Uh, do you have any doubts, questions? If you guys are in team replay, I always come back after and answer all your questions and all the comments that you leave here. Um, I know uh I try to pick a time zone, the time that's gonna be consistent for the majority of you, but we do leave out some of our friends in Australia, in New Zealand, and in East Asia, as well as in the West Coast in California. So uh do let us know if you're on Team Replay, and if you guys have topics that you you'd like to see coming up, uh let me know. I respond to you as the the fast track community. Um, submit a video question here. Um and if you guys are interested in working more closely together, um uh then I would encourage you to check out how we think about research, how we approach it, what our systems are like. And uh I've got a QR code for you up here. See if that approach resonates with you. And if so, I'd encourage you to reach out, uh, book a call, and let's have a chat and see if uh we're a good fit to work together and help you in your journey. But with that, I bid you all a very good weekend, and I look forward to seeing you all next week, same time, same place, Friday, our fast track live.