BJD Talks
The official podcast of the British Journal of Dermatology
BJD Talks
Episode 9: Peer Review
Use Left/Right to seek, Home/End to jump to start or end. Hold shift to jump forward or backward.
All modern scholarship relies on peer review. But how does this process work? What does peer review offer researchers and readers? What steps should one take as a peer reviewer? BJD Editor-in-Chief, Dr John Ingram, and Global Health Section Editor, Dr Esther Freeman, join Dr Jonny Guckian in discussing all things peer review at the BJD.
Hello everyone and welcome to BJD Talks, we're the official podcast of the British Journal of Dermatology. In this podcast, we try to look beyond our published studies and explore real-world implications of dermatology's research in an accessible way. This podcast is designed for anyone who's interested in skin health research. Whether that makes you a dermatology professor, researcher, patient, registrar, or just simply a skin enthusiast. We hope you'll join us as we build on our world-leading research through friendly discussion. My name is Dr. Johnny Guccian, and I'm a dermatology registrar here in Leeds in the UK, as well as being the BJD's Podcast Associate Editor. Together in this podcast series, we'll dive into a massive range of issues which are relevant and important to dermatology, including patient and public involvement in research, social media, and global dermatology.
SPEAKER_02I'm really pleased to welcome our latest guest for this uh episode. You've met them both before on BJD Talks. All the way back from the first episode, we have Dr. John Ingram, who is the BJD editor and clinical reader and consultant dermatologist. Um hi John, nice to see you again.
SPEAKER_03Johnny, thanks for having me back. It's been a while, but uh yeah, it's it's great to uh join you again.
SPEAKER_02Fantastic, thank you. And I've got Professor Esther Freeman, who is the BJD section editor for Global Health and Equity, as well as uh Director of Global Health Dermatology at Massachusetts General Hospital and Associate Professor at Harvard Medical School. Great to speak to you again, Esther.
SPEAKER_00Hi Jenny.
SPEAKER_02So you know you've made it when you're back into the BJD Talks podcast twice. Um there you go. That's something you can that's a line you can add to your CV. So we we're here to talk about peer review, because something uh something that I've heard since being part of the BJD is that peer review really matters to our journal and to our uh our our dermatology profession and uh our scholarship community. Can you guys tell me a little bit about why peer review matters so much?
SPEAKER_03Yeah, peer review, you know, is the bedrock of uh medical and scientific publishing. It it's not perfect, but it's the best we've got, and it stood the test of time, really. So uh I think that's the the sort of ultimate accolade. Uh, you know, in that uh getting a paper out there uh that's been reviewed by our peers, you know, is ultimately the the artistic that that you know readers uh you know want to be reassured as has uh taken place. And um so you know there's been lots of different models of peer review, but um you know here at BJD we operate a process where our reviewers are are able to be blinded from the the submissions um and they're able to give their free opinions and uh and then of course our editorial team helps to integrate those into a decision on a manuscript. But um, yeah, I can't stress enough how much we rely on our peer reviewers because they really provide us with uh the sense check for a paper, the quality bar that you know they're able to sort of guide us on, uh, and you know they cover all the sort of different elements of a paper that uh it allows us to uh help the or to make it as good as it can be.
SPEAKER_00So delighted that we're talking about this, Johnny, because I really wanted to demystify the process of peer review. I think it can be a little bit intimidating, and especially if you're listening and you're a trainee, um, you know, a medical student or a resident, and you're thinking, well, this doesn't really apply to me, you know, the people I expect to be peer reviewers are, you know, full professor at like some academic institution. And the reality is that's just not true. And I think it can be a little intimidating to get started. Um, but realizing that you already probably have the skills to be a good peer reviewer and that there are, you know, with a peer review, we don't just send a manuscript up to one reviewer, we send it out usually to two to three reviewers. And so your particular perspective is really valuable. So if you look at a paper that's sent to you and you say, gosh, you know, I have expertise in one area, but not everything, um, you know, that doesn't actually disqualify you from being a reviewer because we want to hear what you're familiar with and what you know about. And trust us that as editors, we're gonna find someone that can fill in the gaps of that peer review to give us a perspective on the other areas. So I just wanted to say peer review is something you already are a peer. And so this isn't something that you need to wait another 10 or 20 years to be part of, you can be part of it now.
SPEAKER_02I love that. It's that kind of like equalizing uh hierarchy disrupting take on it, um, and that can be quite uh powerful. You mentioned skills to be a good peer reviewer.
SPEAKER_03Both of you, what would you think those skills are? You know, the the mechanics of of peer review are really just assessing a paper for uh its impact and its methodological quality, um, and those two elements come together, and they're not a specific skills set overall. There are there will be some extra skills to come into play in terms of the particular type of paper. So, you know, sometimes, for example, when we uh publish a clinical trial, then there's a set of things that we look at at as a journal for the quality markers of a well-performed trial. So there are some things that can be learnt for sure, but many of our papers uh they're actually you know very much clinically orientated, and we're we're looking for clinicians, uh juniors, uh, you know, uh you know exactly uh right to be giving uh their views. Uh and uh anybody can do with dermatology experience can then peer review. But uh Esther, what do you think about sort of particular types of paper and the skills of that?
SPEAKER_00Well, sometimes it's almost helpful to think of an example. So I'll just give you an example paper that was published recently in the BJD in our section on the U melon and skin uh score, and basically how we assess skin color and skin phototype. And I think, you know, if you look at a paper like that, I think there are multiple things that come into play. Um, there's the how you design the study, and so there might be someone who's reviewing the paper who has an expertise in methods or in epidemiology. That may not be you. Um you could be someone who has a lot of experience in the literature around structural racism and history of how we perceive race and skin tone. That could be you. Um so you know, there are different pieces, and then there could be something that um, so I'll think of another paper that involves, you know, if you were writing an and now I'm gonna pull a fictional paper that doesn't exist. So um, you know, if you're writing a paper on bullis pemphigoid, um, and I'm not sure why I chose bullspemphoid, but we're gonna go down this route. Imagine you're writing a paper on bullis pemphigoid, and um, there's the study design of how the study was set up, and you might be able to critique on that, but that may not be your area, and that's fine. But you might have experience clinically in bullis diseases and you can comment on that. Um, or you might have expertise on inclusion of race and ethnicity and how that data is incorporated. So there are many different aspects of a study that you might have skills or expertise that you can bring to bear. And obviously, we want you to comment on everything you can, but don't be afraid if you're like, I'm not an expert in all of these things. You know, I myself, you know, none of us are experts in all of these things. Um it's the same when we think about reviewing grants, right? We're we're asking you to comment on what's in the area of your own expertise.
SPEAKER_02Fantastic. Thank you.
SPEAKER_03Yeah, and I and I love it when a reviewer will say, This is what I can bring to the paper, and these are things that I can't bring. That's so helpful then to know, you know, in their comments to the editor, uh, I've, you know, as you say, so I've reviewed on my the bit that I'm uh very familiar with, and um and I want to highlight that there are some elements that that will need a different uh reviewer to um give input to. Uh and for example, at BJD we have statistics reviewers, and they're uh within the journal team, and our editors, uh you know, our associate editors can request that reviewer to provide that air of expertise that of course quite often it's missing in a in a set of of clinical reviewers or clinical academic reviewers. So we we make sure we've got that specific skill and and we can bring it in. And and we we might um we often rely on our uh reviewers to highlight that this paper would benefit from a statistical review, and then uh we can um arrange that. So it's encouraged to uh yeah, as you say, focus on what what you can bring and to highlight the areas that need different attention.
SPEAKER_02What I hear from reviewers sometimes, or from you know, the trainee perspective is that it's it's hard to calibrate um what kind of review you are going to be in terms of am I being some people think, oh am I being too harsh, or am I not being am I not being harsh enough? You've talked about not being expert enough. And I guess one of the opportunities of this podcast is to be able to peek behind the curtain, because you get, John, you get to review all the reviews and review all the reviews. Where do you think this falls? Do you think that you know people are hitting the right note? Do you think that this is something that people should be worried about or not so much?
SPEAKER_03There are a dozen hawks, aren't there, whenever people are you know giving criticism on a piece of work. I think for me what I value most is constructive criticism. Uh, a reviewer who is looking to help to enhance a paper and elevate it. And you know, it's perhaps it's for the editorial team to make the the call in terms of the impact of a paper and whether it's reached the the right bar. You know, BJD's bar is going up, so we have pretty high standards. But the review doesn't necessarily have to make that call. You know, that's um the editorial team and myself ultimately would will do that. So the reviewer, I think, is there, knowing that there's been some front-end triage that the papers that have come through are already quite strong. I think their primary role is to see if they can um, yes, highlight um major issues, but mainly to look to improve the things that they can and highlight things the authors could do that are achievable that would make it a stronger paper.
SPEAKER_00And Johnny, I wanted to kind of touch on just some concrete points of you know, what does it mean to do a review? Because it may be that you were never trained on how to do a review. And so if someone sends you a paper and you're like, great, I'm supposed to review this, like what do I actually do? So I just wanted to give you some kind of some concrete steps that I usually take. So the first thing is you you give it a read through just a once-over overall, without trying to really get into writing anything down or critiquing it. The first thing you need to do is just get a kind of an overall sense for the paper. Um, and so once you've done that, you've kind of had a chance to think about sometimes I even read it once, put it down, you know, come back to it at another time or, you know, a couple minutes later. And then you're gonna want to go through and you're really gonna want to go through section by section and think about, you know, each section, you know, for example, starting with, you know, the introduction, and and you could do this in whatever order you want. So I often, you know, do start with the introduction. And the things that you're kind of thinking about during that second read-through, you know, is that does the introduction set out the argument that they should set out? Do they really summarize recent research related to the topic? And do they really explain the objectives for what they're gonna be doing and set out their aims? Um, the next section you want to go through is your method section. And you want to think about is this replicable research? Do these have repeatable methods? Do they explain what they're doing? Does this follow best practice? Then you're gonna go into the results in the discussion section. So you really want to see that this is gonna tell a nice coherent story and that the results are you know simple, also reflected in the tables and the figures, and that the conclusions when you get to the discussion section are really supported by the data in that paper, and that there is a section that talks about the implications of that work and that they should also clearly describe the limitations as well. And so, and kind of put it in context of the overall work that they've been done. So that's the general way I think about each section. And then when we actually look at a review, the general format that I often like to see or that I teach my mentees and how to do it is that there's gonna be a first paragraph where it's you're as a reviewer, you're gonna write this paragraph that says, you know, thanks for letting me review. You don't even have to say that part. We know that you think that, you know, we appreciate you reviewing. So you don't have to say thanks because really, you know, we should be thanking you. But anyway, you might say, you know, this is a paper that focuses on X, Y, and Z. And the strengths of this paper overall are, you know, the strength of their methods, but you know, some limitations of this paper is that their discussion does not, you know, acknowledge the prior body of work or over states the findings based on their results. So you're gonna have like an overall summary paragraph that kind of lays out what your overall feeling is. And then what we recommend doing is really going section by section, and it's helpful if you can number your comments, because then when the authors of the review are gonna write back to your review, they can say, you know, response to you know, comment number 13 in the results section, here is how we're gonna change our data. So it's helpful to then go section by section. So we're kind of looking for that overall paragraph and then a nice formatted thing where you say, This is my issues with the intro, this is my issues with the methods, here's the results and the discussion. I mean, that's just one approach. I don't know, John, if that differs from how you often approach writing a review.
SPEAKER_03Yeah, I mean, I think that's um exactly right. And SRI, the only other things that I often like is um for people to maybe consider major points and minor points, that can be useful because sometimes in a uh a review, you know, sort of small typos and so on, you don't want to have those up front. You know, they're probably something to put at the bottom just for uh additional sort of uh help to the authors. But yeah, so that that that I think can be useful. And I think that also that you know you get the box, certainly BJD, where you have comments for authors and a box of comments to editors, and a couple of bits of guidance on on that front is if there's any view around you know whether um the work is uh publishable in the particular journal, in this case BJD, um, that can go in comments to editors, you know, so that um everything you know in the author section uh uh it should be you know able to be fed straight back to the author, so you know, just uh respectful and constructive, and then allowing the editors to make that call on whether the paper should be accepted or or you know, sort of or not. But essentially, you know, those are sort of the yeah, just how you present the uh the review. And I think you can also put in that box to the editors where you think that additional uh reviewers would be helpful if there's some gaps you've no reviewers notice that could be filled by by others. So um yeah, just a few thoughts on that one.
SPEAKER_02That's really helpful. It's a really kind of comprehensive framework, particularly, Esther, that that that you provide with us. And so if any listeners are literally sitting down to do some peer review, that is a great step-by-step process. It is a process, and that uh it's just about following these steps, and that can really reduce the cognitive load of that overwhelming thinking of what do I do now when you sit and look at a really big, complex, exciting paper? Um, so that's really helpful, thank you. An Esther, just while I have you, I wanted to ask you, just within your role of the section editor for Global Health and Equity, I wanted to consider the global implications of peer review, because peer review is something that really brings together perspectives from all of the world. Do you think we're doing enough in a global perspective in terms of getting reviewers from around the world? Or is it one of those things where we're dominated really by the that global north, as with a lot of other areas within our scholarship?
SPEAKER_00Well, Johnny, I'm so glad you asked that question. I feel like you already kind of answered in your question what I wanted to say. So it's it's really a propos. But you know, I think this is something we really struggle with. There is a tradition, you know, in academic publishing that the people doing the peer reviews, you know, for the large part have been in the global north, for the large part have been white, um, often, you know, male. And I think that as we work towards diversifying our peer review, that's why it's so important. That's why we're really excited to do this podcast, because you know, you are all the people that we are trying to reach, which is that, you know, younger people coming in with wonderful and important expertise. Um, you don't need to be based in indeed in the global north. The idea is that really diversifying our peer review, because when I have a paper now that's coming into the global health and equity section on leprosy in Ethiopia, you know, I would really like to be able to call on peer reviewers who have expertise in that part of the world and who have expertise in leprosy, for example. And so I think building up our reviewer pool who have really a diverse clinical set of interests and a diverse background and can really think of, I think that just brings so much more to the peer review process. So I think, you know, I'm really excited that the BJD, I think, is really a leader in the space in dermatology in terms of bringing in this section in global health and equity and thinking about how important peer review is. And the wonderful thing about peer review is if you do a lot of work in peer review, then ultimately you often become an editor. And so, you know, really, I would love to say maybe John and I are kind of trying to work ourselves out of a job, right, John?
SPEAKER_03Well, you know, it it's it's brilliant because um we need to be um you know gaining in in the sort of capacity in our system to better manage all the increase in manuscripts that come through. We we need those peer reviewer skills, and and then of course it's great when those folks can progress and become uh members of the editorial team. And Esther, you you cut touched on the sort of mentor menteeship process where we can nurture you know new reviewers. And I've often found that people pairing up is a really nice way of doing that, where you you can sort of produce a review together where the um the less experienced person can can just you know have that guidance.
SPEAKER_00Yeah, it's a great idea. If you are someone who's in training um and you have a mentor, you can suddenly ask and say, you know, do you have peer reviews coming in that I can help with and and do your first couple of peer reviews with your mentor? I do this with my trainees, and I think it's a great experience for them because you know I have them do their own review, I do a review, and then we kind of merge them together and we talk through it. And I think sometimes just the experience of doing a review with someone like looking over your shoulder is very helpful.
SPEAKER_03What I love also where we have in in BJT, I would typically select somebody who's performed a really nice review in a timely way and invite them to write the commentary on the paper, which provides a publication in its own right for our reviewer. And there are lots of ways we can reward reward reviewers, but um I'm conscious that there's a lot of of effort that goes on behind the scenes that isn't properly recognized. And so one way we're trying to improve that is by putting someone forward to write the commentary who's been a really you know um helpful uh reviewer for the paper and they know the paper well and uh and the subject well, so then they they become a brilliant commentary writer. So um that's one way we can reward our folks.
SPEAKER_00I feel like this is like a secret in peer review that I didn't even realize as a peer reviewer. Um, I myself try to, you know, I we do a lot for the BJD, but I also as I get a lot of peer review requests for other journals as well, and I try to do at least one a month as my way of kind of giving back to the dermatology international health community. Um and this has actually happened to me twice, John, where I've ended up writing um a commentary in the Lancet and a commentary in the BMJ. You know, these opportunities do come up if you you know write a good and timely review, you know, the opportunity to then be asked to write a publication related to that paper is there, and it's happened to me on several occasions and it's been a really great experience.
SPEAKER_03Yeah, I agree, Esther, and it's a lovely way of really just um adding value to a paper and really you know rewarding and recognizing the efforts of the reviewer who you know there's a lot of work that is going on that isn't always uh acknowledged. So so yeah, I like that. And there is that standoff, Johnny, of registering on Publons to have your uh reviewer uh efforts um you know properly you know documented, and and that's great for appraisal, for example, and that that may be a really nice option for junior faculty in particular, just to demonstrate the you know um the work that they're doing. But yeah, so I hope there's a there's a range of of rewards. Uh one of the other ones being of course that you get to see inside the review process a bit so that by reviewing you understand then uh as an author, if you're you know writing papers, what you should be thinking of when you write your own paper and and uh anticipate the reviews that are coming your way uh on the back of it.
SPEAKER_02I guess it's part of that, you know, thinking more higher level, it's that community aspect, it's that you're paying it forward a little bit because it hopefully if you do a good review for someone, you've a useful review for someone that's constructive and helpful, and encouraging that amongst others, you will then get a decent review for your work. And one thing when thinking about trainees here who are reviewing and then also trainees who are submitting work, maybe for the first time, they can be quite scared as well of either putting in a review or receiving review. And I like to look at this from both sides because it's I try to frame it when I speak to people is that reviews are more of a conversation. By that point, as you say, it's gone through an initial triage, it's not always just gonna be always not always gonna be flatly rejected. People worry about that big reject email. It's actually everyone is there with the same mission, which is to produce good research and to get produce the best research that you can possibly, you know, uh provide to the community. It's all the same mission. So sometimes I know that from at least from my experience when I started getting involved in research, that you get that first email that might be like a a reject or you know, reject with with uh with with comments though, and possibility for resubmission. People see the reject and think, oh dear, you know, that's that's that's it, and I'm a failure because medics don't like to don't don't like rejection. Um but I I always just encourage people involved in research that it's a it's just a start of a conversation and the reviewers are there to help you and help get the the best version of your work out there.
SPEAKER_03Yeah, exactly. You can only really uh understand that conversation by having been on both sides of it, perhaps, you know. And so I would really encourage our authors to review because you see both sides. Um and yeah, it is that sort of investment in the community that um that does pay back, and it may not pay back the following day, but it you know it it will uh I think it'd be be really uh valuable for reviewers and it is what our journal, all our journals rely on. So um it's a vital uh skill. I think it should be um acknowledged more. Um at BGD we have an annual um publication of all our reviewers, for example, we we put their names in print. Always looking for you know uh ideas for how we can um reward our reviewers better, and always looking to think about the training that we can help people with to get into reviewing. So certainly you know the the VAD uh offers um support there in courses and you know, really keen to promote this as a key skill, really. And uh there's certainly a whole sort of support network that's out there to help people to get into peer review, and but the best one is always going to be your local experience mentors who uh uh as Esther mentioned will be receiving um review requests and be delighted to you know sort of um share some of that uh with their their trainees and co-produce a review. Uh and the important thing is that both people should get you know named and credited for that so that uh the more junior person you know gets the recognition they deserve as well.
SPEAKER_02Yeah, and uh these things are about self development, isn't it? It's that feeling of enrichment. And so you meant you mentioned courses, and and and uh I would highlight that that our friends over at CED. Via the BAD run a how-to-pay review course uh directed generally towards um early career uh researchers within dermatology, whether it's registrars or even before that stage, who maybe have done a bit of reviewing or are thinking about dabbling in review uh in reviewing. It's a really good course. I've done it myself. I've just looked in I I can't see the date for next year's up just yet, but for the last two years it's been in January, February time. So do keep an eye out on the BID website um for listen listeners um for that because it's a it's a really good, quite reasonably priced course as well. So and it's uh I I think it's usually virtual, so do do look at that.
SPEAKER_03And there are other resources on the BGD website. Our instructions to authors are quite handy because uh they really highlight what we're looking for. We d we want to demystify the process, um, so you know, actually that they're there to help authors and and reviewers too, in terms of the the checklists that um we uh find helpful to you know that they help to guide our authors and also our reviewers when they're looking at a paper, and you know, the expectations in terms of the uh that the format of the paper and so on. So I think using those resources might just you know provide make things a bit easier, certainly for new reviewers. So um yeah, it's all all I hope helpful.
SPEAKER_02Thank you. And I I would I I would just like to ask one final question to you, John, really. Uh looking back over the time you've had so far with uh the BJD and maybe even from your experience in reading the BJD beforehand, how do you think that reviewers have changed the journal?
SPEAKER_03I think uh our reviewers have been vital to us uh in in several different ways. That firstly, I think there are things that excite reviewers, and you can see that as an editor. So there are you know evolving areas of dermatology, you know, the global health field, some of our qualitative work, the the the deep dive into understanding of a condition, those sort of elements that I you know I see uh reviewers are really passionate about those papers, and and they've helped us develop that section. I think uh some papers can be undermined by their statistics, and so as a journal, we've really gone out and uh tried to bring in a group of uh brilliant satiscians and methodologists who uh we can just rely on to cover that that base, you know, when we um in the review process, and and then we can have our clinical uh reviewers uh send check the article and and tell us will it make a difference to patient care? And I I think that's what has allowed us to grow as a journal is that we focus on those two areas, that getting methodology as high quality as it can be, and then ensuring that there's a clinical message that will really uh influence patient care either today or in the near future. And so that's really where I think our reviewers have helped to drive up our our standards and uh you know it's their hard work that that we rely on. So um yeah, I can't thank them enough.
SPEAKER_02Brilliant. Well, well, on that note of thanks, um, that brings us to the end of today's episode of uh BJD Talks. Uh we have covered the importance um of peer review, why it matters to journals and particularly to the BJD, and we've provided a roadmap for how you as a reviewer might like to break down your review uh and approach it step by step. And of course, we'd like to thank all of our peer reviewers and encourage everyone who has been thinking about uh getting involved with some review to get stuck in. If you've got any questions, feel free to um message us at uh on whatever social media channel that uh you use, uh either Twitter or Instagram at Brj Dermatol on Twitter or at Brj Dermatology on Instagram, or by using the hashtag BJDTalks. Thanks for listening and bye for now.