In science, we virtually never deal in absolutes. Especially in Psychology and the social sciences, we are exceedingly careful to acknowledge that virtually every aspect of the mind, mental life, and the human condition is insanely multi-determined, and there is almost never a one-to-one correspondence between Thing A and Thing B. With that context, I choose my next statement carefully.
A bad peer reviewer is always a bad scientist.
There, I said it. In scientific publishing, there are numerous clichés and tropes about how we “hate” bad reviewers, how “Reviewer 2” is always making unrealistic demands that no sane scholar would ever request, and on, and on, and on. These tropes are often more memetic than anything — cultural in-jokes that serve as idle, mindless, water-cooler chitchat.
But, holy smokes, I really mean it: a bad peer reviewer is always a bad scientist. It means that you’re missing the entire point of scientific inquiry, scholarship, and just good, old-fashioned critical thinking. Science is a really special pursuit, dedicated to collective forward progress and knowledge building for humanity. I place special emphasis on collective here — we’re all in this together, folks. And, ultimately, the logical conclusion here is that if you’re a bad reviewer, you’re actually undermining science itself.
As a peer reviewer, you carry the genuinely important responsibility of providing high-quality feedback to authors. When you provide a high-quality review, everyone benefits from the process: the authors, editors, and everyone who eventually reads the paper that (eventually) gets published. In my experience, you, as the reviewer, benefit as well. When you’re a bad reviewer, well… you’re accomplishing the exact opposite of those things.
To lean into another trope, the fact is that most bad reviewers don’t know that they’re doing a poor job. Not all of the time, of course — some bad reviewers are simply insufferable children who throw tantrums for personal or political reasons, such as sabotaging a lab, theory, or area of research that they don’t like. Other times, I suspect that knowingly bad reviewers are some form semi-sentient potato-like creature from the 6th dimension whose motives are both malevolent and inscrutable. If you’ve ever been on the receiving end of an intentionally bad or obstinate review, you’ll usually be able to tell whether it’s the former or the latter.
The good news is this: if you’re a bad peer reviewer, you can become a good peer reviewer. For those of us with any experience serving as a peer reviewer: we’ve all been bad reviewers at one point or another. But, like our good friend Ebenezer Scrooge in A Christmas Carol, it’s never too late to change.
There are many, many resources out there with “top tips” and recommendations for being a “good” peer reviewer. Bad reviewers are pervasive enough to where most academic publishers have dedicated pages (see, e.g., here, here, and here) to try to teach people how not to be such huge, dense, obstinate jerks. In my view, most existing guides stink out loud and pollute the idea of being a peer reviewer with empty, vaguely bureaucratic catechisms to insulate themselves from complaints about breakdowns in the editorial/publishing process.
At the risk of adding to the landfill, I’ve thought long and hard about what I believe makes for a good peer reviewer, and these are principles that I try to live by. This is advice that I give to virtually every grad student when they first join the world of peer reviewing and publishing, and I treat these ideas more as a “way of thinking” than as a checklist of items to be followed in a rote, mechanical fashion. I have a sense that these ideas tap into some manner of “doing it right” — over the years, I’ve had several authors of papers that I reviewed reach out and thank me (yes, actually thank me) for my reviews, praising how helpful and constructive my feedback was. That sounds ridiculous, and I don’t judge you for rolling your eyes, jabbing the person next to you with your elbow, and groaning “Sweet, sweet mercy, get a load of this self-absorbed bozo.” But, dear reader, I say this to you: it’s true.
With that, I provide to you below, in no particular order, ideas that I try my absolute, most sincere best to embrace when peer reviewing the work of others. Take or leave what you will, but if you write a bad review after taking these ideas to heart, then that’s on you, you semi-sentient potato-like creature from the 6th dimension.
Focus on Improvement
Your mantra as a peer reviewer is the following: “I’m here to help the authors.”
You are not the judge, jury, and executioner of someone else’s research or scholarly writing. Know your role: the editor is the person who calls the shots. Maybe, someday, I’ll write a separate blog post about bad editors. But right now, we’re talking peer review.
Remember, your primary goal — really, your ONLY goal — is to help the authors improve their paper beyond its current state, if necessary. This isn’t about being a gatekeeper; it’s about being a constructive collaborator. Write every review as though the authors are close friends who have asked you for help in making their manuscript better. Peer review is called “service” for a reason. If you’re not genuinely enthusiastic about helping other researchers and scholars, you shouldn’t accept the role of peer reviewer, like, ever.
Offer Solutions to Every Problem
Do not raise an issue without providing a reasonable solution that authors can practically implement to enhance their research. If the writing is weak or confusing in certain spots, offer concrete suggestions for what you think would make it more readable. If the analyses are incomplete or problematic, describe in precise terms what the authors can do to resolve the issue.
Again, your job is to help. Griping about what you don’t like or what the authors didn’t do isn’t helping, it’s just griping. Helping is work, and sometimes it’s really hard work. If you have a gripe that you can’t provide an actual solution to, then you’re not helping — you’re just sitting down in the mud and complaining.
Don’t sit down in the mud and complain. Offer helpful solutions to problems that you identify. If you don’t have a solution, create one — dig into the literature, ask your colleagues… do the work, and then offer your solutions to the authors. Don’t demand, but offer. And that brings us to the next item…
Respect Boundaries
While you may have ideas about how the research should be presented, remember that it’s not your paper or your study. The authors have ownership and creative control over their work. Even if you disagree with some aspects, respect their decisions and choices. Just as you value autonomy for your work, afford them the same courtesy.
I really can’t stress this enough: it’s not your paper. If you wish that the authors did a paper the way that you want it done, then you write that paper. You conduct the research and write it up. You write the perspective piece. You can (and should) offer helpful suggestions. You should never make demands.
Be Precise, Make Feedback Actionable
When you encounter issues in the manuscript, be specific in highlighting the areas that require improvement. It is simply unhelpful to say things like “the methods are problematic” or “the introduction is confusing” — be precise in describing what exactly the issue seems to be on your end. If you can’t be precise or provide actionable feedback, you’re not helping.
If you identify a problem — say, for example, the authors have omitted what you feel is a critical body of work or literature — you better provide a strong list of citations that the authors can dig into. Not one paper and a vague, hand-wavey “there’s a lot of work out there like this one,” but a strong list.
Give Reasons
It is not sufficient to simply state what a problem is — you need to describe why it is a problem. Do not assume that the authors know as much as you do — instead, assume that they are trying their best and, if something seems to be wrong, your job is to put in the work to help them understand a better approach.
Give Feedback on the Research As-Is
Your job is not to dream up a dozen ways in which the research should be done or improved. In many (perhaps most!) cases, researchers cannot easily run additional studies or collect more data — the work is being provided to you “as-is” and you should treat it as a finished research package. NEVER recommend additional studies or data collection unless there is a public dataset that is absolutely, unquestionably perfect for their needs. Even then, remember: you can suggest as an option, you cannot demand.
Recognize the Good
Reviewers who think that their primary job is to “catch” flaws and find evidence of failure, without genuinely praising the positive, are often the product of poor graduate training and seminars where students and professors sit around and just dump on papers they don’t like, typically without taking a step back and trying to understand what a paper does right. This is a major failure in training, and the blame is most often best placed on the professors themselves. Professors: stop being hypercritical clowns with a Philosophy 101 student mindset. Being hypercritical and “not buying it” makes you look smart as an undergrad student as if you have exceptionally well-honed critical thinking skills or have some special knowledge that others lack. As an actual working scholar and scientist, it just means that you’re failing miserably at furthering the goals of science, and it’s performative nonsense.
Go out of your way to point out things that you like about the manuscript and things that the authors have done well. Reviewing isn’t about finding “flaws” and highlighting the negative aspects of a manuscript — it’s about providing feedback. It is incredibly helpful to authors, editors, and other reviewers to hear what an unaffiliated reader enjoys about the work. If you are struggling to find things to praise about a manuscript, you need to either reset your expectations or recuse yourself from the review. There is always something to like, and if you can’t see it, you’re not genuinely looking.
Don’t be a “Karen”
Honesty is crucial, but so is tact and respect. If you wouldn’t say something to the author’s face, it shouldn’t be in your review. Once you have written your review, re-read it (or ask a colleague to read it) and edit your review so that is 100% clear that you intend to be genuinely helpful.
Optional: sign your reviews. I’ve signed all of my reviews for nearly 10 years now. There are many reasons why you wouldn’t want to sign a review, and several other instances where you shouldn’t (especially if the journal has very strong double-blind policies). But, for me, the upsides are two-fold:
- It keeps you accountable for your work. If you wouldn’t give your feedback to someone’s face, there’s a good chance that you know, deep down, that your review is problematic. In my reviews, I take great pains to make sure that I’m being as even-handed and fair as I possibly can — knowing that I’m going to sign my name at the end is a strong motivator to provide the best, most helpful review that I possibly can.
- This is a smaller, somewhat throw-away point, but I’ve found that having a policy of signing my reviews is a great way to ensure that I avoid getting mired in ridiculous inter-lab politics. On three separate occasions, I’ve been approached at conferences by absolutely enraged scholars who had a mind to really “let me have it” for a bad review that they think I wrote. I have to say, those situations have been really, really easy to defuse by simply asking “Hey, just to check, did the review have my name at the end of it? If not, then it wasn’t me. Want to grab a coffee and chat about it?” Whew.
Acknowledge Your Limitations
It’s perfectly acceptable to admit when a particular method or concept is beyond your expertise. Instead of guessing, openly and explicitly acknowledge aspects of the manuscript/review that you are not qualified to provide feedback on. The authors will appreciate your honesty, and it ensures that you provide accurate feedback. Additionally, this can serve as valuable feedback to the authors on what areas of their work may need to be explained more simply or in more detail.
Strive for Realistic Expectations
There is no such thing as a perfect paper. Focus on helping the authors make meaningful progress — even if it’s a really bad paper, your job is to help them reach a higher standard.
You Could be Wrong
Science is about discourse and evaluating ideas. You’re not special: we’re all wrong a lot, and it is important — critical, even — to remember that this is a an ongoing dialogue. Authors might push back on something, and, as it so happens, they might be right. Be open to the possibility that, even if you followed all of the above items, you might have still gotten it wrong. That’s okay — science moves forward at the fastest pace when people acknowledge that they were wrong about something, are open about making mistakes, and so on.
And hey, sometimes, you’re just never going to see eye-to-eye with the authors. Different traditions often operate with completely different ideas. If you think that your discipline is more accurate or correct than someone else’s discipline, you’re failing to appreciate that virtually every discipline has hosted countless, certifiable geniuses who have thought long and hard about their topics, methods, theories, etc. If you’re arrogant to the point where you are willing to write off entire fields of research or scholarship… well… you just might be a semi-sentient potato-like creature from the 6th dimension. Look in a mirror and judge for yourself.
Leave a Reply