Peer Review

"Be warned, I shall uphold my sacred oath to protect this realm as its gatekeeper."
Heimdall, Thor. Marvel Studios.
How many of you have actually stopped to wonder what peer review is? If you’re like me, you didn’t really pay it much attention…you just knew that it was an almost mythical group of beings that controlled your paper’s fate and entrance into the Valhalla. So, to help you understand the entire process, let’s actually take a look at how peer review works. Peer review begins once the editor of a journal receives your paper. Depending on the journal, he/she will decide if the paper is sent out for revision, or rejected without further consideration after the editorial review. If the manuscript passes the editorial review, the editor will then request a team of reviewers (ideally three people) help review the article and provide feedback on whether the paper is worthy of being published in this journal. There is actually more than a yes or no answer to your paper submission.
Possible outcomes include:
  • The paper is accepted as is.
  • The paper is accepted, but some minor changes are needed.
  • The paper is accepted if certain changes are made.
  • The paper is rejected, but may be re-submitted with major changes for de novo evaluation.
  • The paper is outright rejected, even if you significantly alter it.
As mentioned earlier, reviewers are chosen by the editor. Ideally, the editor picks people who are experts on the topics discussed in the paper. For obvious reasons, reviewers should not be colleagues, friends, or relatives of the authors. The authors usually have the possibility of naming people they would like to include or exclude from being a reviewer (e.g., if they wouldn’t want a direct competitor to see the paper). Reviews can be:
  • Double-blind (reviewers don’t know who the author is and vice-versa)
  • Single-blind (reviewers know the author)
  • Open (reviewers know the author and vice-versa)

Even after acceptance, some journals allow for post-publication edits, where a reader of the journal can comment on a paper and make suggestions.

While many consider being chosen as a reviewer to be an honor, it’s not an easy position to take on. Reviewers are not paid, as this is a part of their academic duty. Plus, it can take a considerable amount of time and effort to review these articles. No system is perfect, and peer review has its own set of flaws.

Double-blind reviews can help prevent bias.
Vikings, History Channel.
Some peer reviews can take well over a year before a paper is accepted. There’s also the immeasurable cost of the time taken away from reviewers’ main focus: research. There can also be inconsistencies between these reviewers’ perspectives. For example, these were two reviewers’ comments on the same article1:

Reviewer A: I found this paper an extremely muddled paper with a large number of deficits.

Reviewer B: It is written in a clear style and would be understood by any reader.

The Avengers, Marvel Studios.
And, while you hope that scientists would be able to live up to their duties, human bias and emotion is going to play a role. A particularly strong example of this comes from a study conducted by Peters and Ceci. They took 12 articles already published in psychology journals and remade them with minor edits to the title, abstract, and introduction. The authors’ names and institutions were changed to less prestigious names like the Tri-Valley Center for Human Potential. They then submitted the papers to the same journals they were previously accepted in. Only three of the papers were recognized as being submitted before. An astounding 8 of the remaining 9 articles were rejected due to “poor quality”, demonstrating a bias against smaller institutions2 (and possibly a lack of error-catching on the editor’s part).
There has been strong evidence that female authors face even more difficulty in securing grant funding and publishing papers3. A controversy recently emerged when a PLOS ONE reviewer commented on a paper on academic gender bias submitted by two female authors (Fiona Ingleby and Megan Head). He insinuated that male authors publish more articles and in more highly esteemed journals as a result of innate ability. And, to strengthen their paper, the female authors should add a male biologist or two to the authorship. PLOS ONE has since apologized and removed that reviewer from their selection pool4.
Women may have to fight harder to get published. Vikings, History Channel.

A reviewer’s comments on Ingleby/Head’s submission.
Aside from this, there have been a number of other recent scandals, including researchers reviewing their own work5 and several fake peer-review rings6, 7, 8. On top of this, some scientists worry about having their work slowed, stolen, or plagiarized.
In the face of all these controversies and biases, what can we do? There are several ideas that could be implemented to improve this process1:
  • Standardizing procedures for review and protocols for researchers.
  • Blinding reviewers to the identity of authors (although certain equipment and writing styles can still give away an author).
  • Training reviewers and providing feedback.
  • Choosing reviewers more stringently.
  • Reward reviewers!
  • Creating professional review agencies.

While the review process is certainly not perfect, it’s not something we should abandon altogether. It’s a process that we can improve upon so that hopefully one day, all articles can be judged on equal footing. Have any suggestions on how to improve peer review? E-mail us at

Just when I thought I got the hang of science…and my paper gets rejected.
Vikings, History Channel.
  1. The peer review process
  2. Peer-review practices of psychological journals: The fate of published articles, submitted again
  3. Peer review bias against women
  4. Sexist reviewer
  5. Peer review ring #1
  6. Peer review ring #2
  7. Peer review ring #3
Contributed by Ken Lau, PhD.
Insert Note Here
Save Close Clear
Lab Timer
Login / Register
Remember me
Forgot your password? Reset password?
Create an Account