How editors of academic journals can increase the willingness of scholars to review papers and get better reviews

Scholarly research is significantly improved through peer review. Although for profit organizations, like book publishers, usually pay reviewers a nominal honorarium in the form of money or books, if the research is submitted to an academic journal, peer review is typically done by external reviewers for free.

Experts who are asked to perform these types of review usually make an informal cost-benefit calculation about agreeing to review and the level of their effort. If the expected reward (i.e., commitment to service or a field, etc.) is higher than their projected expenditure of level of effort (including competing obligations) they may agree to review a paper.

However, if the process of peer reviewing is unnecessarily bothersome and burdensome, not only will reviewers be disinclined to assist journal editors, but they may also be reluctant to submit papers for review to those journals too.

I get it. Acting as a journal editor is a thankless, but necessary task. And it is difficult to secure reviews from qualified experts in a timely manner, especially ones that will assist both the researchers who submit papers and ones that will make your job easier.

That’s why it’s very important that journal editors continuously ask themselves, how can they make the donation of this valuable free labor as painless and attractive as possible, and insure that reviewers will do it again?

One way to insure that potential reviewers will accept invitations to review is by making the process and experience as user friendly as possible and recognizing reviewer’s efforts in a manner that is meaningful.

Here are my suggestions, some of which journals do and others that some may benefit from:

1. Provide explicit instructions (and/or questions that should be answered) by reviewers regarding the appropriateness of the papers under peer review. Periodically re-examine the instructions and/or questions to be answered by reviewers, not to mention their format, to determine if they enable the quality reviews you are seeking.

2. Don’t have your managing editor spend what appears to be a ridiculous amount of time, going back and forth with the author/s of a paper that has been recently been submitted, forcing them to insert one sentence due diligence information directly into the manuscript, only to give the author/s a bench reject.

3. Advise the managing editor to respond in a timely manner (e.g., not a month) to the author/s of new submissions that there papers need be reformatted to accommodate to the to nuances of the journal format.

4. Be careful about turning the day-to-day management of the journal over to others, unless you are going to properly supervise them. If you are a journal editor, and you are too busy with your own teaching, research, or administrative/service work, then consider sharing the task of running a journal, in an equitable manner with one or more competent colleagues.

5. I understand the importance of giving all paper writers a chance, but by the same token, don’t give reviewers papers that you honestly believe are not going to make it through the peer review process. If you feel that the paper will not pass the review process, then write a few thoughtful sentences or a paragraph why, and send this to the paper writer. Don’t simply pass the buck to your reviewers. Most academics hate to get bench rejects, but it is better to learn early in the review process that your paper is inappropriate and why, than hearing it two-six months later from grumpy reviewers.

6. Make sure that the criteria for evaluation of peer reviewed papers are clearly signposted and properly explained. Better still, have separate fields for the reviewers to fill out with a minimum number of characters.

7. Do not use your editorial board’s members as window dressing. Use them, as much as possible to review papers, assist you in finding appropriate reviewers for papers, and in making thoughtful policy decisions you are considering. If one or more of them continuously refuses to review, takes ridiculously long times to complete this task, and needs to be continuously reminded to return their reviews then with fair warning they should be dropped from the editorial board. Have a system to regularly (e.g., yearly) add or replace members on your board. Add new people to the board, from your most active and helpful reviewers to accomplish this task.

8. Avoid burning your reviewers out. Unless you are on the editorial board, sending reviewers more than two papers a year is inappropriate.

9. Send your reviewers periodic reminders when their reviews are due. If they don’t respond to you in a couple of days or at most a week send them a friendly reminder. Reviews that linger more than 2 months are unacceptable. Quick turnaround times are appreciated by authors and word gets around. You get more and better submissions that way.

10. Clearly indicate on the invitation to review when you want the review returned. Due dates should not come as a surprise in the next e-mail the reviewer receives.

11. Don’t ask for a review due in a week’s time after the reviewer has agreed to write it.

12. Mentor reviewers whom you believe could benefit from this kind of assistance so that they can improve this skill. Get a diverse set of reviewers and editorial board members (different countries/racial and ethnic backgrounds/genders, academic ranks, etc.). This usually increases diversity of ideas, gives opportunities to under-represented groups, etc.

13. When the reviews are completed and you have sent them to the author, take the extra step and send an extra blind copy to the reviewers (blind their names too). We like to see how our evaluation compares to our colleagues. Sometimes this gives us an informal benchmark on how to improve our reviewing.

14. In addition to partnering with an organization like Publons that tracks reviewers efforts, once a year publish the names of the reviewers of the articles as an appendix.

There are plenty things wrong with scholarly journals, but one of the easily solvable ones have to do with the peer review process. Over the past two decades, with the advent of e-mail and web-based reviewing platforms significant improvements have been introduced into the academic paper reviewing process, but the above listed suggestions may improve the process and experience, and assist editors of scholarly journals to recruit and maintain appropriate reviewers.

Photo Credit:
Photographer: William Murphy
Title: LOST VENN DIAGRAM [GEO-TAGGED]-1574548