Sins of omission? Why do researchers neglect to review or cite relevant scholarship, but reviewers and editors accept this material for publication?

Most academics, editors, and reviewers constantly make decisions about what types of scholarship to review, what to cite, and what is necessary for researchers to make their case.

Failure to review or cite appropriate scholarship or literature should be expected if the publishing venue does not engage in peer review. Why? In general, peer review (where the identity of the the researcher is not known to the reviewer) is assumed to minimize the possibility that flawed research and writing is accepted and published, and that personal biases towards the investigator affects the reviewers’ decisions. Thus, we should not be surprised if research and writing that appears in newsletters, blog posts, or newspaper/magazine articles (which are not peer reviewed), fails to include relevant research in what they publish.

On the other hand, peer reviewed academic articles, chapters, and books, frequently neglect to review or cite important scholarship. This situation, however, gives readers an incomplete picture of a domain, and may open up questions about the rigor of the scholarship that is produced, and the legitimacy of the publication source. Academic writing like this may be called sloppy and unprofessional.

Why does this occur?

To begin with, it’s important to understand that sometimes omissions are intentional, whereas other instances are unintentional. Additionally, rarely is there consensus on what the “important” or relevant literature is.

That being said, some may argue that if a scholarly field is relatively new, then there’s a possibility that the omitted research has not come to the attention of other investigators, and this may explain why references to it do not appear in academic venues. Although this may be the case with recently released work, many published works neglect to mention relatively old scholarship too. Moreover, failure of an author/researcher or team to include or cite appropriate literature may be forgiven once or twice, but if this persists, then something else is going on. If scholarship, and the people who engage in this activity, are to be taken seriously, it should include the most up to date work.

Often scholarship that is written in a language other than English is omitted. This is understandable, as few academics are willing to learn a foreign language, pay someone to translate a written piece, or manipulate files to run them through a translation program. What does this mean? Great work by relative unknowns, new scholars, and those outside of firmly established and entrenched networks (e.g., from the global South, etc.) get ignored,

Another possible reason is that the journal or book may not allow authors enough space to review almost everything that is relevant. This argument may have been true with old publication models, but the citation process is supposed to accommodate for this sort of limitation. With the right crafting a considerable amount of scholarship can be accommodated in the citation process. Plus with the advent of on-line publishing, space considerations are less of a problem.

Additionally lots of academic training can be faulted. Even in the highly ranked graduate programs, it’s not uncommon for students to have large gaps in their knowledge. Sure classes, comps, and a dissertation is supposed to mitigate this kind of outcome, but big gaps in knowledge still happens. Grad students are typically exposed to their instructors and mentors‘ specializations, and rarely to the larger contexts of the field’s scholarship.

Some scholars argue that the neglect of relevant literature is because of a lack of access to published scholarship. Although on-line portals like academia.edu and researchgate.org include lots of useful scholarship, not all articles and chapters are located there. And definitely not books. If one turns to www.google.scholar.com you will quickly find out that some of this is paywalled. This presents challenges especially for independent scholars who do not have an academic affiliation. They may not have sufficient funds to pay for this kind of materials, or have a colleague at an academic institution who can do them a favor by securing items for them. That being said, although it might require some reminders, in most cases if you e-mail to the author of the article or chapter they will gladly send you a pdf.

Sometimes the reason why material is not reviewed or cited lies in poorly informed authors, editors, and peer reviewers. Editors are supposed to be generalists. They don’t know the entire scholarship in a field and that is why they depend on reviewers. But most editors these days will confess that it’s increasingly difficult to secure adequate reviewers. Sometimes the reviewers are inappropriate. And the referees may not take their reviewing obligation seriously. Or they are bomb throwers, out to prove a point.

One reason why some scholarship is ignored are power dynamics in scholarly fields. Some academics including, editors, and reviewers believe that by neglecting or ignoring certain pieces of scholarship (or scholars) they can advance their own scholarly agenda (or those of their closest allies) or reputation. Similar to the notion of academic tribes, some writers, editors and reviewers may not want to give credit to newbies or researchers they consider to be less legitimate so that their own publication record can appear more meaningful, or draw attention away from others. Citing or reviewing scholarship is often a political act. To the author being cited means that you exist, and that you need to be contended with. This kind of omission is disingenuous.

Finally, exhaustion, frustration or just laziness on the part of authors, reviewers, and editors could explain failure to review, cite. This includes being overwhelmed by the numerous demands constantly placed on many scholars to not only produce high quality research, but to be effective instructors and provide service to numerous constituencies. It also involves a disinclination to consider or review scholarship that is physically and intellectually difficult to access. In days past this may have meant having to make a physical trip to the library. Nowadays this might involve the hassle of submitting a request to your university’s Interlibrary Loan Department, and then waiting for the book, chapter or article to be delivered, not to mention additional glitches that this process might involve. Alternatively some scholars work is difficult to understand. This may require more effort than predicted to properly comprehend.

How to minimize the omission of relevant scholarship from peer review work

Most journal and academic editors and reviewers are hardworking and thoughtful individuals. They entered the job with a lot of excitement and enthusiasm, but get burned out or overcommitted, and then take shortcuts. They occasionally give a pass to inadequate scholarship (sometimes produced by their friends and allies) and then clamp down on other work for what often appears to be inconsequential reasons. Both editors and reviewers need to redouble their efforts in their review of papers. In short, if you don’t have time to do a proper job reviewing a paper pass on it.

Another way to minimize the omission of important research is reduce the reliance on “the usual suspects” who are called upon to review, and increase the number of qualified reviewers by drawing from a variety of disciplines and diverse individuals to participate in the peer review process. Increasingly over the years it seems that an increasing number of journals are using less reviewers to make important decisions on the papers that are submitted to them.

We also need to teach people in our profession about the differences in the quality of journals, and what it means to do a thorough peer review. In particular, we need to systematically teach our students how and why some publishing venues are better than others.

Another point to add: we need to hunker down on our grad programs and ensure students have appropriate training. As a profession academics need to be reminded that the research process is a marathon, not a sprint. Academic institutions need support and best practices, and not simply encourage their students (and faculty) to attend seminars on “here’s how you google something.”

Photo: Spanky from “Little Rascals”