Thomas Sowell wrote that: "The fact that I have never murdered an editor is proof that the death penalty deters." Since publishing journal articles impact many professors' tenure, prestige, promotions and pay, editors' decisions are guaranteed to alienate many people along the way. Their decisions to reject manuscripts are painful enough, but the process itself should not be a pain. Yet at many journals, manuscript reviews have evolved from scientific quality control into a gauntlet to be endured.
Departing as editor of Journal of Communication, Mark Levy's last editorial reported that he once told his predecessor that his tough choices were causing him to lose friends at a rate of five percent a year. "Only five percent?" George Gerbner replied. "You need to raise your standards." But it is not just an issue of standards. The experience has become as lengthy and involved as writing a doctoral thesis, with delays and trivial revisions ad infinitum, or so it seems.
In theory, reviewers are to assess the quality of research and make editorial recommendations on the contribution to the literature. And based on those reviews, the editor decides whether to accept or reject the manuscript. Sometimes a revision is needed because points are not clear or some parts need more explanation to fully assess how well the research was conducted.
But many increasing numbers of review comments now include all sorts of strange and irrelevant notes, making authors wonder if they read or understood the paper. Are all referees really experts in the field, or just generalists whose names came up in the rotation? And like an overbearing doctoral advisor, one referee for each manuscript might discuss a different study the reviewer felt should have been conducted.
They criticize "failures" to address points that were addressed. They dislike the conclusions, while not stating why they might be in error. They note places where additional papers should be gratuitously cited, though it is not clear how that paper would have influenced the research conceptualization. And sometimes they "don't like" the numbers, or that the researcher didn't use the statistical methods they'd prefer. They seem driven to make comments that are a virtual brochure on their favored research method or approach to data analysis.
This might just seem like a problem of callous reviewers, but editors are starting to confuse the process with that of a doctoral committee. As the number of reviewers per manuscript has increased to four (or more?) at many journals, editors indicate all of them are to "sign off," in effect voting on the final version like a committee. Each revision, no matter how trivial, goes out for another set of reviews, and reviewers are increasingly involved with rewriting of pedantic details.
A thesis or dissertation is a learning experience for which a student gets course credit toward a degree. In that situation, rewrites or new requirements for additional citations and data analysis are to help the student learn new materials. Committee members might raise totally new issues on third or fourth readings as part of their role as teachers.
On the other hand, the journal's double-blind referee's job is more narrowly constrained. They can help improve a paper, as evidenced by the authors who often "thank the anonymous reviewers" for their comments, but they are not to be redirecting a research project along the lines they like to see. They should not be continually raising new lessons for the author to "learn."
And the editor needs to do his or her job, making editorial decisions instead of seeing the reviewers as "voters." Not every change needs to go back to the reviewers; final decisions are not a simple vote counting of determining the "majority." Sometimes a minority view is correct.
A friend told of the time that he was asked to review a paper whose conceptual foundation was numerous articles written prior to 1985. A solid expert in the field, my friend provided the editor and authors with extensive critical comments and also sent copies of more recent articles with a note stating that the manuscript should be rejected unless the authors are able to show why the more thorough and more recent articles are in error. The editor replied that he will ignore this review since it was "odd person out," even though the other reviewers' expertise was much more limited and directed at other aspects of the paper.
In the end, a manuscript that goes through the review process is deemed a worthy contribution to the literature and that the research is solid, clear and honest, and not that it is something with which the editor or reviewers philosophically agree. It does not need a disclaimer saying "The editor does not endorse this subject" just because the project was funded by a cigarette manufacturer or a pornography publisher. As J. Scott Armstrong has pointed out, scholarship is harmed and new ideas are suppressed if the journals require publication to be based on a voting process with full consent of all referees.
Yet a colleague often tells me that there are only two types of papers, published and unpublished, and he is good at getting his work in the former category. So as authors, all of us include the gratuitous citations reviewers "suggested." We write, revise, alter and rewrite -- though the research is long done (and not to be altered), we try to meet the editorial demands of the referees so the paper will be published. Some papers go through four or five editorial revisions extending over two years; some editors will give rejections even after several referee-directed revisions. . . .
And if we successfully go through the process, we might then discover that the insults did not end there. The paper was copy edited to meet the whims of style of somebody else, a process that at some journals is very heavy-handed. After all that rewriting, we might not even recognize the final article, with many of the reviewer-directed changes now deleted.
And after it is accepted, the authors then list it in their vita and annual reports, and some might also hope someone will read it when it is published.