Revise & Resubmit is damaging to science and should be abandoned

I have written about the bad effects of Revise & Resubmit (R&R) earlier (here and here), but I keep hearing from people who see no problems with this type of editorial decision in journal editing, so I need come back to it. This is also because I hear more and more about the unhappiness that it causes in people’s lives, and I feel that much of this is unnecessary. Rejection of applications for jobs and applications for funding is unavoidable, because the amount of money is limited. But the way we handle journal publication is in our own hands: Administrators and politicians do not normally tell us how to report on our scientific findings, so it’s our own choice. (They may tell us that only publications approved by Clarivate Analytics count, via SCCI or JIF, but that is another sad story, quite independent of R&R.)

So here is why R&R is bad: It gives the reviewers the role of Supreme Judges and potential coauthors, while in fact they should have the role of advisors to the editors and helpers of the authors.

In a system that only has accept (with recommended revisions) and reject, the reviewers will not become quasi-coauthors, because the authors can choose which of the recommendations to follow. They have the role of friends or colleagues who hear a conference talk and then point out possible ways of improving the paper. Any author will be happy about constructive comments (and may even be interested in severe criticism), regardless of whether the editor’s decision is accept or reject. (In such circumstances, the reviewers need not be anonymous, whch might motivate some people to write more reviews.)

But if the system includes a third option “revise and resubmit”, the whole dynamics changes, and reviewers are no longer colleagues. The relationship becomes asymmetric: the reviewer is not a friendly “peer”, but a kind of supervisor who can tell the author what to do and what not to do.

And because we are desperate to have our papers published (as our careers depend on it), we do whatever it takes to get a paper into a journal. If a reviewer asks us to add a citation to their work or to their friends’ work, we happily comply because adding a citation is cheap. Other reviewers’ demands are harder to meet, but we do what we can to please the reviewer. Often we think that a reviewer’s comment is completely tangential, and by adding something to address the point, we are forced to make our paper worse. I’m not saying that all reviewer comments are like this – but these are problems that arise only because of Revise & Resubmit. If there were only reject/accept, we wouldn’t have to try and please the reviewers, and we would concentrate on making good use of those comments that help us.

All these problems wouldn’t exist if reviewers were merely helpers and advisors. The help would be appreciated by the authors, and the advice is crucial for the editor. But why do so many people think that this is not enough? Why does the reviewer have to serve as a co-decider (along with the editor), and why do they have to examine the revised version of the paper to make sure that the author has fulfilled their demands?

One can also ask from a different perspective: Why do so many scholars accept that the reviewer has the role of Supreme Judge? I find this quite puzzling, but the idea seems to be that the editor cannot be an expert in the narrow topic of all the papers submitted to the journal, so she needs to delegate the judgement to the reviewers. But are experts always the best judges? In the legal system of most countries, judges hear expert advice and then take their own decisions, and I always thought that this is the editor’s role in journal publication. Clearly, the editor will make many “wrong” decisions, so it is important to have alternative outlets for a rejected paper that is in fact very good after all. I do not think that there is any evidence that expert reviewers (who are often junior scholars, typically people with their own axe to grind concerning the topic) are better decision-takers than editors (who are often senior scholars with extensive experience in the field). Decisions on journal publications involve a large amount of randomness anyway, like most other decisions that are not based on simple indicators.

Journal editors sometimes say that a submission is not quite “publishable”, but after revision it becomes publishable (e.g. here), as if there were some kind of objective measure of “publishability”. Maybe the reviewers or the editor have strong views of what is missing in a paper or what must be changed, but typically the views of these three or four people are quite divergent, and a fifth or sixth reader would have still different opinions of the same paper. Why should the reviewers be singled out as having absolute power over the paper? I think that if a paper is not good enough so the author can be trusted to submit a good revised version, it should simply be rejected.

To be sure, not all papers can be published in the first journal chosen by the author, but the really damaging effect of Revise & Resubmit is that it strongly encourages authors to change their papers not only in response to reasonable suggestions made by the reviewers, but also in ways that the author finds unreasonable. In this way, the reviewers effectively become anonymous coauthors. I have sometimes criticized aspects of a colleague’s published paper, only to be told by the author that “the reviewers wanted it”. Are such changes “improvements” of papers? Maybe from the perspective of the reviewers whose wishes were fulfilled, but surely not from an objective perspective, because such papers are no longer fully accountable in the way as a scientific paper should be – nobody knows the name of the anonymous coauthor who made a definite contribution to the paper.

(Sometimes this has eventually led to reviewers becoming real coauthors, and this is surely positive. But this can happen also without R&R.)

I believe that to a large extent, the unhappiness associated with journal publication comes not from being rejected and from the additional work of revising a paper, but from thinking about ways of satisfying the reviewers. After all, it is not just that the reviewer (effectively) forces the author to make an unwanted change (this might be swallowable), but also that the precise nature of the change is unclear (this uncertainty causes real unhappiness).

But the author’s unhappiness and lack of accountability is not the main way in which R&R is damaging. It is primarily damaging because it is an enormous waste of time and energy. Many journals send the revised version to the original reviewers, who then have to go through the paper again and say whether they are now satisfied. If they find the time to look at the paper again, they will often still be unhappy (because few authors can match a reviewer’s expectation perfectly), and they will spend more time on the paper and perhaps make further critical comments.

(It often seems to be difficult to get the reviewers to look at the paper again, for understandable reasons, thus causing more work for the editor. Many editors even seem to send a revised version to an original reviewer even if they had recommended straight rejection, which will be frustrating for all except the most patient reviewers. All of these problems would not exist without R&R.)

One might speculate that some reviewers and editors are motivated by the desire to have an impact on their colleagues’ published work. I have often heard people complain that they were asked to review a paper that they had previously commented on for a different journal, recommending R&R. Now they get the same paper (which was apparently rejected by the earlier journal), and the author did not make any change!! Understandably, they feel that their work is not sufficiently appreciated if they regard themselves as a kind of anonymous coauthor.

(By contrast, if a reviewer’s role is not to have an impact on a paper, but to help a colleague, then this looks quite different. In such a case, as a reviewer I would simply tell the journal editor that I reviewed this paper before for a different journal, and that they should look for a different reviewer, or I could send the same review again. An author is free to accept or reject my help, and an editor is free to follow my advice or not – there are no bad feelings, and no unhappiness.)

Revise and Resubmit also leads to a lot more work on all sides (for the reviewers, the author, and the editor), but perhaps the most important way in which R&R is truly damaging is that it leads to very protracted publication cycles. In the traditional journals, it can take months before the first decisions, and additional months after the submission of the revised version. I don’t know of any statistics, but I think a year between initial submission and appearance is quite normal for linguistics. We may be used to this, but especially for younger scholars with an insecure job situation, this is quite terrible.

New journal publishers are trying to make publication more efficient, and the Frontiers https://www.frontiersin.org/ journal series is an interesting development, as it integrates the social media experience into its online platform. Interactions with reviewers now look like discussions on Facebook. This can be interesting, but if the reviewer is the Supreme Judge, it’s more like a professor-student interaction – as an author, I have to strive to make the reviewer happy. (My first Frontiers experience was the recent paper on “Human linguisticality and the building blocks of languages”, and while it was interesting, it was also frustrating, because there were many rounds of back and forth; I kept thanking the reviewers but telling them that their requests went beyond the scope of my paper, and they kept saying how disappointed they were, etc. Really weird…)

I’m not a financial experts, and I have mostly commented on the negative aspects of the practice from a scholar’s point of view. But university administrators should perhaps pay attention: If they found a way to encourage journal publication without R&R, then a lot of money could be saved: editors would have less work, authors would be more productive, and the shortage of reviewers would be mitigated.

But can R&R be abolished? Isn’t this the way it has always been, and the way it must be? Not at all. Peer review is in our hands – our scholarly associations could pass recommendations, each of us could choose venues that employ better practices, and if we want to get rid of this damaging habit, we can. A better world is possible!


5 thoughts on “Revise & Resubmit is damaging to science and should be abandoned

  1. Martin, I have various thoughts about this but I think I’ll start by asking you to clarify how you think things should work. One is whether journals will be at all allowed to demand changes – that is, “we’ll publish your paper but you have do this and that”. The other is what happens when a paper is rejected. Will the authors get the reviewers’ recommendations, and also perhaps more critically, the editor’s motivation for the rejection?

    • I think a journal should not “demand” changes – after all, the paper is the author’s paper, not the editor’s or reviewer’s (who is not a coauthor, but a colleague). Editors and reviewers are not supervisors who have superior knowledge. If a paper is so bad that the author cannot be trusted to improve it by taken the reviewers’ comments into account, it should be rejected.

      I don’t think hearing the motivation for the rejection is particularly important – the reviewers’ reports will usually be quite critical in such cases. In fact, I think that being told what is the problem increases unhappiness, because authors almost never think that they are being treated fairly (quite rightly – because they are not being assessed objectively, so it’s not fair). Moreover, a journal can only accept a certain minimum and a certain maximum number of papers, so acceptance will always be dependent on relative quality, never on absolute quality. (Well, maybe this is not true for some of the new online-only journals, such as PLOS ONE.)

  2. I have no trouble with three or five successive rounds of review. These do improve papers, including my own and frankly the ones I review (if I ask for new experiments, I then need to see how the methods and results are described in the next version).

    But “reject with resubmission encouraged”, a decision heavily favored by some journals, is nothing short of a scam, and an amazingly misguided one at that. Its purpose is to improve the prestige of a journal by cutting the time from submission to acceptance – when in reality what people complain about is the time from acceptance to publication, which easily amounts to several months. (At least online “advance” publication appears to have eliminated the many cases where the time from acceptance to publication filled a year or sometimes two.)

    I just wasted 2 weeks revising a paper to appease reviewers when I genuinely felt like adding the reviewer’s issues made the paper worse. After each resubmission and revision comments, the reviewer comments just got snarkier and snarkier.

    Complain to the editor. If that doesn’t help, complain higher up.

    Finally, the paper was accepted (by the editors!)

    …Oh. Am I having a culture shock here?

    Over here in the life sciences, only editors get to make decisions. Many editors rely exclusively on the reviews, some don’t even read the manuscript – but they’re under no obligation to do everything or anything the reviewers want, and I have experienced editors explicitly disagreeing with specific reviewer comments in their decision letters.

  3. Completely agree! I just wasted 2 weeks revising a paper to appease reviewers when I genuinely felt like adding the reviewer’s issues made the paper worse. After each resubmission and revision comments, the reviewer comments just got snarkier and snarkier. None of the issues were really intellectual, the reviewer comments were completely unscholarly and centered around whether we were allowed or not allowed to defer issues that we deemed irrelevant to the point of the paper (the acoustic correlates of stress and tone) to issues outside of its scope (whether Chacobo had evidence of a metrical system). We had to spend 10 pages rambling about all the different ways to derive surface contrasts through the various analytic machinations of metrical / prosodic phonology only to point out that the issue was completely irrelevant to the results of our experiments. Some of the reviewers comments literally just said things like “I think in Tallman (2017) some sort of an analysis like this is suggested in some way…”, followed by a transparent misrepresentation of my own work, followed by the comment on the next review “the authors didn’t engage with my comments at all…”.

    I even emailed the editors practically begging them to render a rejection after receiving the second round of starky off topic comments from the reviewers about them wanting to address issues in the paper that were already addressed in other published material. Finally, the paper was accepted (by the editors!) and the reviewer comments vanished (I suspected that the editors just dismissed them in the end after having wasted our time).

    I was so angry about responding to these comments, which were just ideological (like facebook discussions), that I had to take a break from linguistics for like a week.

    The final “Response to the Reviewers” was such a detailed take down, that I felt like the I should have added that these reviewers should have their degrees revoked (assuming they had linguistics degrees, which wasn’t obvious from their comments).

    If you don’t like a paper, don’t make the author write a different paper, just reject it.

    • Thanks for this impressive personal testimony of unhappiness (I edited some expletives out of your comments).

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.