In praise of harsh reviews
I read the recent Times Higher Education feature criticising supposedly bad peer review with a growing sense of unease. Not so much the criticism of peer review, but the ideas that peer review could be saved by shackling referees’ freedom to write intemperate reviews.
Entitled ‘the worst review I’ve ever received’, the piece concluded with a tale of unacceptable personal abuse. That referee had used their freedom to stoop to criticising the paper for its supposedly sub-tabloid writing style, abysmal grammar, equating it to a mid-level undergraduate essay.
Luckily that author could publish the piece elsewhere to plaudits, perhaps inviting us to conclude there’s something fundamentally wrong with peer review. It’s part of a growing chorus of voices calling for its wholesale abolition or change, making it open or post-hoc.
Peer review is clearly idiosyncratic, and these contributors made much of its randomnity. Some criticisms imply it no longer serves as a dispassionate system of measured scientific governance, but has descended into a snake pit of personal abuse, petty jealousies and pet theories.
And as the editor of a journal, Regional Studies, Regional Science, that proudly offers early career authors a mentored publication route, I’d be happy if peer review become just one tool available to authors trying to make the best ideas public. But I’d be wary of calls to temper the peer review process.
I’d even go so far as arguing that the option to write harsh reviews is central to any thriving science system that prizes integrity over system-gaming.
I must here declare an interest, because the comments the last reviewer received may very well have been mine. If not, then I have certainly written reviews – no more than a handful – that have taken a harsh line.
But I’ve also received two particularly harsh reviews myself that stuck in my memory. The first was as paper I wrote as a postdoc, where one paragraph said that my paper’s whole premise was based on my excitement with a literature that was probably only new to me. The second was an eight-page line by line demolition of a qualitative paper by a reviewer who apparently wanted p-values.
And although each time I was stunned at first, I have to concede that the reviewers were almost certainly right. Both papers were submitted to leading journals that favour a very particular kind of article both in terms of style as well as content, and my research was not a comfortable fit in either.
If the reviewers hadn’t sent the harsh reviews, then the editor might have been tempted to give me a chance to squeeze my findings into the journal style. With the lure of a leading publication, the younger me might have been tempted to ‘play the game’ to say more than my findings would strictly have warranted.
The exciting new idea I sat on for a decade and finally it became a few paragraphs in a publication with two other authors, and the other paper matured through a European Network of Excellence to find a good home. In both these papers, I believe we really say something novel, and make a scientific contribution, where a more tortuous fitted paper may have failed.
I feel strongly that the harsh reviews I received exemplified peer review working correctly. They helped small ideas to grow until they was ready to make a contribution to a collective research community striving to create good new knowledge.
That brutal honesty has two sides – I’ve also written reviews where I have almost pleaded in the review for an unorthodox, badly expressed idea to be given a chance, again underpinned by a profound belief that that was the right thing to do.
Harsh reviews provide a brutal honesty allowing scientists to take decisions they believe to be right. Many of the proposed changes risk losing this space for brutal honesty.
Moving to open review may see reviewers giving the reviews that are good for them, on a Muggin’s turn basis, not the reviews they honestly believe are right for scientific advance. Editors will make calculating decisions, mechanically selecting only those papers that pass the reviewers’ beauty context rather than taking a punt on the outsider.
Authors will respond by not presenting their ideas and arguments, but rather what they think the reviewers will accept. Publishing will become a little bit less honest, and it will be science, and scientists, that lose out.
At a time when we are all under so much pressure to compromise to hit artificial targets, peer review even if occasionally abused is an important bastion of scientific honesty. If we want to change the system, surely we should focus on changing what people believe to be right rather than creating yet another norm to perform against.
This is an earlier version of a piece that appeared in Times Higher Education, 10th December 2015 p. 38 under the same title.
About Paul Benneworth
Paul Benneworth is a Senior Research Associate at the CHEPS. Paul’s research interests relate to the relationships between university and society, particularly focusing on regional engagement, social innovation and policy-making. Paul is an occasional contributor to the LSE Impact blog, and tweets as @heravalue.
Follow me on TwitterMy Tweets
- An unusual contribution from higher education to building the future? 15 March 2018
- University funding needs a revamp 21 February 2018
- The First Law of Quality Assurance: Conservation of Quality Work 18 February 2018
- In need of research on research: Encourage replication studies 5 February 2018
- Lifting university mission overload to unlock regional potential 19 January 2018