In praise of harsh reviews

I read the recent Times Higher Education feature criticising supposedly bad peer review with a growing sense of unease. Not so much the criticism of peer review, but the ideas that peer review could be saved by shackling referees’ freedom to write intemperate reviews.

Entitled ‘the worst review I’ve ever received’, the piece concluded with a tale of unacceptable personal abuse. That referee had used their freedom to stoop to criticising the paper for its supposedly sub-tabloid writing style, abysmal grammar, equating it to a mid-level undergraduate essay.

Luckily that author could publish the piece elsewhere to plaudits, perhaps inviting us to conclude there’s something fundamentally wrong with peer review. It’s part of a growing chorus of voices calling for its wholesale abolition or change, making it open or post-hoc.

Peer review is clearly idiosyncratic, and these contributors made much of its randomnity. Some criticisms imply it no longer serves as a dispassionate system of measured scientific governance, but has descended into a snake pit of personal abuse, petty jealousies and pet theories.

And as the editor of a journal, Regional Studies, Regional Science, that proudly offers early career authors a mentored publication route, I’d be happy if peer review become just one tool available to authors trying to make the best ideas public. But I’d be wary of calls to temper the peer review process.

I’d even go so far as arguing that the option to write harsh reviews is central to any thriving science system that prizes integrity over system-gaming.

I must here declare an interest, because the comments the last reviewer received may very well have been mine. If not, then I have certainly written reviews – no more than a handful – that have taken a harsh line.

But I’ve also received two particularly harsh reviews myself that stuck in my memory. The first was as paper I wrote as a postdoc, where one paragraph said that my paper’s whole premise was based on my excitement with a literature that was probably only new to me. The second was an eight-page line by line demolition of a qualitative paper by a reviewer who apparently wanted p-values.

And although each time I was stunned at first, I have to concede that the reviewers were almost certainly right. Both papers were submitted to leading journals that favour a very particular kind of article both in terms of style as well as content, and my research was not a comfortable fit in either.

If the reviewers hadn’t sent the harsh reviews, then the editor might have been tempted to give me a chance to squeeze my findings into the journal style. With the lure of a leading publication, the younger me might have been tempted to ‘play the game’ to say more than my findings would strictly have warranted.

The exciting new idea I sat on for a decade and finally it became a few paragraphs in a publication with two other authors, and the other paper matured through a European Network of Excellence to find a good home. In both these papers, I believe we really say something novel, and make a scientific contribution, where a more tortuous fitted paper may have failed.

I feel strongly that the harsh reviews I received exemplified peer review working correctly. They helped small ideas to grow until they was ready to make a contribution to a collective research community striving to create good new knowledge.

That brutal honesty has two sides – I’ve also written reviews where I have almost pleaded in the review for an unorthodox, badly expressed idea to be given a chance, again underpinned by a profound belief that that was the right thing to do.

Harsh reviews provide a brutal honesty allowing scientists to take decisions they believe to be right. Many of the proposed changes risk losing this space for brutal honesty.

Moving to open review may see reviewers giving the reviews that are good for them, on a Muggin’s turn basis, not the reviews they honestly believe are right for scientific advance. Editors will make calculating decisions, mechanically selecting only those papers that pass the reviewers’ beauty context rather than taking a punt on the outsider.

Authors will respond by not presenting their ideas and arguments, but rather what they think the reviewers will accept. Publishing will become a little bit less honest, and it will be science, and scientists, that lose out.

At a time when we are all under so much pressure to compromise to hit artificial targets, peer review even if occasionally abused is an important bastion of scientific honesty. If we want to change the system, surely we should focus on changing what people believe to be right rather than creating yet another norm to perform against.

 

This is an earlier version of a piece that appeared in Times Higher Education, 10th December 2015 p. 38 under the same title.

 

About Paul Benneworth

Paul Benneworth is a Senior Research Associate at the CHEPS. Paul’s research interests relate to the relationships between university and society, particularly focusing on regional engagement, social innovation and policy-making. Paul is an occasional contributor to the LSE Impact blog, and tweets as @heravalue.

Advertisements

Tags: , , ,

Leave a Comment

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: