Be careful what you wish for

Peer Review Monster

Peer Review Monster Gideon Burton

 

 

 

Some say peer review of research articles is broken and needs to be changed, others that peer review needs a killer app that allows readers within a social network to vote a paper up or down. Cameron Neylon says Don’t (peer review) – the risks outweigh the benefits of publication. He says “publishing anything does no harm’ whereas peer review is costly in expert time.

jch Mary Mc Kinnon

 

 

 

Before we throw the baby out with the bath water, let’s think about what’s good about peer review and what could be improved.

Peer review can be good when:

  • it offers blind review when reviewers do not know gender/age/race/status of authors (unless this is revealed by text and context)
  • reviewers offer constructive advice for authors to improve their article/ creative work
  • it winnows submissions to journal (or other publication outlet) to improve quality of what is presented and filters content for readers

Peer review can be bad when;

  • it introduces delay in publication of work (tardy editors/ reviewers, backlog of accepted articles)
  • opinionated reviewers destroy confidence of authors

Alternatives to ‘traditional’ blind (author does not see identity of reviewer) or double-blind (neither author or reviewer see each others’ identity) reviewing include open peer review and the various recommender systems such as reviews on Amazon.com, Facebook ‘likes’, etc.

What are the risks of abandoning peer review ?

Well of course, the short answer to that is “I don’t know” but a search of scholar.google.co.uk reveals that it is a topic worthy of discussion.

“The refereed journal literature needs to be freed from both paper and its costs, but not from peer review, whose ‘invisible hand’ is what maintains its quality,” Steven J.Harnad from Peer Review : A critical inquiry by David Shatz.

What concerns me is that abandoning peer review will leave us more exposed to the gender, race and other identity biases that remain with us, and flourish in online communication where identity is key.

What do you think?

Do the benefits of abandoning peer review outweigh the disadvantages, and why?