Peer Review
Weighing up the benefits of the peer-review system
The peer-review system isn’t perfect but people are hesitant to abandon it in favour of post-review commenting, concludes a debate at the ALPSP annual conference. Siân Harris reports
In many industries people are reluctant to contribute comments and ask questions at the end of a conference session. Not so the scholarly publishing industry. This was particularly true in the lively debate about peer review that formed part of the recent ALPSP conference in Wyboston, UK in September. The debate centred on the question of whether Web 2.0 should replace the traditional peer- review process. Richard Smith, director of UnitedHealth Chronic Disease Initiative and board member of the Public Library of Science, lead the debate for this motion, while Tracey Brown, managing director of Sense about Science, lead the counter argument. ‘All the evidence says peer review is useless.
Peer review is not working in sifting out stuff that is poor quality. We have examples of really appalling papers that get through,’ argued Smith. ‘The real peer review is afterwards when people decide if it’s important.’ He went on to point out that peer review
slows down the publication process. ‘There are still plenty of journals that take years to publish and it is a very expensive process. I’m arguing that that time is largely wasted.’ Smith also highlighted a lack of agreement between reviewers on the value of a paper and cited the example of a study where 20 papers that had already been published were retyped and resubmitted to the same journals with different names and institutional affiliations. ‘Only three of the papers were recognised as having already been published and eight were rejected,’ he said. ‘Scrap peer review and open science up to let it breathe,’ he concluded. Tracey Brown of Sense about Science, an organisation that helps
www.researchinformation.info
people make sense of science on the internet, disagreed: ‘We’re not in the least bit interested in publishing models but we care about quality. Peer review is a system of organised human judgement and it introduces some kind of regulation.’ Answering Smith’s comments about bad
papers being published, she said, ‘If you look at the great sea of information the problems with peer review are just drops in the sea.’ The big question, she pointed out, is
whether there is a realistic alternative to pre-publication peer review. One option often suggested, and favoured by Smith, is post-publication review. Brown was doubtful: ‘The editor puts an article up, but nobody comments on the methodology so the editor ends up soliciting comments. If we do away with peer review we will just end up replacing it with something similar,’ she said. So, what is the purpose of peer review? Its role
in ensuring quality is a big draw for Brown and for the people whom Sense about Science help. ‘The process starts in authors’ heads because they know it’s going to reviewers,’ she said.’ Charlie Rapple of TBI Communications
agreed on the issue of quality: ‘I don’t want the world to
decide [whether a
paper is high quality]. I want qualified people to decide,’ she observed, although Richard Smith argued that in the post-review situation the comments would be from qualified people too.
ALPSP CEO Ian Russell noted another
benefit of the current review system: ‘Peer review is there to help academic community deal with information overload,’ he said. This observation was echoed in another
of the sessions at the ALPSP conference by David Crotty, executive editor of Cold Spring Harbor Protocols: ‘I’ve never met a top scientist with a lot of spare time. Leaving comments on published papers takes time to do it right and has no career benefits,’ he said. ‘If you review three articles a month and read 50 you know those 50 have been peer reviewed too but if you just have post review then you need to review 50 articles a month.’ Crotty added that using social media doesn’t necessarily select for quality but for popularity and that people are actively discouraged from publicly criticising the work of others. Tracey Brown agreed: ‘There are discussions that need to happen in private but if you put them in public, they would be defamatory. For example, “Do we think the pharmaceutical industry has put pressure on these authors?”’ Although Richard Smith’s arguments
against peer review did not sway the audience – the votes at the beginning and end of the session were overwhelmingly against dropping peer review in favour of Web 2.0 – the discussion did highlight many limitations of the current approach. As Peter Richardson, managing director of Royal Society of Medicine Press observed: ‘Richard identified the real problems of peer review but there
isn’t an alternative that is better.’ Wendy Warr suggested that the debate move beyond post-publication
review. ‘We haven’t really looked at Web 2.0 beyond posting comments. We should look at things like web metrics. These things are complementary.’
Research Information December 2010/January 2011 11
Page 1 |
Page 2 |
Page 3 |
Page 4 |
Page 5 |
Page 6 |
Page 7 |
Page 8 |
Page 9 |
Page 10 |
Page 11 |
Page 12 |
Page 13 |
Page 14 |
Page 15 |
Page 16 |
Page 17 |
Page 18 |
Page 19 |
Page 20 |
Page 21 |
Page 22 |
Page 23 |
Page 24 |
Page 25 |
Page 26 |
Page 27 |
Page 28 |
Page 29 |
Page 30 |
Page 31 |
Page 32 |
Page 33 |
Page 34 |
Page 35 |
Page 36 |
Page 37 |
Page 38 |
Page 39 |
Page 40 |
Page 41 |
Page 42 |
Page 43 |
Page 44