Lachlan Coin writes on how peer review is changing

Contact: twitter @lachlancoin

http://figshare.com/articles/Making_peer_review_free_and_open/1580088

Peer review is not open.  Passing peer review asserts to scientists and the public alike that the methodology was sound; that the conclusions are correct; that the experimental protocols work ;  that policy should be written; that medical  interventions should, or should not be made.   When some of these claims are later retracted, both scientific and public trust in peer review  and the scientific method is eroded.   Imagine then, if the entire peer review literature were open, as it already is in a handful of journals including  BMJ Open, Gigascience and PeerJ.  Journalists, scientists, policy-makers, doctors and patients could assess how rigorously the peer-review process was applied and how well the authors were able to address the issues raised. Rather than seeing the scientific literature as uniformly correct, we could begin to accept  that every  manuscript has limitations as well as strengths.

Publons is a start up from New Zealand which is making huge in-roads towards making peer review more open.  Publons has enabled reviewers to publish ~10,000 reviews under a CC-BY license. The vast majority of these are pre-publication peer review (although the reviews are not made public until the article is itself published) and are now cross-referenced to the original articles via Europe PubMed Central.    Publons also provide the option for reviews to be registered but not shared publicly,  enabling reviewers to be credited for their reviewing activity.

The “slow, cumbersome and distorting practice of pre-publication peer review”  has led PLOS co-founder Mike Eisen to advocate abandoning pre-publication peer review altogether and switching to a  model in which papers are published without review and subsequently evaluated openly by the community post-publication.  Such  services are now provided by F1000, ScienceOpen and The Winnower.  PubMed Commons  is an National Institute of Health run service which enables any academic (listed as an author on a PubMed-indexed paper) to comment on another PubMed listed paper.  PubPeer allows anyone to comment anonymously on any published paper, which has on several occasions led to retractions.

A more popular form of the ‘publish-first-get-reviewed-second’ model is provided by preprint servers. Posting preprints to arXiv  is common practice in mathematics and physics.  With the launch of bioRxiv  this is gaining traction in biological sciences. The majority of preprints submitted to bioRxiv are published in a peer-reviewed journal within 12 months.  Preprint servers have essentially made sharing scientific manuscripts a free service. The operating costs for arXiv are estimated to be US$826,000 p.a, which is supported by a membership model in which participating universities contribute up to US$3000 p.a.

Peer review, however,  is still not free, both in the sense that it costs money, and also that the ways in which it can be accessed are limited.  As an author, I can choose to give up my copyright and  restrict who can access my work by submitting to a subscription journal, or I can choose to pay an Article Processing Charge (APC)  of anywhere between US$695  and US$5200 by submitting to an open-access journal.  Both types of journals ultimately access the same pool of reviewers to provide peer review.  Either way, publishers make lucrative operating margins by controlling access to peer review.  It is ironic that the only sense in which peer review is free is that the reviewer is not paid by the publisher for their effort.

I am co-founder of another Australasian startup (Academic Karma) whose mission is to  make peer-review free as well as open.  We  envisage a ‘1. post-preprint; 2. get peer-reviewed and 3.  submit to a journal’  model of scientific publishing.  In order to achieve this, we have launched a pilot ‘global peer review network’ together with librarians from The University of Queensland, Imperial College London, The Australian National University and Cambridge University.   Any auhor from one of these universities can use this network to access peer review  for a arXiv or bioRxiv listed preprint outside the journal system. The reviews, together with an editorial summary of the strengths and limitations of the paper are collated into a document which can be submitted together with the manuscript for consideration at an  open-access journal.  The reviews will be published ( at http://academickarma.org/reviews) once the manuscript is published.The author pays for peer review not in dollars, but with ‘karma’ they earned by reviewing for others. While there is no penalty for a karma debt, we hope this system helps remind reviewers to try to perform as much review as they consume – an absolute necessity for the system to be self regulating.

Although it has been almost 15 years  since the open-access publishing movement was launched in earnest with the establishment of the  Budapest Open Access Initiative,  the founding of  BiomedCentral, PLOS’s open letter to scientific publishers and then the launch of PLOS as an open access publisher, publishing in open access journals is still a long way from reaching 100% penetration.   Perhaps one of the main remaining reasons for this is cost – many researchers, particularly junior researchers face tough choices in deciding between paying to publish or paying for other lab expenses to further their research.   Co-ordinating peer review has been estimated to make up from 25%, to almost all the running costs of an online open access journal  We hope that providing high quality open peer review for free prior to journal submission will enable open-access journals to drop their APCs, thus making open access publishing more accessible to all.

About the author: Lachlan Coin is Group Leader, Genomics of Development and Disease Division
Deputy Director, Centre for Superbug Solutions at the University of Queensland

Conflict of interests:  Lachlan Coin is the founder of Academic Karma