The Open Science Peer Review Oath

from the transparent,-reproducible-and-responsible dept

Fri, Nov 14th 2014 12:49am — Glyn Moody

Open access is about making academic research more widely available, particularly when it is publicly funded. But there is a broader open science movement that seeks to make the entire scientific process -- from initial experiments to the final dissemination of results -- transparent, and thus reproducible. One crucial aspect of that complete process is peer review, whereby experts in a field provide advice about the quality of new research, either to editors prior to a paper being published in a journal, or more directly, by reviewing work publicly online. Recognizing the importance of this step for the integrity and validity of the scientific process, a group has drawn up what it calls the "Open Science Peer Review Oath": We have formulated an oath that codifies the role of reviewers in helping to ensure that the science they review is sufficiently open and reproducible; it includes guidelines not just on how to review professionally, but also on how to support transparent, reproducible and responsible research, while optimising its societal impact and maximising its visibility.
The Oath's 17 components include commitments to act fairly and ethically, for example, the following: While reviewing this manuscript:

i) I will sign my review in order to be able to have an open dialogue with you

ii) I will be honest at all times

v) I will not unduly delay the review process

vi) I will not scoop research that I had not planned to do before reading the manuscript

vii) I will be constructive in my criticism

x) I will try to assist in every way I ethically can to provide criticism and praise that is valid, relevant and cognisant of community norms
It also includes actions specifically designed to foster science that is truly open: xi) I will encourage the application of any other open science best practices relevant to my field that would support transparency, reproducibility, re-use and integrity of your research

xiii) I will check that the data, software code and digital object identifiers are correct, and the models presented are archived, referenced, and accessible

xiv) I will comment on how well you have achieved transparency, in terms of materials and methodology, data and code access, versioning, algorithms, software parameters and standards, such that your experiments can be repeated independently

xv) I will encourage deposition with long-term unrestricted access to the data that underpin the published concept, towards transparency and re-use

xvi) I will encourage central long-term unrestricted access to any software code and support documentation that underpin the published concept, both for reproducibility of results and software availability
Although the framing of an "Oath" for open science peer review may sound rather over the top -- slightly pompous, even -- it rightly underlines the seriousness with which peer review ought to be conducted. It remains to be seen what kind of response it receives from the wider scientific community, and whether it becomes a fixed element of the open science movement.

Follow me @glynmoody on Twitter or identi.ca, and +glynmoody on Google+

Thank you for reading this Techdirt post. With so many things competing for everyone’s attention these days, we really appreciate you giving us your time. We work hard every day to put quality content out there for our community.

Techdirt is one of the few remaining truly independent media outlets. We do not have a giant corporation behind us, and we rely heavily on our community to support us, in an age when advertisers are increasingly uninterested in sponsoring small, independent sites — especially a site like ours that is unwilling to pull punches in its reporting and analysis.

While other websites have resorted to paywalls, registration requirements, and increasingly annoying/intrusive advertising, we have always kept Techdirt open and available to anyone. But in order to continue doing so, we need your support. We offer a variety of ways for our readers to support us, from direct donations to special subscriptions and cool merchandise — and every little bit helps. Thank you.

–The Techdirt Team

Filed Under: knowledge, open access, open sciencie, peer review, sharing

7 Comments

If you liked this post, you may also be interested in...

Reader Comments

Subscribe: RSS

View by: Time | Thread

Anonymous Coward, 14 Nov 2014 @ 5:40am

I appreciate the idea behind this, but I don't think this will actually prevent incidents like that one scientist who "proved" that wine was amazingly healthy for you (and happened to own several vineyards). People who twist the truth to suit their own desires wouldn't think twice about breaking an oath.
[ link to this | view in chronology ]
tqk (profile), 14 Nov 2014 @ 7:48am

"cognisant of community norms "
What does this--^ mean, and what has it got to do with independent research? I don't want to see researchers hamstrung by tradition. They're *supposed* to be exploring the boundaries of the known and unknown, not giving fealty to ideas of common decency and accepted institutionalism and the "status quo."
[ link to this | view in chronology ]
Scott Yates (profile), 14 Nov 2014 @ 10:33am

What about negative results?
I know it is probably outside of the scope of this kind of oath, but I think there should be more work done to encourage publication of results that did NOT show what the researchers/scientists set out to. Negative results are being swept under the rug and ignored. This is a problem that really seems like it should be addressed.
[ link to this | view in chronology ]
- tqk (profile), 14 Nov 2014 @ 11:12am
  
  Re: What about negative results?
  I know it is probably outside of the scope of this kind of oath, but I think there should be more work done to encourage publication of results that did NOT show what the researchers/scientists set out to.
  
  I agree. Imagine:
  "In a recently completed, peer reviewed study, backed by four other studies replicating the experiment, it has been shown that violence on television, in books, and in video games shows no correlation with real world violence whatsoever. A related study shows the same is true for sexually related activities. It turns out that fiction is fiction and the reality of its observers is unaffected by fictional representations. This result is contrary to the researchers initial hypothesis and they continue to study their data in an effort to determine where they went wrong."
  
  In a perfect world.
  [ link to this | view in chronology ]
  - Scott Yates (profile), 14 Nov 2014 @ 11:22am
    
    Re: Re: What about negative results?
    I was not aware of this one and had to google it.
    
    http://www.gamespot.com/articles/research-shows-no-link-between-video-game-violence/1100-6423385/
    
    Thank you!
    
    Maybe there IS hope for more of this.
    [ link to this | view in chronology ]
toyotabedzrock (profile), 15 Nov 2014 @ 7:33pm

This will never work. You cannot have people who have not earned their reputation reviewing things.
The majority of society is not qualified to tell when a politician is lying.
Further there is too much to be gained in cheating this system and this lowers the price for them to do so.
[ link to this | view in chronology ]
Anonymous Defender, 18 Nov 2014 @ 9:15am

Rule #1 is a bad idea
De-anonymizing reviews is a bad idea. A review is either valid or it's not, but the reputation/identity of the reviewer should not be a factor in that determination. If reviewers are not anonymous, more deference will be paid to reviews by respected, elder members of the profession, and less to reviews by junior, younger members. In addition, de-anonymizing reviewers reduces the pool of potential reviewers or calls into question the quality of the review since, for example, a junior scientist is less likely to negatively review the work of someone she knows will be on a board making recommendations for future grant awards.

An editorial board can mitigate many of the bad effects of an anonymous peer review system, but can do much less about the deleterious effects of a non-anonymous system.
[ link to this | view in chronology ]