The scientific journal, Nature Methods, has made a transformation. From closed to open, the journal now embraces open science practices with the purpose of enabling true reproducible research.
This is an account of how this transformation came to be.
Open science is a collection of practices that are required to enable reproducible research. That is, reproducibility verification is the goal, and open science is an efficient path to reach that goal. It is by making openly available all of the materials that describe an experiment, that we enable others to perform independent verifications of that experiment. Some traditional journals are embracing open science practices with the goal of improving the reproducibility record. One is Nature Methods.
It was back on November 4, 1869, that Nature published their first issue. Since then, the journal has grown into a collection of journals on diverse scientific topics considered to be top-rated resources. Interestingly, back in 1864, science-teaching in schools was a concern as it is today (illustrated by one of the articles in that first issue). Then, after 14 decades of publishing scientific articles, a credibility crisis happened. The public found out that many published articles were not reproducible, and as a reaction to this revelation, some Nature journals began to adopt policies that require authors to share their code and data.
I think of Linus Torvalds’ famous quote: "Talk is cheap, show me the code."
The journal Nature Methods is one that has taken on this new "code of ethics", and has written and included a new principles in their instructions to authors:
"An inherent principle of publication is that others should be able to replicate and build upon the authors' published claims. Therefore, a condition of publication in a Nature journal is that authors are required to make materials, data and associated protocols available to readers promptly on request."
Software tools for scientific research is another important factor in helping scientisits and researchers create work that is reproducible. In 2007, Nature Methods wrote:
"Software that is custom-developed as part of novel methods is as important for the method's implementation as reagents and protocols. Such software, or the underlying algorithms, must be made available to readers upon publication."
"If a software program is the focus of the report, we expect the programming code to be made available. Without the code, the software—and thus the paper—would become a black box of little use to the scientific community."
“Some authors who favor the highest degree of transparency and sharing for their software elect to develop their programs in an open-source environment. By doing so, the authors not only provide accessibility and transparency, they also allow the community to build upon their own developments and make continuous improvements to the tool.”
More recently, this year Nature Methods wrote more on their motivation for embracing open science practices:
"The usefulness of computational methods can be improved by releasing code and designing software that supports reproducible research."
The application of these policies has already had an effect on the amount of code that is shared across the scientific community, but clearly, more remains to be done.
"Since 2007, we have published 133 articles with accompanying supplementary software files, of which approximately 70% included the source code. Although many researchers have clearly embraced the idea of releasing their code upon publication—thus increasing the usability, reproducibility and impact of their work—more could do so."
Knowing the magnitude of work to be done, and with a forward-thinking mindset, Nature Methods has embraced modern tools for code management.
"Nature Methods strongly encourages researchers to take advantage of the opportunity that code repositories, such as GitHub, provide to improve a software tool before submission. [...] Use of a repository can also improve peer review by streamlining and exposing code revisions."
The source code for more open science is not enough, we also must also implement recipes for building and running the code.
"But even code release is insufficient when the software is used to acquire new biological results for publication. Nearly all software has user-defined parameters that can or must be tuned to the data characteristics or analysis goals. These parameter values should be reported alongside the output. This practice helps ensure reproducibility even in the lab doing the work."
Another critical method for making research and science more reproducible is the practice of testing software regularly:
"Unstable code that fails to provide consistent output can also compromise reproducibility. Here, the application of good software coding practice such as unit testing and automated builds can help ensure that code changes do not have unexpected consequences that alter the output obtained from standard inputs."
The 2014 editorial by Nature Methods closes with this note:
"An open implementation that supports reproducible research not only provides confidence in the performance of a method but increases the likelihood that other researchers can use and build upon it."
More specific guidelines for scientific software publication were provided by Nature Methods this year in a blog post.
We need more respected, traditional journals revising their principles to be more open. By making these steps clear, Nature Methods has made their transition to embracing the modern practices of scientific research easy for their authors to follow, scientists to understand, and for others to emulate. Who's next?
View the complete collection of stories for Open Science Week.
Comments are closed.