The primary goal of the Journal of Open Source Software (JOSS) is to give researchers who develop, contribute to, and maintain open source software a means to get citable credit for their work within today’s research ecosystem. In short, JOSS is a developer-friendly journal for publishing research software packages. It’s also an online academic journal (ISSN 2475-9066) with a formal peer review process that is designed to improve the quality of the software submitted by making sure it meets minimum standards and includes standard identifiers for content on digital networks (Digital Object Identifiers, or DOIs) for all accepted papers.
In a perfect world, we would want to see credit given directly to software, but in today’s research world, citations to papers and other products are the standard means for tracking the impact and use of research work. (Certainly there are problems with citations; this is simply a practical choice.) We recognize that for most researchers, papers, not software, are the currency of academic research, and citations are required for a good career. In academic and government research, citations of papers are one of the two main factors that count in hiring and promotion decisions, along with getting research funding. We’ve created JOSS so that software developers can get the same credit as traditional authors, by making it easy for developers to turn their software into citable papers.
After you've done the hard work of writing great software, it shouldn't take weeks or months to write a paper about your work. In fact, as we say on our About page, if your software is already well documented, then paper preparation should take no more than an hour.
This includes a simple submission workflow and extensive documentation to help you prepare your submission, which creates an issue in our GitHub repository. The paper itself is a short markdown file that should contain:
- A summary describing the high-level functionality and purpose of the software for a diverse, non-specialist audience
- A clear statement of need that illustrates the purpose of the software
- A list of key references, including a link to the software archive
- Mentions (if applicable) of any ongoing research projects using the software or recent scholarly publications enabled by it
After this, it’s our turn. An editor-in-chief and 18 topic editors make up our editorial board. The editor-in-chief does a brief scan to make sure the paper looks suitable for JOSS (is it about a software package likely to be used and cited in the research ecosystem?), then an editor takes responsibility for the rest of the review process. The editor finds one or more reviewers who agree to review the paper, creates a new GitHub issue for the review itself, and closes the pre-review issue.
A bot (named Whedon) monitors the JOSS issues, and acting on keywords from editors, performs many of the core mechanical parts of the process, such as assigning reviewers and editors, creating review issues, and compiling PDFs upon acceptance.
The review issue contains a checklist of 18 items for the reviewer(s) to work through, including agreeing with JOSS’s policies on conflicts of interest and code of conduct; general checks on source code availability, use of an OSI-approved license, matching the software version in the paper and the repository and confirming that the submitter is an author of the software; checking on functionality, including installation and performance claims, if appropriate; checking on documentation, which should include a statement of need for the software, installation instructions, example usage, and documentation of functionality, tests, and community guidelines; and finally, checking on the software paper itself, including ensuring that the list of authors is reasonable, the statement of need is in the paper, and that sufficient and well-structured references are present.
Because this is done via a GitHub issue, when a reviewer finds a problem, they can comment in the review issue or open a new issue for the specific problem in the repository of the submission being reviewed. The article submitter can then interact with the reviewer, asking questions, making changes, etc. until the reviewer is satisfied, as moderated by the assigned JOSS editor. The goal of JOSS is not to reject a percentage of papers (although some rejections do happen when authors decide to withdraw rather than revise the submission based on the reviewer’s feedback), but to improve submissions until they are acceptable. This usually takes a few weeks, but if everyone is very responsive, it has been known to be completed in a day. However, in some cases, particularly with larger projects, the review process can take much longer, especially when documentation is unclear or features need to be added.
Once the reviewer(s) and editor are satisfied with the paper, the submitting author deposits a copy of the source code repository in an archival data repository such as Zenodo or figshare. A permanent link to this software archive is then added to the paper, which is converted from markdown to PDF, and article metadata is deposited in Crossref. The review issue is closed, and the JOSS website is updated with the final version of the JOSS paper (PDF), including its DOI issued by Crossref. Just like the papers being reviewed, the final articles and metadata are versioned in a public repository as well as the JOSS website.
Now the authors have a paper with a DOI for the software and can add this information to their repository, in either the README or a CITATION file, telling users how to cite the software in a way that fits naturally into today’s research ecosystem. When users cite the software, the authors will be able to see this and get credit for the software, just as they would had they written a research article.
In its first 19 months (May 11, 2016 - Dec 10, 2017), JOSS has accepted 187 papers. You can read more about JOSS and its status in an arXiv preprint.
1 Comment