How strong is peer review in open source?

No readers like this yet.
User experience vs. design

Opensource.com

While code review is commonplace amongst proprietary software development firms, the nature of open source software development often makes standardized code appraisal difficult to achieve. As such, successful industry leaders or project managers in the open source community frequently employ peer review techniques as a criteria for quality control in their development cycle.

An example of a standard open source peer review process begins with a software author submitting their code and documentation to their project's mailing list. It is then examined by other contributors and project managers; potential problems and improvements are discussed amongst the community and author before the changes are either accepted or rejected. GitHub uses the version control software Git to offer a streamlined system in which project managers can oversee their source code while still allowing for code review. Due to its ease of use, GitHub has become a popular host for version control and code review, with over 2,000,000 repositories uploaded to the site as of 2011.

While there are tools available for developers to utilise quality control, the calibre of smaller project's code review techniques are often overlooked, which results in messy and unchecked patch releases. More mature open source projects are less prone to these issues due to explicit guidelines available to their developers. Even when smaller development groups do employ the use of software peer review, the results are hardly uniform. As Peter Rigby states in his paper, "Peer Review on Open Source Software Projects: Parameters, Statistical Models, and Theory," "We find that OSS peer reviews are conducted asynchronously by empowered experts who focus on changes that are in their area of expertise. Reviewers provide timely, regular feedback on small changes. The descriptive statistics clearly show that OSS review is drastically different from traditional inspection."

Evidence suggests that peer review is potentially the most effective form of finding defects in software. In "Code Complete," author Steve McConnell states that effectiveness of code review for determining faults in software is between 30% and 35% more effective than standard unit testing. An advantage open source development offers to new members of a project is that reviewing code is an easy way for unfamiliar developers to simultaneously become productive in a project as well as ingratiate themselves within the culture of the community. This also results in knowledge of the project's entirety to be distributed amongst a team of programmers without diluting the community's priorities.

Open source development teams need to focus on improving their approach to peer code review or begin formulating software evaluation programs. Fortunately, there is plenty of documentation available for this very activity. Articles pertaining to the importance and the improvement of software peer review are a great resources; one that I like is: 11 Best Practices for Peer Code Review by SmartBear Software. It is a short summary of basic software peer review principles. While the article isn't particularly in-depth regarding specific details of code appraisal guidelines, it does provide a short list of 'rule of thumb' principles in guiding developers toward more streamlined and useful code review techniques.

As open source software is becoming a more popular alternative to proprietary software, the development proficiencies behind the products must be refined to in order to improve the community's reliability. Software produced by teams employing more adequate and standardized peer code review techniques yield more user friendly and dependable programs. The time required to ensure a standardized quality control scheme is worth a more polished product.

User profile image.
Eden is a software developer, focusing on the web, based in London. They are an advocate of open source software, and likes to contribute actively to the open source community.

7 Comments

It seems to me peer review by outsiders has become *steadily more difficult* with the adoption of formalized "collaboration" tools and hosted services.
A generation ago, I fixed a bug in the Linux kernel driver by mailing the driver's author a uuencoded patch and a short description of the problem. I also suggested a feature (mouse integration) to the author of the leading vi clone at the time, and he implemented it. (Vim adopted it a few months later, it's :set mouse=a)
I would have no idea how to accomplish either of those things today. Or even how to find out how to do them. Have you ever tried getting a bug in Firefox fixed, or even acknowledged?
Perhaps open source developers are just defending against kook spam and sabateurs, but they've erected gauntlets to outside reviewers that didn't used to be there. If you're looking for why shellshock and heartbleed took so long to get reported and fixed, start there.

s/the Linux kernel driver/a Linux device driver/

In reply to by Guest (not verified)

I agree with you one hundred percent. I think developers need to widen their current approaches to editing their code not only within their companies, but also with the general populace. If they are afraid of sabateurs, they need to employ more efficient bug report moderators, not silence the public. I understand the problem with that is funds, but like with most open source projects, there are people willing to give their free time to such a cause.

In reply to by Guest (not verified)

I agree with you one hundred percent. I think developers need to widen their current approaches to editing their code not only within their companies, but also with the general populace. If they are afraid of sabateurs, they need to employ more efficient bug report moderators, not silence the public. I understand the problem with that is funds, but like with most open source projects, there are people willing to give their free time to such a cause.

In reply to by Guest (not verified)

Interesting article! The open source analytics platform alternative to Google Analytics, Piwik, is considering Pre-merge code reviews for most contribs including from core team members: https://github.com/piwik/piwik/issues/6916

Creative Commons LicenseThis work is licensed under a Creative Commons Attribution-Share Alike 4.0 International License.