The OpenStack project feels different from other open source projects to me. Let me try to explain.
Henrik Ingo did an excellent analysis of open source project size versus governance structure a few years ago. Essentially, the nine largest, most vibrant open source communities were anchored inside nonprofit foundations. The 10th largest was 10 times smaller than the ninth and existed inside a corporation. Like a good engineer, Henrik provided his data and listed his assumptions. He was not suggesting that the growth was causal, simply that there was a strong correlation. He rightly observed that it will be fascinating to see what it all means using OpenStack as an example when he presented his findings in summer 2011. (He wrote an excellent follow-up post comparing cloud projects later.)
Henrik’s analysis and observations presume certain criteria about how the open source licensed project must be functioning in and of itself. It has to be a “well run project” before getting to the foundation amplifier. Lots of people have described essentials of successful communities. Some of my favorites include Dave Neary, Donny Berholz, and Kohsuke Kawaguchi. And, of course, there’s Jono Bacon’s essential reference. I tried to capture and collect together the patterns and practices of successful open source projects in a blog post a while ago.
Paula Hunter and I offered rationale for the foundation amplification when we were executive and technical directors at the Outercurve Foundation. What might be the cause of the amplification? We believe it comes from the one thing all foundations do for their stakeholders around their projects. Foundations exist at the behest of their stakeholders to make clear the IP management requirements and provide IP risk mitigation. The project community must be well run from engineering and governance perspectives, but once a foundation exists, corporate participants have a clear on ramp for participation and the investment in the community can grow substantially.
So solid engineering practices + strong community governance + clear IP management enables growth. So far so good.
But OpenStack is somewhat unique as an open source project. It was 2010. Amazon had enormous early mover momentum in providing cloud services (2006). Eucalyptus was open source licensed, but controlled by a single vendor (née 2009). Cloudstack was just published under an open source license (May 2010), but still closely held by Cloud.com. The time was ripe for a vendor-neutral option. OpenStack was announced into existence by then Rackspace CEO Lew Moorman and NASA CTO Chris Kemp at OSCON 2010.
A large collection of vendors jumped early and hard onto a simple infrastructure to evolve it rapidly forward into a marketplace of potential customers. The project governance was created and people started gathering at summits. OpenStack was correctly forced into a neutral, non-profit foundation within two years to clarify IP ownership. (Everyone had MySQL as an example on their mind, as it had been bought by Sun Microsystems (2008), and then bought and ransomed by Oracle (2010).)
But here is where OpenStack begins to break patterns. There was relatively little code at the start. It was created from the start to engineer forward. OpenStack is going through forced growth in a time frame that is a 20-25% of the time of other large-scale, successful open source licensed infrastructure projects. Instead of 20 years, the effort is becoming enormous in four, already driving serious vendor-led products to market.
| Linux kernel | Apache (httpd) | gcc | OpenStack | |
| Project started | 1991 | 1995 | 1987 | 2010 |
| Foundation formed | 2000 | 1999 | 1987* | 2012 |
| Re-architected | 2002 | 2002 | 1998 | ??? |
| Lines of code at midpoint | 4,000,000 | 980,000 | 1,150,000 | 500,000 |
| Lines of code today | 17,000,000 | 1,700,000 | 6,900,000 | 2,300,000 |
| Contributors at midpoint | 186 | 17 | 50 | 174 |
| Contributors today | 1,000 | 17 | 94 | 575 |
Notes: GCC started in the Free Software Foundation; data pulled from Ohloh.net, Wikipedia, and interviews
OpenStack didn’t evolve in simple use cases with the experimental and experiential use that other infrastructure open source-licensed projects saw (e.g. Linux, Apache) before large vendors got involved. OpenStack continues to demonstrate enormous growth and participation. As vendors begin to develop cloud delivery products and services out of the various OpenStack projects, they are discovering holes in functionality requiring them rapidly to create new OpenStack projects to fill the gaps. Vendors are also starting to discover that OpenStack itself may not scale to the needs of certain industry desires (e.g. Telco desires around NFV). As well, all core infrastructure open source projects hit points in their organic histories where a reset was required in the architectural design to account for the stresses of new use and deployment. Linux, Apache, and gcc were each re-architected at points in their histories to accommodate the organic growth of the project into new deployments and use.
Some interesting questions:
- When do the developers at the heart of OpenStack projects collectively re-architect/re-factor/re-write core OpenStack components to scale to the real world workloads that people are discovering customers need to manage over the next 5 years?
- How does the OpenStack Foundation capture those consumer requirements in a fiercely competitive landscape of competitors?
- How will the OpenStack Foundation adapt to the needs of its vendor, user, and consumer stakeholders?
Open source communities are amazingly adaptable organisms. It will be fascinating to watch OpenStack as a community of projects and as a foundation continue to grow and evolve to meet the challenges of the cloud marketplace.
Originally published at Once more unto the breach. Republished here via Creative Commons.
1 Comment