Netflix's Spinnaker tale

How did Netflix's in-house engineers adapt to the sea change inherent in open source collaboration?
270 readers like this
270 readers like this

In Dianne Marsh's keynote at OSCON, one of the biggest open source conferences in the US each year, the Director of Engineering at Netflix explained a tale of two systems.

At Netflix, the Asgard continuous delivery platform was developed internally, then shared openly. Did this cause headaches? Yes.

Asgard was developed for a single use case, for a single cloud platform (Amazon), and for a single region within Amazon. Other cloud providers then forked the project and made changes (innovations) to the project. The headache came when those contributions (changes and innovations) needed to be implemented back into the main Asgard project.

Marsh explained they had not given enough thought to this process despite there being many projects who had done this before.

When it came time for Netflix to do it all over again with Spinnaker, they would do it differently. First, they engaged a diverse group of users and cloud providers to emulate and learn from. Then, they created a process that worked for Netflix engineers and their particular project, Spinnaker.

From the start, Spinnaker was developed in the open. Cloud providers contributed code that was specific to their platforms, which allowed for distributed maintenance and a community to grow around it, resulting in cloud deployment best practices for multiple industries.

Joining forces

Joining Netflix on this open source journey were major software players Google, Microsoft, and Kenzen. Great, right? These partners helped create buzz and press coverage, however, the attention generated more input from more industries than expected.

How would Netflix's in-house engineers adapt to this sea change?

  • Communicate regularly
  • Share the roadmap
  • Build trust and accountability

Clear communication, a consistent vision, and team trust were critical to the success of this endeavor. Netflix understands leaders of open source projects must hold them with an open hand—no one company can dominate the conversation. It's crucial that the team build a culture of valuing any input, from individuals and large organizations. Working in the open also helped keep the engineers accountable both internally and externally to their main work at Netflix, and to the open source project that now belonged to so many more than just those in the company.

Ruth Holloway has been a system administrator and software developer for a long, long time, getting her professional start on a VAX 11/780, way back when. She spent a lot of her career (so far) serving the technology needs of libraries, and has been a contributor since 2008 to the Koha open source library automation suite. Ruth is currently a Perl developer and project lead at Clearbuilt.

Comments are closed.

Creative Commons LicenseThis work is licensed under a Creative Commons Attribution-Share Alike 4.0 International License.