Andrey Petrov spoke at a Sourcegraph open source meetup about lessons learned from his successes and failures creating open source projects.
Andrey is author of the popular urllib3 library and several other popular open source libraries on GitHub. During the day, he works at Briefmetrics and is also a YC alum, ex-Googler, and a cat person. He writes mostly in Python, JavaScript, and Go.
Why open source?
Andrey began the talk by outlining some of the key benefits of open source:
- It's a good way to get free, replicated backups of your code.
- It makes it easy to link to and ask for help.
- It helps give you a bigger footprint of influence in the programming world.
- It lets you get the most out of sunk costs.
You've invested a lot of time and effort into your code. Open sourcing it makes it available and reusable. Employees come and go, as do companies, but open source code will always be available to you and others who find it useful.
A snippet from urllib3, a popular Python open-source library that Andrey created.
Open sourcing has many benefits, but there's a lot of open source code out there already. How do you make your project stand out? Here's 4 steps you can use to build great open source software.
Step 1: Define what success looks like
urllib3 is used by millions of people everyday, has 785 stars on GitHub, and has been growing since its creation in 2008. Andrey has another project, ssh-chat, that has 1,560 stars on GitHub, but is only used by about 15 people today. So which project is more successful? In order to answer that question, Andrey described how defining success at the beginning of project will help you understand if it's successful or not.
The beginning of every open source project looks like this:
- Build something. Anything.
- Get frustrated with the hardest parts, build libraries and tools to help.
- Discard the original project to focus on the libraries, which become your new project.
- Go back to step 2 and repeat.
The story of urllib3 begins with the problem of trying to upload billions of images to S3 in 2008. Using existing Python libraries, this would have taken more than three weeks because there was no good concurrency support. You could use s3funnel, a multithreaded S3 client, but managing threads is painful. You could use a worker pool in s3funnel instead, but existing HTTP libraries didn't reuse connections, and most solutions weren't threadsafe or lacked multipart (filepost) encoding. Enter urllib3.
A snippet of requests, a popular function in urllib3
In Andrey's words, "The solution space [in engineering] is about building something and solving a problem. For urllib3, the space was really small and the solution to other things were really useful, and that's what made it successful."
When defining a project's goals, think about how solving a small problem can make a large impact.
Another form of success in open source is exemplified by ssh-chat, an encrypted chat-over-SSH program written in Go. Andrey started ssh-chat as a weekend project. One of the main contributions to the project's success started with creating a thorough README. As Andrey puts it, "Having a great README is basically 80% of the work to success. You need to be able to answer three questions for your contributors: "Who else uses it?" "What do they use it for?" and "Where can I get more help?"
A snippet of ssh-chat, a popular ssh-chat program written in Go
But just creating an awesome README didn't kickstart the impressive growth of ssh-chat. To make it thrive, it needed more traction.
Step 2: Recruit core contributors
To spread the word about ssh-chat, Andrey started asking for help, asking for improvements, and finding ways to bring more people into the project. In order to help build interest, Andrey reached out to people on Twitter and offered free Go programming lessons in exchange for opening pull requests. This overcame the initial inertia of getting a few people involved and interested in the project. These early contributors eventually became champions of ssh-chat and started answering questions on Stack Overflow.
Another component of building and maintaining open source projects is learning to be inclusive. Andrey says "accepting pull requests very generously, and very graciously" was a key step in building more community interest and is something that many open source authors miss in the beginning. And asking specific people to make a pull request was much more effective than making more generalized asks of his Twitter followers.
Step 3: Market and promote your project
In general, programmers tend not to be self-promoting types, so actively marketing and promoting an open source project doesn't come naturally. With this in mind, Andrey identified ways that programmers can promote their projects without being self-aggrandizing.
One approach is to write interesting blog posts about your projects to provide clearer context of the project's story and mission. Medium is a great platform for sharing in the technology community and has more potential readers than a self-hosted blog.
Other opportunities for connecting with contributors that worked for urllib3 and ssh-chat included:
- Answer questions on Stack Overflow (set up alerts on Stack Overflow for specific topics)
- Participate in discussions on Hacker News, reddit/r/programming, etc.
- Sell to other open source projects and establish partnerships with them. "The only reason urllib3 is the most popular third-party Python library today is because it's part of requests."
- Feed the non-trolls: Getting upvotes on your announcement post is only half the equation. More activity and discussion yields more people clicking on it and more updates, so if you respond to almost every comment, then that's 2x as many comments.
Step 4: Profit?
"We want to be like Docker!" Don't we all?
Two things helped Docker develop into a widely adopted open source project:
- It's very well funded.
- It has thousands of contributors helping for free.
One of the key insights behind Docker's success is that they didn't over-focus on the business model until the OSS project was insanely popular. So how can you replicate this? Enter the "corp'en source model"—where open source work becomes the core competency and business of a corporation.
The Corp'en source model
Some of the most well-known companies that embrace corp'en source include:
- Docker (via an open core)
- Gratipay (via dogfooding their own product to receive revenue)
- Nginx (via consulting and support)
- Mozilla (via advertising)
- GnuPG (via charity and grants)
So how do you monetize your contributions as a free agent?
Well, you don't. Not directly, at least.
Despite widespread adoption of urllib3, Andrey shared that he only received $5 from a single user (which he appreciates!), proving that widespread adoption of a project does not necessarily provide a sustainable revenue model to justify the time and effort.
Some companies, like Stripe and Sourcegraph are sponsoring open source work.
Open source: just do it and do it right
Open source is incredibly valuable and rewarding, but to do it successfully, you gotta hustle.
In this case, much of the work that went into building urrlib3 meant focusing on a small, but common challenge, writing a great README, reaching out to people to contribute, and promoting the project through blog posts. It's also worth mentioning that it took some time for each of these projects to gain traction, and Andrey's commitment to improving the project over time helped build his reputation and respect within the programming community.
In short, here's how to make an open source project thrive:
- Define a small solution space that affects a lot of people.
- Write a great README and recruit people who can and will help build it with you.
- Market and promote the project with blog posts and social media activity.
- Identify and reach out to companies who will benefit from your work.
1 Comment