DuckDuckGo is a search engine known for putting privacy first for users. So, when we passed 3 billion annual searches last year, we knew it was critical that we continue to serve users without sacrificing their privacy. The key, we realized, was open source.
First, we created an API. Then we started Instant Answers, where anyone can join us in improving search results. Instant Answers appear above the organic search results, and we now have hundreds of people from all over the world contributing. Since our first community commit almost four years ago, we've learned a lot of powerful lessons about what makes an open source community thrive. They've contributed hundreds of data sources and thousands of pull requests, which amount to around 800 Instant Answers at the time of this writing, capable of answering millions of searches.
We certainly have a long way to go, but we've learned that building a successful open source community—like any active community—comes down to providing value for individuals. Learn how to tune in to their motivations, find ways to align those motivations with your goals, and work hard to get barriers out of their way.
Communities always come down to people. The more DuckDuckGo has invested in helping them, the more we have received in turn.
Go with the flow
Community has always been a part of the mission at DuckDuckGo, even from the earliest days when Gabriel (our founder) was working in his basement on DuckDuckGo, asking for help and feedback on Hacker News and IRC. People suggested websites for topics that they believed should be ranked first (or highly). To make use of these valuable insights, we decided to create a way for users to build their own search results. So, we launched DuckDuckHack, and the idea was that anyone could improve the search results by using their favorite websites and data sources.
The first DuckDuckGo Quack & Hack, in the headquarters in Philadelphia in 2012
Admittedly, the first Instant Answers were more Easter eggs than anything. Soon enough, however, people began using the search engine to make useful sources of data more accessible. Over time, despite an ambiguous API, our search results gradually became better thanks to contributions.
We learned how to get inspiration from what was naturally happening. Where are people naturally offering to contribute? What kind of contributions do people naturally want to make?
Lesson #1: Double down on where your users are already contributing.
Be valuable to your community
As contributions began to come in, we realized that people were contributing for a whole range of reasons:
- To create a feature they personally wanted
- To gain visibility for another project they're working on
- To build professional experience and credentials
- To work with smart people all over the world
- To be part of the organization's overall mission
As we made decisions regarding our API, we kept these motivations in mind during important decisions, making sure they would still be relevant.
These motivations affected:
- How flexible we are regarding contributions and ideas
- What kind of data sources we accept, and how we credit them
- The communication tools and processes we set up for the community
We want to make sure that we don’t inadvertently make a change that dampens motivations—and do as much as we can to fuel them.
Lesson #2: Observe why people contribute, and amplify those motivations as your community grows.
Open source should help your company, too
Early on we were just three people managing a search engine with a million queries a day. It kept us pretty busy, and as the community grew, we found it hard to keep up with pull requests, issues, and quality documentation. After all, open source communities require ownership, leadership, and active curation.
Our mission was, and is, to compete with billion-dollar search giants, but we want to do so in our own way, with our own methods and practices. At the same time, we realized that the most delightful Instant Answers would not come from inside the company. Human knowledge goes way beyond what the best crawlers might ever figure out.
That's when we had our lightbulb moment: Community is the key to the search engine's success! So, we put the needs of our contributors at the top of our company priorities, and saw contributions increase in return.
Lesson #3: Open source communities require attention and investment. Make sure your community is a real priority and aligns with your company's goals.
Create a home for your community
At first, our collaboration channels were all over the place. We had a Uservoice ideas board, a Zoho forum, a separate docs site, GitHub for code... We needed to simplify the community's presence as much as possible, and create clear paths to each task. To make things easy for community members, there should be a central place for information and collaboration. It should help people decide where to pitch in, how to contribute, and how to communicate.
Lesson #4: Centralize your collaboration channels, documentation, and navigation.
The Kochi meetup group in India taking part in the second Global Quack & Hack in 2016
Make it easy to start
It may sound ridiculous, but our original development environment took two hours to set up. It required local installation of a custom tool, plus all its dependencies.Then, we switched to a cloud-IDE with a ready-to-go environment, and it now takes 5 minutes or less. The result? More than twice the number of unique contributors every month.
The most intimidating part about contributing to open source is getting up to speed. Do everything you can to lower barriers to entry.
Lesson #5: Make logistics as easy as possible for someone new to get to their first commit.
Make it easy to win
The best source of open source contributors is people who are new to open source. Many developers are looking for that first opportunity to get involved—one where they feel they can make an impact.
A problem we had at first was that it was difficult to complete one's first Instant Answer. In addition to learning new technology, there were plenty of things to consider, including idea, data source, and design. One of the best investments we made in our open source project was making it easy to add a personal "Hello World" Easter egg to our live search engine. This made it easy to just start coding and focus on learning the system. When we did this, we were flooded with first-timer pull requests! Turns out that seeing code run live after just an hour is an incredible motivator.
Another early issue was that contributors could not preview their own work without an employee running a dev server with their new code. As you can imagine, this made development far less exciting. To solve this, we built a command line tool that allowed anyone to spin up a local server simulating the DuckDuckGo search engine, allowing contributors to immediately visually preview their work. This drastically increased the speed of developing an Instant Answer and made it much more fun.
Lesson #6: Invest time in creating instant gratification. Make it as easy as you can for contributors to close a simple loop.
Keep people around
The final step is to help contributors stay involved.
We learned that people stay engaged when they feel others are also engaged. When we respond quickly to pull requests, we find that contributors match our efforts. When we participate actively on the forums, people tend to use them more and engage us more.
If a contribution can't be directly incorporated, try to capture as much value from every contribution by applying their work somewhere else in the repository. No matter what, make sure to positively reinforce and acknowledge every effort and contribution.
Lesson #7: Every interaction with the community should make people want to stick around.
Moving forward
Despite learning these lessons, we certainly don't claim to have all the answers. We're constantly trying out new ideas, often based on contributor feedback, and seeing how the community responds. Even preparing this article has been a chance to reflect and consider new ways to improve and move forward, and hopefully help your community thrive too.
Incidentally, we've just announced the recipients of our FOSS donations for this year, which include various open source projects we admire. They all fit our theme of raising the standard of trust online, and we encourage you to support their activities.
5 Comments