Lately I feel like I'm trapped in an endless loop of a certain Steve Ballmer moment, except the refrain is “crowdsourcing, crowdsourcing” on one hand, and “collaboration, collaboration” on the other. It seems everyone has jumped aboard either the crowdsourcing or the collaboration train. Call me a fence-rider, but I'm staying firmly on the platform.
Sure, I believe there is wisdom in crowds. But there is also power in collaboration.
Why crowdsourcing works... but not that well
Statistician and wearer-of-many-hats Sir Frances Galton made a fascinating discovery at a livestock fair in 1906 (so the story goes). When he saw a weight-guessing contest open to the general public, he requested permission analyze the 787 guesses. Galton found that the mean of the guesses was off by merely a pound—in other words, the “average” of all guesses was remarkably close to the actual weight.
Like most popular tales, there's a bit of embellishment here. Galton actually published information about the median, not the mean, because he felt the median was a better indicator. It was nine pounds heavier, but still within 1% of the actual weight and thus highly accurate. But let's not bother ourselves too much with the details. There's wisdom in crowdsourcing, and Galton proved it.
Or did he?
Perhaps there is something fundamentally different about “sourcing” ideas and innovation from a crowd... and averaging their numerical guesses for a specific mathematical problem. It seems to me that most crowdsourcing more commonly asks a large number of people to be “lone innovators,” and expects that from that group a few excellent ideas will rise (amidst lots of static).
Still, having a number of bright individuals submitting ideas must be better than the common practice of relying on just one or two internal people to do the heavy-lifting. Despite his invaluable contributions to science (like standard deviation and “nature vs. nurture”), Galton serves as a reminder of the limitations of the lone innovator. The publication of his Hereditary Genius sparked the modern eugenics movement, leading to forced sterilization programs and various and sundry horrors.
So crowdsourcing has some obvious benefits:
- It's often accurate at predicting mathematical solutions.
- It brings more innovators into an organization to present solutions to complex problems.
But crowdsourcing nearly always excludes one powerful force: collaboration.
Why collaboration isn't much better
There's a difference between effective collaboration and ordinary brainstorming. Ideas produced entirely by a team tend to be worse—both on average and when comparing the best ones—than those produced when team members go off on their own and return to share.
In a 57-page paper titled “Idea Generation and the Quality of the Best Idea" (PDF), researchers Girotra, Terwiesch, and Ulrich cite some big problems with a purely team-based approach:
“In a comprehensive series of studies, [researchers have] identified that team brainstorming leads to production blocking (the inability to articulate ideas when others in the team are speaking), evaluation apprehension leading to censoring of potentially good ideas, and free riding (i.e., collective performance measures impeding individual incentives to perform).
[...]
Previous research has shown that team members affect one another’s perceptions, judgments and opinions. [Detailed observation] has found that often “high-status” members dominate the discussion.”
Consider your own experiences in team brainstorming sessions. I have certainly seen groups latch onto an inferior idea, then quickly dismiss better ones. And who hasn't had doubts about an idea with which their manager is enchanted, yet kept quiet for political reasons? Not to mention the problem of groupthink.
So what's the solution?
If crowdsourcing lacks the power of collaboration, and collaboration's weakness is that it limits the quality of ideas, where can a leader find the most innovative ideas?
The Wharton authors' research suggests a hybrid approach that looks something like this:
- Convene to discuss the problem and objectives.
- Give everyone a clear assignment: return with a specific number of ideas, by a set deadline.
- Work individually on the problem.
- Regroup to present the ideas and formulate a plan of action.
The study does not make many recommendations for this final step. I suspect that one en-vogue practice (writing all ideas on a whiteboard and giving each team member an equal number of sticky notes to mark their favorite ideas) likely produces better outcomes than choosing as a team, given the drawbacks of group dynamics mentioned earlier. Yet I wonder if an independent and knowledgeable panel, removed from the idea creation process, wouldn't make better selections.
Has anyone experienced this hybrid approach in action? How well does it work outside of a research lab?
6 Comments