Crowdsourcing vs. collaboration: Which yields superior results?

No readers like this yet.
Crowdsourcing versus collaboration

Opensource.com

Lately I feel like I'm trapped in an endless loop of a certain Steve Ballmer moment, except the refrain is “crowdsourcing, crowdsourcing” on one hand, and “collaboration, collaboration” on the other. It seems everyone has jumped aboard either the crowdsourcing or the collaboration train. Call me a fence-rider, but I'm staying firmly on the platform.

Sure, I believe there is wisdom in crowds. But there is also power in collaboration.

Why crowdsourcing works... but not that well

Statistician and wearer-of-many-hats Sir Frances Galton made a fascinating discovery at a livestock fair in 1906 (so the story goes). When he saw a weight-guessing contest open to the general public, he requested permission analyze the 787 guesses. Galton found that the mean of the guesses was off by merely a pound—in other words, the “average” of all guesses was remarkably close to the actual weight.

Like most popular tales, there's a bit of embellishment here. Galton actually published information about the median, not the mean, because he felt the median was a better indicator. It was nine pounds heavier, but still within 1% of the actual weight and thus highly accurate. But let's not bother ourselves too much with the details. There's wisdom in crowdsourcing, and Galton proved it.

Or did he?

Perhaps there is something fundamentally different about “sourcing” ideas and innovation from a crowd... and averaging their numerical guesses for a specific mathematical problem. It seems to me that most crowdsourcing more commonly asks a large number of people to be “lone innovators,” and expects that from that group a few excellent ideas will rise (amidst lots of static).

Still, having a number of bright individuals submitting ideas must be better than the common practice of relying on just one or two internal people to do the heavy-lifting. Despite his invaluable contributions to science (like standard deviation and “nature vs. nurture”), Galton serves as a reminder of the limitations of the lone innovator. The publication of his Hereditary Genius sparked the modern eugenics movement, leading to forced sterilization programs and various and sundry horrors.

So crowdsourcing has some obvious benefits:

  1. It's often accurate at predicting mathematical solutions.
  2. It brings more innovators into an organization to present solutions to complex problems.

But crowdsourcing nearly always excludes one powerful force: collaboration.

Why collaboration isn't much better

There's a difference between effective collaboration and ordinary brainstorming. Ideas produced entirely by a team tend to be worse—both on average and when comparing the best ones—than those produced when team members go off on their own and return to share.

In a 57-page paper titled “Idea Generation and the Quality of the Best Idea" (PDF), researchers Girotra, Terwiesch, and Ulrich cite some big problems with a purely team-based approach:

“In a comprehensive series of studies, [researchers have] identified that team brainstorming leads to production blocking (the inability to articulate ideas when others in the team are speaking), evaluation apprehension leading to censoring of potentially good ideas, and free riding (i.e., collective performance measures impeding individual incentives to perform).

[...]

Previous research has shown that team members affect one another’s perceptions, judgments and opinions. [Detailed observation] has found that often “high-status” members dominate the discussion.”

Consider your own experiences in team brainstorming sessions. I have certainly seen groups latch onto an inferior idea, then quickly dismiss better ones. And who hasn't had doubts about an idea with which their manager is enchanted, yet kept quiet for political reasons? Not to mention the problem of groupthink.

So what's the solution?

If crowdsourcing lacks the power of collaboration, and collaboration's weakness is that it limits the quality of ideas, where can a leader find the most innovative ideas?

The Wharton authors' research suggests a hybrid approach that looks something like this:

  1. Convene to discuss the problem and objectives.
  2. Give everyone a clear assignment: return with a specific number of ideas, by a set deadline.
  3. Work individually on the problem.
  4. Regroup to present the ideas and formulate a plan of action.

The study does not make many recommendations for this final step. I suspect that one en-vogue practice (writing all ideas on a whiteboard and giving each team member an equal number of sticky notes to mark their favorite ideas) likely produces better outcomes than choosing as a team, given the drawbacks of group dynamics mentioned earlier. Yet I wonder if an independent and knowledgeable panel, removed from the idea creation process, wouldn't make better selections.

Has anyone experienced this hybrid approach in action? How well does it work outside of a research lab?

Tags
User profile image.
Rebecca Fernandez is a Principal Program Manager at Red Hat, leading projects to help the company scale its open culture. She's an Open Organization Ambassador, contributed to The Open Organization book, and maintains the Open Decision Framework. She is interested in the intersection of open source principles and practices, and how they can transform organizations for the better.

6 Comments

Very interesting! Thinking ..

What is Crowdsourcing? for this article, it appears to be cherry-picking multiple innovations from multiple innovators. What is Collaborating? for this article, it appears to be a team effort to produce an innovation.

Wikipedia is neither. It's a zillion innovations from a zillion innovators who, via communication, coordinate to build and maintain a whole slew of innovations and a bitchin' collection of informational articles.

The point is there are continuums of sources, and continuums of contributor communication/coordination. The two are not really directly related, but do share a correlative relationship.

When I think of crowdsourcing, I picture someone presenting a problem or need to a "crowd" of people who are independent of the organization. These individuals work independently and return with a huge number of suggestions or ideas or solutions, which are then handed back to the original problem-poser. He sifts through them and pulls out the best ideas and then implements one or more of them.

Collaboration, on the other hand, would be folks working together to solve a problem. Typical modes would include brainstorming, team sessions, task delegation, etc. It's marked by cooperative work, rather than independence.

Of course there is a spectrum; a crowdsourcing contributor might actually be two people working together, and when collaborating in a typical environment, some folks will bring independent work to the table.

But my point is that, according to some research, neither model is the ideal working environment for solving problems or building ideas. What's ideal is (1) presentation of the problem, (2) independent work on the problem (and idea generation), and (3) regrouping to collaborate on the shared ideas.

And in my experience, projects are very, very rarely approached that way. Typically it begins with a brainstorming session and then independent thought or work... or a crowdsourcing initiative followed by idea-sifting.

My experiences suggest little gets done until someone states the problem clearly.

Once the problem is stated there are a lot of variations, and I'm not sure the research has anything approaching a consensus that either crowdsourcing or collaborative models is "better" (although again I rail against this false dichotomy).

If you look at the modern bug ticket system you'll see what I mean. Clear, succinct reports tend to attract patches/fixes, some of them with a flurry of collaboration, some in a flurry of competition, some in a methodical responsible assignee/lone coder/hero fashion. A few confused reports result in a flurry of brainstorming, trying to figure out what the problem is, then who's responsible for causing/fixing it. The majority of those hang out in limbo, not understood or addressed until someone cleans house (probably with a script closing everything older than XX with "abandoned, won'tfix").

I like your ideal problem approach, but I think you're missing two steps: 2.5) test solutions, and 4) Regularly planned refactors.

When I heard the terms, it crossed my mind that, Crowdsourcing appears to be throwing ONE problem or ONE Issue (that needs innovation) to a crowd of people, and each will come up with hundreds or thousands of ideas of solution or innovation

While Collaboration appears to be throwing MULTIPLE tasks to solve ONE problem to multiple individuals or multiple teams with different expertise. The way collaboration work by assigning task according to each expertise, then Combine them together as a formula of problem solving / innovation.

Within Collaboration, each team has ability to do crowdsourcing as well.

both have advantages: Time and Cost.

My question come up is: HOW and WHO selects the best Solution for Crowdsourcing? The Crowd (vote)? Boards of experts? or the
one throwing the problem /issue?

Thanks for this post. I found the definitions to be helpful. To your question about feedback to these approaches, I work at a creative ad agency, and we definitely experiment with both "crowdsourcing" and "collaboration" as you've defined them. I buy your thesis, Rebecca, that neither is fool proof, but there I think in combination the odds of arriving at solutions do increase.

Great collaboration indeed starts with an articulate description of the problem ("a problem well defined is a problem half solved," right?). For ad agency types, the counter-intuitive thing about concepting sharp solutions is that tighter boxes lead to better outcomes. Instead of ambling around trying to pin down what we're solving for, teams can focus on the best solutions to address that problem, whether that solution involves communications, applications, or some other experience.

On the flip side, like many companies in our field, we've sharply rejected the "old way" of conceiving of those solutions (art director and copy writer tuck away into a secret corner to concept a "big idea"). Now we bring together a team of developers, writers, visual artists, strategist, UX experts, etc. (that list is long, but smaller is better in many cases...we call it casting for the role). However, such work sessions are a total waste of time without some mediation. We start with a statement of the problem (a "creative brief" in our business), and then kick off group collaboration. It requires ruthless moderation, though, and is really focused on the "many solutions to one problem" angle that the last commenter described. This first step is about quantity not quality.

Quality follows, as an even smaller group (the team leads) curates the ideas, ditches the dregs and instructs "the cast" to continue to work independently on the directions that seem fruitful. Then more group collaboration, meandering down to a coherent short list of solutions, again ruthlessly edited by a small group.

It might sound like a mess, but it does work (mostly!). To my mind the combination of collaboration and crowdsourcing is an integral requirement today. No one person or even two-man team knows enough to get it all right.

Creative Commons LicenseThis work is licensed under a Creative Commons Attribution-Share Alike 3.0 Unported License.