The innovation delusion

Innovation is a messy process. Our stories about it aren't. We shouldn't confuse the two.
154 readers like this.
gears and lightbulb to represent innovation

If traditional planning is dead, then why do so many organizations still invest in planning techniques optimized for the Industrial Revolution?

One reason might be that we trick ourselves into thinking innovation is the kind of thing we can accomplish with a structured, linear process. When we do this, I think we're confusing our stories about innovation with the process of innovation itself—and the two are very different.

The process of innovation is chaotic and unpredictable. It doesn't operate according to clean, regimented timelines. It's filled with iterative phases, sudden changes in direction, various starts and stops, dead ends, (hopefully productive) failures, and unknowable variables. It's messy.

But the stories we tell ourselves about innovation, including the books and articles we read about great inventions and the tales we tell each other about our successes in the workplace, tidy that process up. Think about how many social media posts you've seen that feature nothing but the "high points."

That's the nature of good storytelling. It takes a naturally scattered collection of moments and puts them neatly into a beginning, middle, and end. It smoothes out all the rough patches and makes a result seem inevitable from the start, despite whatever moments of uncertainty, panic, even despair we experienced along the way.

We shouldn't confuse messy process with simplified story. When we do, we might mistakenly assume we can approach innovation challenges with the same practices we bring to neat and linear processes. In other words, we apply a set of management techniques appropriate for one set of activities (for more rote, mechanical, and prescriptive tasks) to a set of activities they aren't really suited for (more creative, non-linear work requiring autonomy and experimentation).

If traditional planning is dead, then why do so many organizations still invest in planning techniques optimized for the Industrial Revolution?

An innovation story

Here's one of my favorite examples of how this idea in action.

In the 1970s, the British motorcycle industry was desperately trying to figure out why its U.S. market share was plummeting while Honda's was skyrocketing. The company hired my former employer, the Boston Consulting Group, to help them figure out what was going wrong. BCG gathered some historical data, reviewed a two-decade sequence of events, and developed a neat, linear story explaining Honda's success.

Honda, BCG concluded, had executed an ingenious strategy: enter the U.S market with smaller motorcycles it could sell at lower cost, use the economies of scale it had developed in the Japense market to set low prices and grow a market, then further leverage those economies of scale to grow their share in the States as demand grew. By all accounts, Honda had done it brilliantly, playing to its strengths while thoroughly and accurately assessing the new, target U.S. consumer. It had outsmarted, outflanked, and outperformed competitors with a well-executed plan.

It sounded great. But the reality was much less straightforward.

Yes, Honda did want to enter the U.S. motorcycle market. It initially attempted to copy its competitors there, building the larger bikes Americans seemed to favor. But bikes like that weren't one of Honda's strengths, and their versions had reliability issues. To make matters worse, their models didn't look much different than other offerings already in the market, so they weren't standing out. Suffice it to say, sales were not booming.

But in a happy coincidence, Honda's Japanese representatives visiting the States had brought their own motorcycles with them. Those bikes were different than the ones the company was attempting to sell to the American market. They were smaller, zippier, less bulky, more efficient, and generally less expensive. Sears took notice, contacted the reps, and the companies struck a deal that let Sears carry this new motorcycle—called the "Super Cub"—in its American stores.

And the rest, as they say, is history. The Super Cub would go on to become the best-selling motorized vehicle of all time, and Honda continues to produce it today.

In hindsight, the events that brought the Super Cub to the U.S. seem logical, almost boring. But Honda owed its success less to an ingenious master plan and much more to serendipity and happenstance than most people care to admit.

When success depends on things we don't or can't predict, is getting exactly what you've planned for good enough?

Open (and messy) innovation

Organizations (and especially leaders) like to think that success is always planned—that they've become masters of chaos and can almost predict the future. But they're often making those assessments with the benefit of hindsight, telling the stories of their haphazard journey in a way that organizes the chaos, essentially reflecting on a period of uncertainty and saying "we meant to do that."

But as I said, we shouldn't assume those stories are mirror reflections of the innovation process itself and build future initiatives or experiments on that mistaken assumption.

Imagine another motorcycle manufacturer looking to replicate Honda's success with the Super Cub by following BCG's narrative to the letter. Because the story of Honda's success seems so logical and linear, the new company might assume it could use similar processes and get the same results: plan objectives, prescribe behaviors, and execute against knowable outcomes. But we know that Honda didn't really win its market with that kind of "plan, prescribe, execute" mentality. It won through flexibility and a bit of blind luck—something more like "try, learn, modify."

When we're able to appreciate and accept that the innovation process is messy, we allow ourselves to think differently about approaching innovation in our organizations. We can begin building the kinds of open and agile organizations capable of responding to innovation as it happens instead of over-investing resources into pre-formed plans that try to force innovation into a linear timeline.

I saw this kind of approach several years ago, when Red Hat released a new version of a product that included a major technology update. Version 5.4 of Red Hat Enterprise Linux was the first to include full support for a technology called the Kernel-based Virtual Machine (or "KVM"). For us it was a significant innovation that promised to deliver immense value not only to customers and partners, but also to open source software communities.

The technology was evolving quickly. Luckily, because we're an open organization, we were adaptable enough to respond to that innovation as it was happening and help our customers and partners take advantage of it. It was too important, and the competitive landscape too volatile, to justify withholding just so we could "save" it for a milestone moment like version 6.0.

When you go back and review the archived release notes for Red Hat Enterprise Linux, you'll see that it doesn't "read" like a typical software innovation tale. A game-changing development pops up at an unpredicted and unremarkable moment (version 5.4), rather than a pre-planned blockbuster milestone (version 6.0). In hindsight, we now know that KVM was the kind of "big bang" advancement that could have warranted a milestone release name like "6.0." But that's just not how the innovation process unfolded.

Don't get me wrong, organizations still need to maintain operational excellence and perform execution-oriented tasks well. But different kinds of challenges require different kinds of approaches, and we need to get better at building flexible organizations just as capable of responding to the unforeseen or unknowable.

An organization great at planning (and executing against that plan) will quite likely get the results it planned for. But when success depends on things we don't or can't predict, is getting exactly what you've planned for good enough?

Read this next

What to read next
User profile image.
Jim Whitehurst is President and Chief Executive Officer of Red Hat, the world’s leading provider of open source enterprise IT products and services. With a background in business development, finance, and global operations, Whitehurst has proven expertise in helping companies flourish—even in the most challenging economic and business environments.


Part of the problem is that at some point the hype machine made a regular meme campaign of using the words innovation and entrepreneurship interchangeably. I can only assume that the goal of this aggressive framing is to create a social consensus that unmonetized innovation doesn't count. Maybe it's Team Capitalism's response to those of us in the Tesla-was-more-of-an-innovator-than-Edison camp. Because entrepreneurship is a very competitive winner-take-all arena of combat, and because history is written by the winners, all our stories about innovation are in hindsight, about the one in ten startups that don't fail, and the structure and format of these stories has become very predictable and cliche, to the point that they're telling the same story over and over. Basically the stock news story that has come to be called "perseverance porn," like inspiration porn, except about entrepreneurship/innovation (again framed as being two words for the same thing) instead of disability. One can make a regular drinking game around instances of perseverance porn appearing in a news broadcast and the enterprise in question turns out to be cupcakes.

Jim, you said, "We can begin building the kinds of open and agile organizations capable of responding to innovation as it happens instead of over-investing resources into pre-formed plans that try to force innovation into a linear timeline."

Here's the challenge, for many organizations that attempt to change, they discover that it's difficult to 'unlearn' the things that we've done in the past that led to our prior success. Most people tend to cling to the past (the known) and are uncomfortable when they're asked to consider a somewhat uncertain future state (the unknown).

Let's look at the current disruptive transition that has occurred within the IT infrastructure market. In the past, the CIO and the CTO were assumed to be the only key decision-makers during the buyer's journey. The shift to the public cloud model -- and SaaS offerings in particular -- changed the procurement scenario as 'buyer committees' led by line-of-business leaders became the new normal.

Yet, most traditional IT vendors who acknowledge the transition struggle to evolve their marketing and sales approach. Why is that? Re-inventing your go-to-market approach is like any other form of innovation, it takes courage to let go of the old way and embrace the new way.

That said, savvy IT vendors that transform their narrative storytelling and position themselves further up the value chain will reap the rewards. They'll abandon their prior comfort-zone of only talking with the lower-level technical influencers and IT operations managers where they had existing relationships.

Now they'll reach out and engage 'anyone' who has a decisive point-of-view about Digital Transformation within their customer organizations. If that's primarily the numerous informed executives in the C-suite, then so be it. They're armed with new and differentiated stories that translate core IT product and service offerings into strategic business outcome benefits for those key decision makers.

Jim, I agree with you that while stories about innovation may be jazzy, the process of innovation itself may not be. Why should it be - if we want to encourage innovation? After all, man, by nature, is inspired by success and the outcomes. How many people do we come by who indulge in chaos? I'm not challenging the truth of the statement, but I'm more concerned in understanding what inspires a culture of innovation.

Two, you closed the article with a thought provoking question. Innovation is often viewed with the rear view mirror. It takes courage and conviction to reflect which of the outcomes were on account of plan, or the execution thereof, and which was luck. To my mind, that's mark of leadership

Creative Commons LicenseThis work is licensed under a Creative Commons Attribution-Share Alike 4.0 International License.