Techdirt Reading List: The Disruption Dilemma

from the disrupting-disruption-theory dept

We're back again with another in our weekly reading list posts of books we think our community will find interesting and thought provoking. Once again, buying the book via the Amazon links in this story also helps support Techdirt.

As regular readers likely know, I'm a big fan of Clayton Christensen's famous book The Innovator's Dilemma and the concept of disruptive innovation put forth in that book (more accurately, I became a fan prior to the book coming out, when the Harvard Business Review article discussing the innovator's dilemma came out prior to the book). Many years back, I even tried to explain Christensen's concept in just two minutes at a whiteboard.

Of course, over the years, the popularity of "The Innovator's Dilemma" and the phrase "disruptive innovation" has meant that many people like to use the phrase without having any understanding of what it really means. Many people seem to think that "disruptive innovation" just means "breaking shit" or "ignoring all the rules." That's not what it actually means. Earlier this year, Joshua Gans released a book called The Disruption Dilemma that is both something of a critique / update on Christensen's work, but also providing some useful ways of looking at and thinking about innovation. If you're a fan of disruptive innovation and want to better understand it, this is a worthwhile read.

Filed Under: disruption dilemma, disruptive innovation, innovator's dilemma, joshua gans, reading list, techdirt reading list

Disrupting The Misinterpretation Of Disruptive Innovation

from the the-innovator's-dilemma-dilemma dept

There's a lot of buzz and talk in various communities lately about a new New Yorker story by Jill Lepore, in which she seeks to dismantle Clayton Christensen's famed concept of "the Innovator's Dilemma" and its corresponding concept of disruptive innovation. It's no secret that I'm a big fan of the concepts put forth by Christensen, and five years ago, as part of a sponsored set of videos, I did a quick two-minute white board video explaining the concept. The concept of the Innovator's Dilemma and disruptive innovation became so big so fast that it has, reasonably, come under some attack of late. Part of this is just the inevitable backlash that comes with a popular idea. An even bigger part of it is that many have jumped onto the bandwagon without really understanding it. And, many of those are selling snake oil: the idea that companies can actually "beat" the innovator's dilemma. However, as we noted last year, the very reason why disruptive innovation is so disruptive is because the incumbents don't see it as innovation. Part of the problem with the concept of disruptive innovation is the idea that, when graphed out simply, it always feels like the incumbent can see what's coming. They just have to recognize the "cheaper, not quite as good" offering, and realize that it can disrupt their business.

But that's wrong on multiple accounts. First, truly disruptive innovation usually is barely even on the radar of the incumbent players. It's not that they don't think it's disruptive, it's that they don't even see it as innovation or being in the same realm. Craigslist disrupted the newspaper business, and many in the newspaper business still don't realize that. Disruptive innovation doesn't always come neatly "up from the bottom" as the graphs show. They often come in orthogonally from a different dimension entirely.

Second, industries that are disrupted often face a whole bunch of potentially disruptive competitors -- and (in part because of the point above), it's very difficult to tell which one(s) will actually be disruptive. Economist Joshua Gans has a fantastic explanation of this, noting that disruptive innovation has to fulfill two criteria: perform "worse" than incumbent technology on certain metrics and also be on a fast path for improvement (though I don't agree with Gans' claim that the improvement has to be on those same metrics -- in many cases, I'd argue it's actually on the metrics the customers truly value, rather than the ones they claim to value, which are not always the same). The issue is that it's easy to pinpoint technologies that meet the first criteria, but nearly impossible to spot the ones that meet the second criteria. Thus, you end up with lots of potentially disruptive technologies, but actually spotting the actually disruptive technologies isn't so easy.

That's why Lepore's complaint about companies that tried to disrupt themselves and failed goes wrong.

Time, Inc., founded in 1922, auto-disrupted, too. In 1994, the company launched Pathfinder, an early new-media venture, an umbrella Web site for its magazines, at a cost estimated to have exceeded a hundred million dollars; the site was abandoned in 1999. Had Pathfinder been successful, it would have been greeted, retrospectively, as evidence of disruptive innovation. Instead, as one of its producers put it, “it’s like it never existed.”

Lepore isn't really attacking the nature of "the innovator's dilemma" or "disruptive innovation" at all. She's really -- rightfully -- attacking the idea that incumbents can successfully recognize and fight it off. In the history of technology innovation, the list of companies that have truly recognized and responded to disruptive innovation is a fairly short list. I can think of Intel flipping from making memory to making microprocessors and possibly IBM flipping from focusing mainly on big hardware to services (though, even that one's a bit fuzzy). There may be a few other stories here and there, but for the most part, big companies miss disruptive innovations by a long shot.

Beyond the reasons listed above, the other big reason why incumbents fail to react properly to disruptive innovation is that they overestimate their own ability to "catch up later," while missing the key reasons why a disruptive innovation is so successful. For years, for example, I've heard execs in the entertainment industry insist that when the internet was really important, then they could really start investing in an internet strategy. But, of course, by then, it's often too late, and they're playing catchup. Not only that, but they're playing catchup in a realm that they don't fully understand, often mimicking the superficial innovations they see, while completely missing the true innovations under the surface.

To be clear, Christensen may be a reasonable target here. As Tim Lee rightfully notes, Christensen himself has built up a pretty big and profitable consulting business trying to "help" big companies successfully navigate disruption. And there are hundreds to thousands of consultants out there pretending they can do the same thing. And, in some cases, those folks may have the right idea, in general. Hell, in the video above, I, too, am guilty of suggesting that companies can sometimes out-innovate disruptors. It's very tempting to believe that understanding this theory is useful in successfully avoiding the disruption wave. But, the combination of it being nearly impossible to figure out which disruptive innovation is really disruptive, combined with general corporate inertia, makes it nearly impossible to predict precisely which disruptive innovation matters.

But that doesn't (and shouldn't) take away from the importance of the underlying concept and trajectory of disruptive innovation as a whole. The fact that you can't precisely predict which innovations will be disruptive doesn't mean that there aren't any disruptive innovations, or that the theory isn't useful in understanding what's happening in the market. It's just not great as a predictor of "this company will beat that company" -- which is where Christensen himself often runs into problems.

But, in terms of understanding wider trends, where incumbent companies are likely to go wrong, and even in pinpointing opportunities for disruption, the concepts behind the Innovator's Dilemma are of massive importance. The lack of exact predictive power of "this technology will disrupt that incumbent in this way" doesn't rid the entire concept of value. Understanding disruptive innovation is a very powerful and very important tool in recognizing and understanding innovation trends. That doesn't mean it's useful in saying Startup A will definitely disrupt Incumbent B. More importantly, it doesn't mean Incumbent C will be able to recognize or prevent disruption to itself. But if that's what you're trying to do with it, you're using the tool wrong.

Bonus reading: Check out disruptor Will Oremus applying the theory of disruptive innovation to dismantling Lepore's article directly.

Filed Under: clayton christensen, disruption, disruptive innovation, innovation, innovator's dilemma, jill lepore, predictive value

Institutions Will Seek To Preserve The Problem For Which They Are The Solution

from the the-shirky-principle dept

We already wrote a detailed analysis of Clay Shirky's recent writeup on complex business models. However, a few of you have sent over Kevin Kelly's recent post about Shirky's piece that also compares it to Clayton Christensen's Innovator's Dilemma, but thankfully highlights the one key line in Shirky's piece that may have gotten lost in the original:

"Institutions will try to preserve the problem to which they are the solution."

Kelly calls this the "Shirky Principle." To me, it calls to mind Upton Sinclair's famous line:

"It is difficult to get a man to understand something, when his salary depends upon his not understanding it!"

They are not the same point, but they are related. In both cases, these are situations where people will often seek to preserve a problem or a falsehood, rather than recognize that it doesn't need to be that way. There are lots of industries where this is a major issue.

But, of course, the real problem is in how they go about trying to preserve that problem. They will go to great lengths to demonize the solutions. This is why the newspaper industry has, at times, lashed out at Craigslist and Google News -- two operations that have essentially removed problems that the newspaper business used to solve. It's why old school video guys lash out at YouTube or Boxee -- because they have removed problems that television used to solve. And, yes, it's why the RIAA and the MPAA lash out at file sharing apps and services -- because they have removed problems in distribution and promotion, that they used to solve.

Filed Under: business models, innovator's dilemma, institutions, problems, solutions

The Siren's Call Of Complexity: How Legacy Businesses Get Led Astray

from the disruption-in-our-time dept

I've talked in the past about the importance of understanding the concept of the Innovator's Dilemma, as put forth by Clayton Christensen years ago. This is the idea that legacy companies often struggle with recognizing disruptive innovation, since when it first appears the offering appears to be not as good as the current offering in many ways. That is, a car might not have seemed as good as a horse-drawn carriage when automobiles were first invented, because the infrastructure wasn't in place, cars broke down more frequently, they were noisy, etc. But the problem is that those making the judgment often misjudge both the trends for improvement, as well as what the customer is really basing his or her buying decision on. Thus, as the quality of the innovative product improves, at some point -- even if the product is still seen as "worse" by the legacy business folks -- it becomes good enough, often at a much cheaper price for the buyers.

That's when the real disruption occurs. And it tends to happen fast, and be quite destructive for the legacy companies who simply can't understand why their business is disappearing, especially for a product that is objectively worse in their eyes.

It's been about a year since the last absolute must read writeup on modern business models by Clay Shirky, so it looks like he's decided to do it again -- this time taking on the "collapse of the complex business model." As I said, it's a must read piece that is really attacking the Innovator's Dilemma question from a different angle -- looking at how legacy businesses get so focused on complexity that they assume complexity must exist and is a part of any reasonable process or business model. And, as such, when they see something simple show up, they just assume it can't possibly work, because it doesn't even fit into their concept of a workable business model:

In the mid-90s, I got a call from some friends at ATT, asking me to help them research the nascent web-hosting business. They thought ATT's famous "five 9's" reliability (services that work 99.999% of the time) would be valuable, but they couldn't figure out how anyone could offer good web hosting for $20 a month, then the going rate. No matter how many eventual users they assumed, $20 didn't even seem to cover the monthly costs, much less leave a profit.

I started describing the web hosting I'd used, including the process of developing web sites locally, uploading them to the server, and then checking to see if anything had broken.

"But if you don't have a staging server, you'd be changing things on the live site!" They explained this to me in the tone you'd use to explain to a small child why you don't want to drink bleach. "Oh yeah, it was horrible", I said. "Sometimes the servers would crash, and we'd just have to re-boot and start from scratch." There was a long silence on the other end, the silence peculiar to conference calls when an entire group stops to think.

The ATT guys, part of a company so committed to the sacred dial tone it ran its own power grid, had correctly understood that the income from $20-a-month customers wouldn't pay for good web hosting. What they hadn't understood, were in fact professionally incapable of understanding, was that the industry solution, circa 1996, was to offer hosting that wasn't very good.

Now, to some extent, that describes the very traditional view of the Innovator's Dilemma, but Shirky actually then takes it a bit in a different (but important) direction to explain why this happens. And it's not just a belief in the idea that the legacy product is of higher quality, but because of the institutional complexity built into large businesses. As a parallel, he talks about research done by Joseph Tainter on the complexity of societies, but the parallel to business is clear:

Tainter's story goes like this: a group of people, though a combination of social organization and environmental luck, finds itself with a surplus of resources. Managing this surplus makes society more complex--agriculture rewards mathematical skill, granaries require new forms of construction, and so on.

Early on, the marginal value of this complexity is positive--each additional bit of complexity more than pays for itself in improved output--but over time, the law of diminishing returns reduces the marginal value, until it disappears completely. At this point, any additional complexity is pure cost.

Tainter's thesis is that when society's elite members add one layer of bureaucracy or demand one tribute too many, they end up extracting all the value from their environment it is possible to extract and then some.

The 'and them some' is what causes the trouble. Complex societies collapse because, when some stress comes, those societies have become too inflexible to respond. In retrospect, this can seem mystifying. Why didn't these societies just re-tool in less complex ways? The answer Tainter gives is the simplest one: When societies fail to respond to reduced circumstances through orderly downsizing, it isn't because they don't want to, it's because they can't.

In such systems, there is no way to make things a little bit simpler -- the whole edifice becomes a huge, interlocking system not readily amenable to change.

From this comes a whole series of missteps, and Shirky highlights a few -- including the idea being pushed by many old school journalism folks that people simply must start paying to access news online, and the belief that this is an incontrovertible fact. But it's not so at all. It may be a fact that people would need to pay to keep the old legacy structures and complexity in place -- but there's no requirement to do that in a world where simplicity can take over.

The same thing is true in the online video market -- the key market that Shirky was addressing in his writeup. Just recently, we had a discussion here about the online video market and its economics, kicked off by a claim by a Frost & Sullivan analyst that assumed that you simply could not offer video for free online and make money. There were a lot of factual errors in that piece -- and with reports now that Hulu is actually profitable, and other reports suggesting that YouTube is rapidly approaching profitability, if it's not there yet, it makes those claims even more amusing.

In the comments to that post, the analyst was joined by Mark Cuban in making arguments against online video, and the arguments kept going back to a basic premise: television shows as made today are really freaking expensive to produce. Nothing online could match that quality, and the infrastructure of online makes it too costly to make money. But there are a lot of built in assumptions there, many of which don't appear to be true. The biggest one is exactly the point that Shirky is making: that the existing complex structure, costly as it is, needs to remain in order to produce good content. But there's simply no reason that's true. Yes, the content may look different, and may not appear to be "as good" from the perspective of the legacy players who judge the content on a different standard. But from the user standpoint, the quality can be fantastic:

In the future, at least some methods of producing video for the web will become as complex, with as many details to attend to, as television has today, and people will doubtless make pots of money on those forms of production. It's tempting, at least for the people benefitting from the old complexity, to imagine that if things used to be complex, and they're going to be complex, then everything can just stay complex in the meantime. That's not how it works, however.

The most watched minute of video made in the last five years shows baby Charlie biting his brother's finger. (Twice!) That minute has been watched by more people than the viewership of American Idol, Dancing With The Stars, and the Superbowl combined. (174 million views and counting.)

Some video still has to be complex to be valuable, but the logic of the old media ecoystem, where video had to be complex simply to be video, is broken. Expensive bits of video made in complex ways now compete with cheap bits made in simple ways. "Charlie Bit My Finger" was made by amateurs, in one take, with a lousy camera. No professionals were involved in selecting or editing or distributing it. Not one dime changed hands anywhere between creator, host, and viewers. A world where that is the kind of thing that just happens from time to time is a world where complexity is neither an absolute requirement nor an automatic advantage.

Now, I can already hear the critics scoffing. Charlie biting his brother's finger is terrible video. And it is -- from any objective measure of the old way the business worked. But it's what's competing for attention these days. And it's winning. From a consumer's viewpoint, the video isn't terrible at all. It's fantastic. And not all of the videos that are getting that kind of attention are that "terrible." Some are really quite amazing -- in part because of the simplicity of production. And that will only grow over time.

But if you try to squeeze that simplicity into a complex system, you'll screw it up:

In spring of 2007, the web video series In the Motherhood, a humorous look at modern motherhood, made the move to TV. In the Motherhood started online as a series of 5 minute videos, with viewers contributing funny stories from their own lives and voting on their favorites. This tactic generated good ideas at low cost as well as endearing the show to its viewers; the show's tag line was "By Moms, For Moms, About Moms."

The move to TV was an affirmation of this technique; when ABC launched the public forum for the new TV version, they told users their input "might just become inspiration for a story by the writers."

Or it might not. Once the show moved to television, the Writers Guild of America got involved. They were OK with For and About Moms, but By Moms violated Guild rules. The producers tried to negotiate, to no avail, so the idea of audience engagement was canned (as was In the Motherhood itself some months later, after failing to engage viewers as the web version had).

The critical fact about this negotiation wasn't about the mothers, or their stories, or how those stories might be used. The critical fact was that the negotiation took place in the grid of the television industry, between entities incorporated around a 20th century business logic, and entirely within invented constraints.

And, of course, this applies in all sorts of industries that we see struggling every day. And it goes beyond the classical (though incredibly perceptive and important) concept put forth by Christensen with the Innovator's Dilemma. Shirky's look at complexity of interlocking systems within a business model as a fundamental hindrance to innovation and change is an incredibly powerful way of looking at things. As you look at many big businesses that have collapsed, you can see it as a core problem. When you look at the problems of Wall Street and the auto industry today -- it clearly applies (and it seems doubly unfortunate, then, that our policy decisions were to simply prop up and re-enable such complexity, rather than moving on to more simplicity).

It also makes it clear why the complaint from many that big companies will simply "copy" the ideas of truly disruptive companies is often quite overblown. In many cases, they structurally are unable to do so. As has been said before, if your idea is truly innovative, you'll almost certainly be ignored. The truly innovative ideas don't look innovative at all to big companies, because they don't even fit into the existing structure. They collapse it. And when you're standing in the middle of such a structure, the idea of collapsing it on top of yourself isn't even in the set of possible moves.

Filed Under: clay shirky, innovator's dilemma

Explaining The Innovator's Dilemma... In Two Minutes With A Whiteboard

from the out-innovate-yourself dept

As you may recall from last month, UPS recently asked us to create a series of videos, where we explain some of the stuff we talk about here on Techdirt on a regular basis in under two minutes, using a whiteboard. The first video was about the economics of abundance and got a great response. The second video is now up, and it's an attempt to explain the Innovator's Dilemma, based on Clayton Christensen's must-read research. If you're unfamiliar with it, it explains how difficult it is for many companies to adapt to changing markets, and is a good framework for understanding both why some companies are so slow to adapt. More importantly, it provides a good system for thinking about your own company and understanding how to adapt and implement new ideas rapidly: Again, with only two minutes, I had to simplify things down a bit, but hopefully it will kick off another good discussion on the innovator's dilemma and how to deal with it. We still have one more video to go, which I believe will be posted sometime next month. And, yes, once again (though, it should be obvious), UPS sponsored these videos, though we had free rein in creating the scripts -- which should be quite obvious as the topic is one we talk about here frequently enough.

Filed Under: clayton christensen, innovation, innovator's dilemma