The Siren's Call Of Complexity: How Legacy Businesses Get Led Astray

from the disruption-in-our-time dept

Fri, Apr 2nd 2010 9:55am — Mike Masnick

I've talked in the past about the importance of understanding the concept of the Innovator's Dilemma, as put forth by Clayton Christensen years ago. This is the idea that legacy companies often struggle with recognizing disruptive innovation, since when it first appears the offering appears to be not as good as the current offering in many ways. That is, a car might not have seemed as good as a horse-drawn carriage when automobiles were first invented, because the infrastructure wasn't in place, cars broke down more frequently, they were noisy, etc. But the problem is that those making the judgment often misjudge both the trends for improvement, as well as what the customer is really basing his or her buying decision on. Thus, as the quality of the innovative product improves, at some point -- even if the product is still seen as "worse" by the legacy business folks -- it becomes good enough, often at a much cheaper price for the buyers.

That's when the real disruption occurs. And it tends to happen fast, and be quite destructive for the legacy companies who simply can't understand why their business is disappearing, especially for a product that is objectively worse in their eyes.

It's been about a year since the last absolute must read writeup on modern business models by Clay Shirky, so it looks like he's decided to do it again -- this time taking on the "collapse of the complex business model." As I said, it's a must read piece that is really attacking the Innovator's Dilemma question from a different angle -- looking at how legacy businesses get so focused on complexity that they assume complexity must exist and is a part of any reasonable process or business model. And, as such, when they see something simple show up, they just assume it can't possibly work, because it doesn't even fit into their concept of a workable business model:

In the mid-90s, I got a call from some friends at ATT, asking me to help them research the nascent web-hosting business. They thought ATT's famous "five 9's" reliability (services that work 99.999% of the time) would be valuable, but they couldn't figure out how anyone could offer good web hosting for $20 a month, then the going rate. No matter how many eventual users they assumed, $20 didn't even seem to cover the monthly costs, much less leave a profit.

I started describing the web hosting I'd used, including the process of developing web sites locally, uploading them to the server, and then checking to see if anything had broken.

"But if you don't have a staging server, you'd be changing things on the live site!" They explained this to me in the tone you'd use to explain to a small child why you don't want to drink bleach. "Oh yeah, it was horrible", I said. "Sometimes the servers would crash, and we'd just have to re-boot and start from scratch." There was a long silence on the other end, the silence peculiar to conference calls when an entire group stops to think.

The ATT guys, part of a company so committed to the sacred dial tone it ran its own power grid, had correctly understood that the income from $20-a-month customers wouldn't pay for good web hosting. What they hadn't understood, were in fact professionally incapable of understanding, was that the industry solution, circa 1996, was to offer hosting that wasn't very good.

Now, to some extent, that describes the very traditional view of the Innovator's Dilemma, but Shirky actually then takes it a bit in a different (but important) direction to explain why this happens. And it's not just a belief in the idea that the legacy product is of higher quality, but because of the institutional complexity built into large businesses. As a parallel, he talks about research done by Joseph Tainter on the complexity of societies, but the parallel to business is clear:

Tainter's story goes like this: a group of people, though a combination of social organization and environmental luck, finds itself with a surplus of resources. Managing this surplus makes society more complex--agriculture rewards mathematical skill, granaries require new forms of construction, and so on.

Early on, the marginal value of this complexity is positive--each additional bit of complexity more than pays for itself in improved output--but over time, the law of diminishing returns reduces the marginal value, until it disappears completely. At this point, any additional complexity is pure cost.

Tainter's thesis is that when society's elite members add one layer of bureaucracy or demand one tribute too many, they end up extracting all the value from their environment it is possible to extract and then some.

The 'and them some' is what causes the trouble. Complex societies collapse because, when some stress comes, those societies have become too inflexible to respond. In retrospect, this can seem mystifying. Why didn't these societies just re-tool in less complex ways? The answer Tainter gives is the simplest one: When societies fail to respond to reduced circumstances through orderly downsizing, it isn't because they don't want to, it's because they can't.

In such systems, there is no way to make things a little bit simpler -- the whole edifice becomes a huge, interlocking system not readily amenable to change.

From this comes a whole series of missteps, and Shirky highlights a few -- including the idea being pushed by many old school journalism folks that people simply must start paying to access news online, and the belief that this is an incontrovertible fact. But it's not so at all. It may be a fact that people would need to pay to keep the old legacy structures and complexity in place -- but there's no requirement to do that in a world where simplicity can take over.

The same thing is true in the online video market -- the key market that Shirky was addressing in his writeup. Just recently, we had a discussion here about the online video market and its economics, kicked off by a claim by a Frost & Sullivan analyst that assumed that you simply could not offer video for free online and make money. There were a lot of factual errors in that piece -- and with reports now that Hulu is actually profitable, and other reports suggesting that YouTube is rapidly approaching profitability, if it's not there yet, it makes those claims even more amusing.

In the comments to that post, the analyst was joined by Mark Cuban in making arguments against online video, and the arguments kept going back to a basic premise: television shows as made today are really freaking expensive to produce. Nothing online could match that quality, and the infrastructure of online makes it too costly to make money. But there are a lot of built in assumptions there, many of which don't appear to be true. The biggest one is exactly the point that Shirky is making: that the existing complex structure, costly as it is, needs to remain in order to produce good content. But there's simply no reason that's true. Yes, the content may look different, and may not appear to be "as good" from the perspective of the legacy players who judge the content on a different standard. But from the user standpoint, the quality can be fantastic:

In the future, at least some methods of producing video for the web will become as complex, with as many details to attend to, as television has today, and people will doubtless make pots of money on those forms of production. It's tempting, at least for the people benefitting from the old complexity, to imagine that if things used to be complex, and they're going to be complex, then everything can just stay complex in the meantime. That's not how it works, however.

The most watched minute of video made in the last five years shows baby Charlie biting his brother's finger. (Twice!) That minute has been watched by more people than the viewership of American Idol, Dancing With The Stars, and the Superbowl combined. (174 million views and counting.)

Some video still has to be complex to be valuable, but the logic of the old media ecoystem, where video had to be complex simply to be video, is broken. Expensive bits of video made in complex ways now compete with cheap bits made in simple ways. "Charlie Bit My Finger" was made by amateurs, in one take, with a lousy camera. No professionals were involved in selecting or editing or distributing it. Not one dime changed hands anywhere between creator, host, and viewers. A world where that is the kind of thing that just happens from time to time is a world where complexity is neither an absolute requirement nor an automatic advantage.

Now, I can already hear the critics scoffing. Charlie biting his brother's finger is terrible video. And it is -- from any objective measure of the old way the business worked. But it's what's competing for attention these days. And it's winning. From a consumer's viewpoint, the video isn't terrible at all. It's fantastic. And not all of the videos that are getting that kind of attention are that "terrible." Some are really quite amazing -- in part because of the simplicity of production. And that will only grow over time.

But if you try to squeeze that simplicity into a complex system, you'll screw it up:

In spring of 2007, the web video series In the Motherhood, a humorous look at modern motherhood, made the move to TV. In the Motherhood started online as a series of 5 minute videos, with viewers contributing funny stories from their own lives and voting on their favorites. This tactic generated good ideas at low cost as well as endearing the show to its viewers; the show's tag line was "By Moms, For Moms, About Moms."

The move to TV was an affirmation of this technique; when ABC launched the public forum for the new TV version, they told users their input "might just become inspiration for a story by the writers."

Or it might not. Once the show moved to television, the Writers Guild of America got involved. They were OK with For and About Moms, but By Moms violated Guild rules. The producers tried to negotiate, to no avail, so the idea of audience engagement was canned (as was In the Motherhood itself some months later, after failing to engage viewers as the web version had).

The critical fact about this negotiation wasn't about the mothers, or their stories, or how those stories might be used. The critical fact was that the negotiation took place in the grid of the television industry, between entities incorporated around a 20th century business logic, and entirely within invented constraints.

And, of course, this applies in all sorts of industries that we see struggling every day. And it goes beyond the classical (though incredibly perceptive and important) concept put forth by Christensen with the Innovator's Dilemma. Shirky's look at complexity of interlocking systems within a business model as a fundamental hindrance to innovation and change is an incredibly powerful way of looking at things. As you look at many big businesses that have collapsed, you can see it as a core problem. When you look at the problems of Wall Street and the auto industry today -- it clearly applies (and it seems doubly unfortunate, then, that our policy decisions were to simply prop up and re-enable such complexity, rather than moving on to more simplicity).

It also makes it clear why the complaint from many that big companies will simply "copy" the ideas of truly disruptive companies is often quite overblown. In many cases, they structurally are unable to do so. As has been said before, if your idea is truly innovative, you'll almost certainly be ignored. The truly innovative ideas don't look innovative at all to big companies, because they don't even fit into the existing structure. They collapse it. And when you're standing in the middle of such a structure, the idea of collapsing it on top of yourself isn't even in the set of possible moves.

Thank you for reading this Techdirt post. With so many things competing for everyone’s attention these days, we really appreciate you giving us your time. We work hard every day to put quality content out there for our community.

Techdirt is one of the few remaining truly independent media outlets. We do not have a giant corporation behind us, and we rely heavily on our community to support us, in an age when advertisers are increasingly uninterested in sponsoring small, independent sites — especially a site like ours that is unwilling to pull punches in its reporting and analysis.

While other websites have resorted to paywalls, registration requirements, and increasingly annoying/intrusive advertising, we have always kept Techdirt open and available to anyone. But in order to continue doing so, we need your support. We offer a variety of ways for our readers to support us, from direct donations to special subscriptions and cool merchandise — and every little bit helps. Thank you.

–The Techdirt Team

Filed Under: clay shirky, innovator's dilemma

19 Comments

If you liked this post, you may also be interested in...

Reader Comments

Subscribe: RSS

View by: Time | Thread

Hephaestus (profile), 2 Apr 2010 @ 10:17am

Great read
This inability to adapt is the direction I see things heading. Where the self imposed rules apply to the movies and TV studios but not to the public at large. Where simple inexpensive software makes TV and movie production something that can be done by anyone. Video editting, CGI, sound editting, all have open source applications that anyone can download. This will lead to greater competition and reduced revenue for the Studios.
[ link to this | view in thread ]
Anonymous Coward, 2 Apr 2010 @ 10:52am

"looking at how legacy businesses get so focused on complexity that they assume complexity must exist and is a part of any reasonable process or business model."

Any engineer with half a brain would tell you that complexity is bad, simplicity is good. The hallmark of good design is simplicity, not complexity.
[ link to this | view in thread ]
Anonymous Coward, 2 Apr 2010 @ 10:58am

Ebay. A perfect example of a business that wandered so far from the center it needed a search party to find it and bring it home. Unfortunately, the off-the-psych-meds patient had already gone on a spending spree, buying Skype, bedding down with big-box retailers, kicking the old and loyal friends to the curb.

Amazon - Has done an excellent job of changing, adapting and snagging new markets without trampling on the old.
[ link to this | view in thread ]
Anonymous Coward, 2 Apr 2010 @ 11:03am

Re:
Are engineers usually decision-makers for big companies? Usually the people making decisions have already made up their minds and are just looking for validation. An engineer who speaks up will be facing office politics, not change in business practice.
[ link to this | view in thread ]
Bill Pickett, 2 Apr 2010 @ 11:05am

Open Source
This article makes me think of Open Source. Right now, Open Source isn't always the best solution but it is good enough. It also upsets the checks and balances of the old "complex" systems. It has done this by being better in really one key area: it is more scalable. Scalability in Open Source means that with peer review more bugs can be found faster and more features can be added on the fly. The complementary practice of "release early, release often" reinforces this. Closed source still has a lot of money to keep itself going but in the meantime, unless it is made illegal to share your efforts with your neighbor (the real basis of Copyleft): Open Source is reconstructing the playing field.
[ link to this | view in thread ]
Matthew Cruse (profile), 2 Apr 2010 @ 11:26am

Compare to NUMMI
This reminds me of one of the the big things that came out of NUMMI and that the auto industry really benefited from, at least initially. That was increased simplicity. Toyota had figured out, strip all of the excesses out of the system, make the cars as simply as possible, with fewer changes in tooling, or simpler changes, share as many components across models as possible. Make the cars "good enough", but not perfect. When the first toyotas hit the shores, they were crappy cars. But, they were cheap and ran and were at least minimally reliable-"good enough". And Big Auto COULD NOT see that that was a desirable quality, they just knew that they were being driven under.
[ link to this | view in thread ]
Steve R. (profile), 2 Apr 2010 @ 11:39am

Complexity Not the Fundamental Problem
Complexity is an issue with large corporations that have legacy products, but I don't think that the concept of "complexity" is the fundamental rationale for why they go astray. Corporations, like humans, evolve. When they start they are small, entrepreneurial, and aggressive. As they become older and established protecting the business becomes an ever greater concern. I will agree that established businesses become increasingly "complex", but that complexity simply adds to internal friction (its a lot harder to build census). While the company may be "complex", the mindset of the management will tend to be focused on maintaining (protecting) the business model.

It is the mindset of management that determines whether a company will adapt or be led astray.
[ link to this | view in thread ]
Hephaestus (profile), 2 Apr 2010 @ 11:44am

My favorite line form that article ....
"“Web users will have to pay for what they watch and use, or else we will have to stop making content in the costly and complex way we have grown accustomed to making it. And we don’t know how to do that.”"

With so many people having their hands in the till, there is no way for the studios to adapt in time the contracts they have in place will take to long to renegotiate if its at all possible. Bad for them, good for the rest of us.
[ link to this | view in thread ]
Hephaestus (profile), 2 Apr 2010 @ 11:53am

Re: Complexity Not the Fundamental Problem
"It is the mindset of management that determines whether a company will adapt or be led astray."

Its the legacy complexities that allow or dis-allow change. If you look at the auto industry they have pensions, unions contracts, supply contracts, etc. Those are part of the complexities that prevents change. The other part is mindset and bureaucracy which are the same thing in a corporation or government.
[ link to this | view in thread ]
Ryan, 2 Apr 2010 @ 11:56am

Re: Complexity Not the Fundamental Problem
I suppose that may be true in the sense that the more a legacy business chooses to adapt to a new marketplace, the more the existing infrastructure has to be written off as a sunk cost - and humans are inherently loss-averse. However, that also means that the cost of doing it the new way relative to doing it the old way is much higher for legacy businesses and so may not be as good a deal for them, especially when there is so much government interference that can be bought for bailouts and competitive advantages.

Additionally, a lot of these businesses are impeded by contractual obligations, as the post alluded to above with the Writers Guild. This is why unions are usually anathema to effective businesses.
[ link to this | view in thread ]
Modplan (profile), 2 Apr 2010 @ 12:03pm

Re: Complexity Not the Fundamental Problem
The Innovator's Dilemma actually specifically states that it's the nature of business for companies to effectively become immovable objects in the face of change and disruptive innovations.

Even if managers and the like spot the change, everything is naturally set up in a way to actively work against making that change from within the existing structure.

Companies only evolve so far as it allows them to do what they currently provide but better (sustaining innovation). As soon as something appears to break that with something that is good enough (disruptive innovation), the incentives work to ignore or flee.
[ link to this | view in thread ]
Modplan (profile), 2 Apr 2010 @ 12:21pm

Re: Open Source
This is a perfect example of Disruptive Innovation.

Look at it from the business perspective, like Microsoft to hep illustrate this.

Microsoft sells its software upfront, and creates it in a way where most of the cost of creating said software is on them. They look across to open source back in the early 90's. All they see are crappy customers - people who aren't as demanding, and consequently less willing to pay. Meanwhile the software that's being produced at the time isn't quite as marketable theirs, so they're not too worried. They're "hobbyists", or whatever other dismissal you can come up with. In other words, small fry, business and customers they don't are about.

Open source improves and moves upmarket. The software gets better, it invades businesses, but alas they're not as demanding and were subsequently not as jazzed about their previous deal with Microsoft. MS still doesn't care - it's still comparative small fry to where they could be heading upmarket.

And so on, until MS is backed into a high end, lucrative niche with nowhere else left to go, with open source business models having mostly taken over.

Of course the reality is still a bit different, due to other complications like their monopoly power. However, this does fit rather well with MS vs Apple, along with what's been happening in emerging markets with don't already have those barriers.
[ link to this | view in thread ]
Anonymous Coward, 2 Apr 2010 @ 12:47pm

Re: Re:
"Are engineers usually decision-makers for big companies?"

No, which is one of the problems with big companies. Engineers also don't make decisions in politics. I guess those who aren't smart enough to become engineers or doctors need jobs and since they're too dumb or lazy to become engineers they take other jobs like management and politics. Then again, I'm not an engineer either, so I guess I'm criticizing myself as well :)
[ link to this | view in thread ]
Mcbeese, 2 Apr 2010 @ 1:32pm

Re:
Actually, I find that engineers fall on the side of complexity, not simplicity. Have you ever witnessed a PowerPoint deck created by engineers?

It is artists, talented writers, and mathematicians who are the masters of simplicity, IMHO.
[ link to this | view in thread ]
Matthew Cruse (profile), 2 Apr 2010 @ 1:45pm

Re: Re: Complexity Not the Fundamental Problem
I think you hit the nail on the head "the cost of doing it the new way relative to doing it the old way is much higher for legacy businesses ". And that is the real killer forn legacy business. When your young and small, 6-8 people in a dorm room or garage working on the next big thing, and you hit a snag or a dead end whether it is with a design, a product, or a business method, you chuck all the previous work and go on to something else. I.e. innovat, adapt change rapidly. When you are GM ot Toyota or HP or MS and you have thousands of employees and millions of dollars invested in a product line, and it hits a snag, it is unworkable (in the sense of cost and bottom line) to throw it all away and start over. Instead you do your best to protect what you have, because to do almost anything else is suicide. As was styated in the post, most major companies are cvompletely unable to adapt and change fast enough to be "good enough" fast enough. There are 8 or 10 or 20 layers between concept and decision. If you are start up and have a staff of 10, problems are recognized and decisions are made early. If you are legacy and staff of thousands, problems are recognized on Monday and decisions are made next December.
[ link to this | view in thread ]
This Just In, 2 Apr 2010 @ 3:11pm

Re: Re:
There is a wide distribution of personality types within most all fields of study. And as usual, the use of stereotypes leads to incorrect statements.

Film at eleven
[ link to this | view in thread ]
abc gum, 2 Apr 2010 @ 3:21pm

Re: Re: Open Source
"crappy customers"

I suppose M$ would view these people in such light because they want interoperability without lockin and other such nonsense.

"aren't as demanding"

Ok, this could mean that these people do not demand that users of their product pay over and over for a product that improves little while consuming more resources.

"invades businesses"

This is funny.
[ link to this | view in thread ]
John Fenderson (profile), 2 Apr 2010 @ 4:10pm

"Charlie bit my finger" is a great example.

To most people who possess a funny bone, it's a fantastic video in the sense that it makes you laugh at least as hard as professionally produced (read: expensive) comedy. That makes it high quality in all the ways that actually matter to viewers. There's no way the old guard can compete with that -- the best they can do turns out like "America's Funniest Videos," which too often take genuinely funny content and reduce its quality by adding unneeded production values such as sound effects, stupid introductions and comments from a stupid host, live audiences, etc.

CBMF shows how to do it right -- just the funny, without the padding.
[ link to this | view in thread ]
Hephaestus (profile), 2 Apr 2010 @ 4:21pm

Re: Re: Re: Complexity Not the Fundamental Problem
"When you are GM ot Toyota or HP or MS and you have thousands of employees and millions of dollars invested in a product line, and it hits a snag, it is unworkable (in the sense of cost and bottom line) to throw it all away and start over."

Actually if you are GM, MS, or any other large cap you have the financing available to create a subsidiary to do what the legacy company can not. Be lean be fast and get things done. Executives tend to not do this because it is outside the box thinking. What is done instead is study groups are created and kept in check with the vision, or lack of, that mirrors the main corporation.

Last year a friend showed me a factory design in MS robotics studio to automate the manufacture cars. Very impressive even though it was only a half done design. CNC tube benders, reconfigurable automated jigs for welding frames, robotic welders, reconfigurable sheet metal presses, all designed work with out human intervention in the simplest way possible. It gave me engineering wet dreams for over a week. I asked one simple question, "Can you also build a factory to build these car factories?". My friend looked at me and said "What a stupid idea. Why would you want to do that?" Then I saw the light go on and he said "Holy Shit!".

The point is one that I have made before, technology is getting cheaper and the capabilities are getting greater. This will close the gap between large corporations and small work groups in a very quick and very disruptive way over the next 20-30 years. The entrenced monopolies and corporate bureaucracies with their inability to adapt will begin failing as these technologies come of age. That is after they run to the government and seek protection in the way of laws and trade agreements ...

JMHO

David
[ link to this | view in thread ]