The Disruption Debate - What's Missing?

I admit that I’m mystified by Jill Lepore’s article in the New Yorker attacking Clayton Christensen and his theory of disruptive innovation. Not only does it have a meanness that isn’t warranted, but it leaves the reader with an unanswered question: if Clay's theories are not helpful (and I still believe they are), how do we explain the cascading disruptions that are playing out in markets and industries around the world?

Some of the commentators may be right: perhaps this article is better read as the most recent manifestation of the cultural conflict between the humanities-driven East and the technology-driven West.  The irony here is that the conflict is specifically between two professors at Harvard University – one based on the east coast of the Charles River and the other based on the west coast of the Charles River.

Clay Christensen has offered a rebuttal of sorts in an interview here. He does a good job of responding to some of the specific factual issues in his case studies highlighted by Jill Lepore and emphasizes that he continues to evolve his theory of disruptive innovation based on new research. But perhaps his best question is why she didn’t simply call him up or walk down the street to visit him before writing the attack to see if he might be able to clarify his position and his fact base.

I want to step back and use this controversy to underscore some key points that have been largely ignored in the recent discussion.

The systemic rise of disruption

First, while Jill Lepore might have issues with Clay’s specific theory and some of his case studies, she seems unwilling to acknowledge one basic fact of life: disruption is occurring with increasing frequency in the business world. Whether it is good or bad, it is happening and becoming increasingly widespread.

For the purpose of this posting, let me define disruption simply as the sudden demise of leaders or incumbents in particular markets or arenas. This demise is typically brought about by one or more players adopting a different approach to a market or arena that represents a significant challenge to the established position of existing participants. Disruptions turn the assets of incumbents into potentially life-threatening liabilities.

We all know the iconic cases of disruption casualties, including such large and well-known leaders as Kodak, Borders, Digital Equipment Corporation. But those are just the tip of the iceberg. There’s mounting evidence that disruption is spreading.

One of the issues with a case study approach is that it obscures the more fundamental and systemic trends and patterns that are playing out around us. When we pull back from individual stories and scan the world around us, it becomes clear that something very profound is happening - and it's largely escaped notice.

One of the metrics in our Shift Index looks at what economists call topple rate – the rate at which leaders fall out of their leadership position. In this case, we focused on the rate at which public US companies in the top quartile of return on assets performance fall out of this leadership position. Between 1965 and 2012, the topple rate increased by 40%.

OK, but the skeptic might reply that this is only about financial performance. Another more significant measure of fall from leadership position is provided by my old colleague and mentor, Dick Foster, who looked at the average lifespan of companies on the S&P 500.  In 1937, at the height of the Great Depression and certainly a time of great turmoil, a company on the S&P 500 had an average lifespan of 75 years.  By 2011, that lifespan had dropped to 18 years – a decline in lifespan of almost 75%.  At the same time that humans are significantly increasing their lifespan, large companies have been heading rapidly in the opposite direction.

So, the bottom line is that leaders are toppling with much greater frequency. Sure, some of that toppling may be the result of incompetence or mistakes made by senior management but, to explain away these trends, one would have to believe that management is becoming significantly more incompetent over time.

It's also clear that some of the toppling is the result of the natural process of market competition. As Joseph Schumpeter famously observed, markets are a powerful engine for "creative destruction" - they invite competitors with a better idea or a better approach to come in and challenge incumbents. It happens all the time. But the key question on the table from this more systemic view of disruption is: why is it increasing so dramatically over a long period of decades?

This is where I found Jill Lepore’s essay most wanting.  It’s easy to criticize an existing theory, but even assuming that this one doesn’t work (which I don’t), what’s the alternative? What’s going on? Jill Lepore certainly doesn’t venture an alternative theory and she clearly believes that the phenomenon of disruption is far too over-hyped. Without rushing to the defense of the disruption evangelists, I would suggest that she should have done more to at least acknowledge there is a disturbing trend here that needs to be explained. One gets the distinct impression that she believes that disruption is not all that significant

Forces at work in the Big Shift

So, what’s going on here? Based on the work that we’ve done on the Big Shift, let me suggest that the trend towards increasing disruption is a result of the convergence of two powerful forces that are playing out on a global scale.  First, we have the advent of digital technology as a disruptive force.

Yes, we've had major technology disruptions in the past – think of the steam engine, electricity and the telephone. But there’s one key difference that helps to explain a sustained and increasing pattern of disruption today.

In all those prior technology disruptions, we saw a dramatic burst of innovation in the core technology followed by a rapid stabilization with only modest incremental improvements in price/performance afterwards. Then one saw a burst of innovation at the infrastructure level in terms of thinking about how to most effectively deliver that technology to the broader economy and society – think for example of the insight that electricity could be more effectively generated in centralized facilities rather than on the premise of the user.  But once again, that was followed by rapid stabilization in terms of infrastructure deployment, giving firms an opportunity to think through how to most effectively harness this technology in their business.

Digital technology is different – in fact, it’s unprecedented in human history.  It’s the first technology that has demonstrated sustained exponential improvement in price/performance over an extended period of time and continuing into the foreseeable future (based on interviews with scientists and technologists pushing the boundaries of this technology).  

So, there’s no stabilization in the core technology components of computing, storage and bandwidth. As a result, there’s no stabilization in infrastructure – cloud computing is simply the most recent manifestation of this infrastructure and it certainly won't be the last.  And therefore there’s no stabilization in terms of how companies can use this technology to create and capture value.

But there’s more. This exponentially improving digital technology is spilling over into adjacent technologies, catalyzing similar waves of disruption in diverse arenas like 3D printing of physical objects, biosynthesis of living tissue, robotics and automobiles, just to name a few. The advent of exponentially improving technologies in an expanding array of markets and industries only increases the potential for disruption. We’ve explored this expansion of exponential technologies in a working paper here.

And this is just one of the forces at work.  There’s a second force at work as well – a long-term shift in public policy on a global basis towards freer movement of people, goods, money and ideas across geographic and industry boundaries.  Certainly this has unfolded at a different pace in different geographies and industries, but if one steps back and looks at the period from World War II to today, the trend is clear and significant.

Now, I will grant that this second force is a wild card.  There’s nothing to guarantee that this force will continue to play out in the direction of further economic liberalization.  In fact, there’s a growing risk that there will be a significant public policy backlash.  Incumbent players, whether they are taxi cab drivers in New York or global multinational companies facing disruptive attackers, will mobilize in an effort to use regulation and taxation to erect barriers and protect their positions from attack. While I acknowledge this risk, I'm an optimist and I believe that ultimately such protectionist efforts will be undermined by increasingly powerful consumers and by global technology infrastructures that will make public policy barriers difficult to enforce.

But, for the moment, these two forces – exponentially improving technology and economic liberalization – are combining to create environments that are increasingly vulnerable to disruption. In economic terms, they are doing two things.  First, they are systematically and substantially reducing barriers to entry and barriers to movement on a global scale. Second, exponentially improving technology is offering untapped capabilities that can be a catalyst to fundamentally re-think business models and institutional arrangements.

Bottom line, they are catalyzing more opportunity for players to adopt new approaches that can be highly disruptive on three levels:

• The approaches render obsolete a significant part of the existing assets/installed base of incumbents
• The approaches cannot be effectively addressed without significantly cannibalizing existing revenue/profit streams of incumbents
• The approaches are driven by a fundamentally different set of assumptions regarding the drivers of value creation/capture relative to the assumptions that have driven the success of the current incumbents – they challenge the mindsets/mental models of the incumbents

 

And, because they are reducing barriers to entry and movement, they are increasing both the motivation and ability of players to pursue these disruptive approaches.

So, at a macro level, I'd suggest that these forces help to explain why we are experiencing more frequent and widespread disruptions on a global scale. (And, by the way, while my focus here has been on companies, I believe that all institutions are becoming increasingly vulnerable to disruption as a result of these forces).

Some open questions

This perspective of course leaves a lot of open questions.  For example, can we predict where and when specific disruptions will play out? I for one don't believe that such predictions are possible but I suspect that we can better understand specific patterns of disruption that are likely to play out and, as a result, better anticipate which markets or industries might be most vulnerable to certain patterns of disruption. In fact, the Center for the Edge is about to embark on a research initiative in this area and I invite any of you who find this interesting to connect and contribute.

Another question is what incumbent players can do to more effectively respond to these disruptive approaches (short of resorting to regulation and other public policy measures). This is a topic for a much longer post. While I certainly don’t pretend to have all the answers to this, I believe that one key requirement for incumbent players is to find ways to expand the horizons of their leadership team beyond the next quarter or next year and to challenge on a sustained basis the key assumptions, often unstated, that they bring to the table regarding what is required for business success. And, perhaps most basic of all, they need to acknowledge the growing force of disruption and resist the temptation to dismiss or deny that it exists.

Comments

[Nick Seneca Jankel]

Its not just all institutions. In my research (and after 15 years designing and leading disruptive innovation processes for multinational, governments and social orgs), "disruption" or more accessibly, 'breakthrough' - non-linear creative change that could not be predicted based on a linear extrapolation of the past - is a core part of evolution (punctuated equilibrium), all social and cultural history (see Foucault's entire project), politics (see Paul Bremmer's work on the J curve) and even chemistry (exothermic reactions).

It is even more important in our own individual transformation away from anxiety, depression and habitual self-sabotage.

Check out blog on subject. http://www.wecreateworldwide.com/inspiration/blog/rehabilitating-disruptive-innovation/

[Lee Miller]

Disruption has been occurring since the Guttenberg printing press. The primary change is the speed and frequency of change. Clay's analysis is not perfect but it is useful. Jill is seeking to disrupt Clay's franchise but doesn't offer a better analysis.

[Larry Irons]

Can't vertical integration explain the topple rate?

[Flip Schultz]

Just to give you an impression of the sentiment in Dutch finance: leading Dutch financial newspaper FD.nl ran a article on Monday stating 'Disruptors primarily make a lot of noise'. And: 'Valuation of newcomers like Tesla bear no relation whatsoever to the stock market'. Quote from the article: 'The majority of companies that do an IPO, can't be regarded as 'disruptors'. You can say a lot about Australian hospitals, Japanese golfcourses, government controlled media in China and the British AA, but they are certainly no disruptors in their industry, nor will they ever be'. Full article here: http://fd.nl/beurs/nieuws/144650-1407/disruptors-maken-vooral-herrie

[Timo Hämäläinen]

Great post John! Have you noticed that your analysis of the current disruption resonates very nicely with Karl Marx's key theory in which changes in "productive forces" (i.e. te hnologies) lead to changes in "productive relations" (organizational arrangements) which are followed by transformation of the society's entire "superstructure" (institutions) and "consciousness" (ideologies & theories)? Some long wave researchers, such as Chris Freeman and Carlota Perez, have elaborated this approach into very interesting models which fit the current and previous historical transformations extremely well and which are very much in line with your argument. I further developed these long-wave theories in my National competitiveness and economic growth: The changing determinants of economic performance in the world economy (2003).

[Mark Montgomery]

Good work John. I am embarrassed for Harvard and to some degree academia, thinking this debate will serve as a nice marker for future historians attempting to understand tipping points. I only hope they prove to be positive trends.

[Dave Newcomer]

For those who enjoy this discussion W. Brian Arthur's book The Nature of Technology, What It Is and How It Evolves would be a worthwhile read.
Comments such as "disruptive innovation' is annoying shtick" don't advance the discussion and show both an ignorance of history and of current events. Read Arthur on the history of the implementation of the electric motor, about page 158 and compare that to the time it has taken Von Neumann architecture to have real impact.

I have lived out in farm country for many years. Fifteen years ago a farmer thought he did reasonably well if he planted 40 acres in a day. Last year 500 acres was good. Last winter the planter became fully digitally controlled and accurate planting speed went from 5mph to 8 or even 10 mph. Any question about what happened to the "family farm" or that this rate of change is fundamentally disruptive?

[Alan Arnett]

John. Like you I can't see why the two protagonists couldn't just discuss the topic like grown ups, but to add a couple of other perspectives to your inquiry:

One thing I almost never see discussed not just in this example but in similar debates is the extend to which we all have preferences for styles of change or innovation. Whether the labels are radical/incremental, disruptive/developmental or other similar pairings, what I see clouding many discussions is people's personal bias for which examples they pay attention to and prefer. The arguments then are not about reality but about personalities. It takes some awareness and maturity to own your bias and not let it dominate your thinking - there has to be at least a degree of that playing out here, whatever the 'objective truth' might be.

The second thing not mentioned is an extension of your point on technology. For me the disruption increase is not so much the fact of technology, but what it has enabled/created. For good or ill, we are now a globally networked species in a way we never were in previous eras. All of us have potential connections to, and discussions with, people we may never have otherwise met (like this :-)

I can't recall the precise reference now, but Stuart Kauffman did research on complex adaptive systems some time ago now which shows what happens to systems when you do no more than increase the connections between people and the information flow along those connections. As those factors go up, the range and unpredictability of outcomes increases, so what we have is less predictability. Sometimes the outcomes are big, sometimes they are small - its the surprise that gets us, because we tend to like patterns and predictability.

The genie is out of the bottle now in terms of connectivity - we can't disconnect - but we do need to stop acting as if we still live in a predictable world and learn to better handle the discussions with people we disagree with.

[John Maloney]

Good post. I line-up with that history professor on the Charles.

Sorry, 'disruptive innovation' is annoying shtick. It's professorial hubris. We need to get away from the 'disruptive' baloney as fast as possible.

For example, most computing today is based on the Von Neumann architecture of 1945. In the 1960s the prominent computing model of was called cloud computing oh, ahh, err, sorry, meant time-sharing. The electronic tablet was patented in 1888. Magnetic storage was first demonstrated at the Paris Exposition of 1900. So forth and so on...

Yet, every time there is an incremental improvement in these decades-old concepts, some centuries old, it is called 'disruptive.' It's revolting embroidery.

Meanwhile, contrary to the overbearing hype, the facts are that business startups are at a 30-year low. Workforce participation is lower than during the Carter Administration.

Furthermore, the US Administration is patently incompetent. Congress, at 14% approval rating, is objectively dysfunctional. Tax 'inversion' is driving household name firms and their cash hoard out of the USA. Not to mention geometric expansion of global terrorism...

Want shift? This hot mess is one huge, stinking pile of shift.

My money is on Porter everyday. 'Clay' is riding his coattails with irritating and superficial management rhetoric. It's Number 85 is a long-line managerial bombast. See:

http://colabria.com/management-science/

'Disruptive Innovation' is management woo. The sooner it fades the better for everyone.

http://rationalwiki.org/wiki/Woo

[Dahowlett]

@john - one question that talks to the reduction in life span of F500 companies - have you looked at the influence of industry consolidation as a factor impacting that reduction in life span, the concentration of economic power and those influences?

[Reinventioninc]

Women who challenge the status quo: "mean" and vilified. Men who challenge the status quo: "wise" and glorified. Respectful note to Mr. Clayton Christensen and John Hagel: authoritarians who can't tolerate/embrace criticism and decry a professional woman's critique as "mean" in their defense lack wisdom. Criticism forces you to think about your work and ultimately makes your products or ideas stronger.

[KaisaKH]

Excellent post! What I am puzzled with in this debate is firstly the very negative 'framing' of disruptive innovation (that might stem from the original purpose of understanding why do firms fail and the fact that we often seem to approach the issue from the 'failing' actor's perspective) and secondly the narrow definition of innovation only as something 'technological' (while a more broader view of innovation as co-evolution of technology and society might provide some new insights). Looking forward to see where the discussion will head next!

[Jessica Lipnack]

Nice piece, John. I remain puzzled by why Jill Lepore wrote this piece and why The New Yorker published it. It's one of the very few pieces I've seen there that deals with business ideas at all (of recent recall, the Janet Malcolm homage to Eileen Fisher that missed the key ideas in Fisher's thinking and how her approach fits into larger leadership trends). I'm always up for a good takedown of icons so I appreciated Lepore's gumption in going after Christensen and clearly she is right in pointing out that disruption also happens from within. But I don't think we know what was behind this article -- Christensen's history with The New Yorker? Who knows?

[Melanie Nicole]

Hi John, In regards to your last question... Has there been any study around size of organizations? And culture? It seems to me, based on personal observations, that those are the two most common things that prevent organizations from sustaining market leadership. From a top-down perspective, leadership either fails to develop the type of environment (r&d) that truly listens to customers and drives innovation, or internal politics and bureaucracy make it too difficult to implement/execute, so they are vulnerable to those smaller, ruthless,leaderless and unrestrained "upstarts" that are hungry, willing to take risks, can adapt quickly, and are nimble enough to move quickly.

Be happy to help you with your next project if I can work from Ohio!

[Louis V. Galdieri]

Great to see you taking the Lepore/Christensen discussion in new directions. I don't really have a dog in this fight, but I'm nevertheless intrigued by it and by the course your argument here takes.

You say early on here that to explain away the increase in the topple rate between 1965-2012 "one would have to believe that management is becoming significantly more incompetent over time." At first, I passed over the remark with a smile. Nobody believes that, right?

But then your penultimate sentence sent me back to the thought. There you suggest that incumbent players need "to find ways to expand the horizons of their leadership team beyond the next quarter or next year."

That's always a challenge, but it’s worth reminding ourselves that the short-term thinking you decry at the end of your post has a history -- and the most interesting chapter of that history starts right around the time the topple rate increases.

Around 1965, as profit rates in manufacturing fall and as the postwar boom yields to post-industrial reality, some new ideas of management take hold. One of them, “the dumbest idea in the world”: the doctrine of shareholder value.

As this doctrine becomes boardroom religion, we see the rise of the “CEO” as cultural celebrity and corporate savior. Short-termism -- and, in some cases, risky financial manipulation -- becomes the name of the game.

I’m not saying the rise of these ideas or practices completely explain the increase in the topple rate, but clearly bad ideas about what counts as business success -- and misguided actions by business (and political) leaders -- certainly make businesses more vulnerable to the kind of disruption you describe here.

So I'd suggest that failed ideas of corporate purpose and corporate leadership are some of the “key assumptions” that have to be questioned -- and maybe radically altered -- if management is to become significantly more competent, moving forward.

Thanks again for your thoughts.

[Thomas Petersen]

Anyone who have any actual experience with the so called disruptive side of businesses know that Clayton is right.

What he did more than anyone else was to put words on things that up until that point was only felt intuitively. For that alone he has already done more than most business writers.

Sure it's not a perfect theory like a physical fact and there are anomalies.
In innovators solution he says "be patient for growth not revenue" which surely wasn't the way that Facebook went.

Does that discredit his advice for 99% af all companies in this world? No of course not. Does that mean that his theory is 100% correct? Of course not. There are always exceptions.

It's a model, a way to think about companies. The theory part is not important it's the creating a frame of reference to think within thats his biggest contribution to entrepreneurs.

[Gil Friend]

Thanks John, for taking the time to cook such a considered and valuable response. I'm not as sanguine about the inexorability of trend over backlash (not today at least, give the momentum of SCOTUS and the "caliphate"), but I think you're mostly spot on.

There's a third element—disruption of fundamental business models. It's enabled by technology, to be sure, but it has a significant enough momentum that it may be worthy of its own category. I expect impacts to reach beyond pricing to ownership models, scale w/o scale, and getting the prices right (eliminating the economistic myth of externalities) to name a few.

As for you new research initiative, I'd be very interested to connect and contribute.

[Dscofield]

John - I echo Saul's words. There were several things that hit me about Lepore's article:

- Disruption is all around us - and it's not just technology as you say. The fact Lepore, a historian, didn't 'see' this is disheartening on many levels (including as a mom whose son is looking at colleges and wishes profs were more open-minded!). There are huge disruptions going on in government, societal structures, definitions of 'work', education, etc. Couldn't one see the Ukraine-Russia issues as the last gasps of an old empire desperately trying to hold on to something that's evaporating? The issues in Iraq highlight the folly of artificial national boundaries that are not sustainable and I could go on and on.

- The fact the New Yorker published the article actually made me think of Putin invading Crimea! Please please, don't take my deluded world view away from me! It was as if, through Lepore, the NYer was asking for the restoration of the 20th C. - perhaps so desperately that they would publish an article that was not really up to the 'old' NYer standards (e.g., walk across the street to talk to the guy before you lambast him in the press).

Thank you again for your sage words. This incident is so telling on so many levels. See you soon at #BIF10!

[Saul Kaplan]

I'm glad you waded in John and agree with your response. I'm particularly glad you mentioned toward the end that these trends and resulting disruption affect all institutions not just companies.

Lepore seems most intent on criticizing anyone who is silly enough to think theories describing how commercial ecosystems work might also apply in the social system world including education, health care and government.

To my way of thinking all organizations have a business model. At least the sustainable ones anyway. Social systems are comprised of networked business models that have evolved to coexist together. The most interesting new business models and the ones that will help us solve the biggest social system challenges we face are networked models that cut across the intransigent industry, sector and discipline silos that evolved over the industrial era.

To say that disruptive forces aren't relevant to social systems is both naive and dangerous.

I hope all is well and can't wait to see you at #BIF10!

Saul Kaplan (@skap5)
Founder and Chief Catalyst
Business Innovation Factory