From Race Against the Machine to Race With the Machine

The recent book, Race Against The Machine, has caught the imagination of a growing body of readers. It’s an important book, but it doesn’t go far enough in highlighting the root causes of the unemployment we are experiencing. Rather than framing it as a technological issue, the book would have generated a lot more insight about both the problem and the solution if it had framed it as an institutional issue.

The authors, Erik Brynjolfsson and Andrew MacAfee, take a current news topic that has engaged the hearts and minds of many – persistent unemployment in the US – and suggest that this is a structural issue, not simply a temporary, cyclical event that will quickly improve. They argue that this issue is unlikely to go away in the foreseeable future and may actually become much more severe before it becomes better.

The economic impact of technological progress

Their goal is simple, as the authors state in a sub-heading of their first chapter - “Our Goal: Bringing Technology into the Discussion.”  Appropriately, they start by tracking the rapid advance of digital technology that is increasingly able to take on tasks that we would never have expected possible a few short decades ago.

They then proceed to examine the creative destruction that is driving growing mismatch between economic winners and losers. The authors in particular explore the paradox that advances in terms of economic productivity can at the same time leave a growing number of people behind. As they observe, “there is no economic law that says that everyone, or even most people, automatically benefit from technological progress.” In particular, they highlight three divergences that are increasingly shaping the US economic landscape: “between higher-skilled and lower-skilled workers, between superstars and everyone else, and between capital and labor.” They say the problem is even more stark because “. . . the winners in one set are more likely to be winners in the other two sets as well, which concentrates the consequences.” This is the richest part of the book and there is much to be learned about the economic impact that technology has had on our workforce.

What is to be done?

Race Against the Machine purports to be an optimistic book, but the description of the challenge is far more detailed and compelling than the brief chapter that addresses “What Is To Be Done?” As a framework for resolution of the growing challenge of digital automation, the authors propose that “. . . the key to winning the race is not to compete against machines, but to compete with machines” and ". . .we can learn to better race with machines, using them as allies rather than adversaries.”

This is a powerful turn of phrase, but when the authors dig into how to make this come alive, the specific recommendations appear pretty general.   First, they suggest that we need to foster “organizational innovation: co-inventing new organizational structures, processes and business models that leverage ever-advancing technology and human skills.”

They indicate there’s an opportunity for creative entrepreneurs to imagine new kinds of businesses that can harness the power of machines, pointing out that there are limitless opportunities for re-combinations of ideas and individuals and that “parallel experimentation by millions of entrepreneurs” is the best way to find the winning combinations. There’s no doubt that entrepreneurial energy can accomplish great things, but why has this entrepreneurial energy not been more effective in responding to the structural unemployment to date?

The second major plank of what is to be done involves investing more in education and training to build skills that are complementary to machines rather than vulnerable to automation.  Their more specific recommendations in this domain include adopting new technology more aggressively, paying teachers more, eliminating tenure and keeping students in class longer.

There are other recommendations as well, including increasing investment in our national infrastructure, spending more on basic R&D and a variety of public policy reforms.

Yet, reading through all of these recommendations, I can’t help but think that the authors are just skirting the real issue. The recommendations are “directionally correct” but, without a deeper analysis of the problem, are likely to fall short of the answers we desperately need to cope with the inexorable onslaught of digital technology as exponential improvements make more and more existing jobs vulnerable.

The authors come closest to a powerful answer with their call for “organizational innovation” but the call is far too vague and general. It can't be made more specific and actionable without another layer of analysis as to the root cause of the problem we confront.

Re-framing the core issue - is it technological or institutional?

Let me suggest that the real problem we are confronting is a growing disconnect between the institutional architectures we have so carefully designed over the past century and the pressures that are mounting in our global economy.  This disconnect goes to the very core of why we have institutions in the first place.  Until we address that most basic question, entrepreneurial tinkering will likely have marginal impact.

Ronald Coase won the Nobel Prize in economics for an essay on The Nature of the Firm he wrote in 1937 that provided a simple and compelling answer to the question of why we have corporations. He argued that we need corporations because they reduce transaction costs – the costs of coordinating and executing economic activity. Put in slightly different terms – the rationale for corporations resided in the quest for scalable efficiency.

This was an amazingly accurate analysis of the rise of the modern corporation in the 20th century.  Huge companies were built and enormous wealth generated by pursuing this quest. But here’s the catch. How was scalable efficiency attained? It was achieved by adopting large-scale push programs.  Driven by forecasts of demand, push programs required highly standardized and rigorously specified work activities that were closely monitored to ensure predictability.  The modern, thick process manual was the end-product. The job of the individual was to fit into this tightly regimented work environment and perform predictably and efficiently.

Now, think about this. If we reduce work to highly specified and standardized instructions that can be performed efficiently and predictably, what have we done? We have reconceived work so that it can be performed by computers and robots. In fact, computers and robots are far more preferable than humans because we humans are ultimately unpredictable and have a really hard time following instructions to the letter, day in and day out.

In this environment, it’s quite natural to view workers as fungible, cost items. When pressure mounts, the natural reaction is to cut costs by cutting workers, replacing them with machines where possible and, where not, simply making the remaining workers work harder.

Given this perspective, it’s no longer surprising that we see the hollowing out of the middle class.  Where did the middle class come from? They are the workers who so faithfully carried out the tightly programmed tasks defined by our push driven institutions.

Sure, as the authors point out, some of the routine work we do cannot yet be performed by computers, but it's just a matter of time before the delivery truck driver and the customer service rep can be fully automated. And what about all those wonderful things that the authors indicate will never likely be automated – imagination, creativity, genuine insight and emotional and moral intelligence? These attributes have no place in the push driven institutions we have built. They are ruthlessly rooted out wherever they rear their ugly heads.

Smaller, entrepreneurial organizations of course buck this trend, but what happens as soon as they start to scale?  They embrace push programs and rapidly join the ranks of the regimented. Why shouldn’t they? That’s the tried and true way to succeed.

Why should we be surprised that computers and robots are taking over more and more of the work in push driven companies?  And it’s not just companies.  The doctrine of scalable efficiency pervades all of our institutions – schools, NGOs and government agencies. We have all embraced push programs as the way to achieve success.

So, what's the answer?

Until and unless we get to the root cause of the problem, we’ll never solve the problem. The authors frame the issue as a technological issue, but it’s really an institutional issue. Until we can develop an alternative institutional model, one that can scale as effectively as the scalable efficiency model, we will face mounting pressure from machines and remain locked in a race against the machine without the ability to finally race with the machine.

But here’s the good news.  The scalable efficiency institutional model is fundamentally and irreversibly broken as I’ve argued at great length in The Power of Pull, with my co-authors, John Seely Brown and Lang Davison.

Why is it broken? Because digital technology has re-shaped our global business landscape in profound ways. It makes forecasts and predictions more and more challenging as we see more and more volatility and the increasing frequency of extreme, unanticipated events (“black swans”).  We increasingly find ourselves in a Paretian world but predictability can only be found in a Gaussian world where averages are meaningful. The push programs that seemed so essential to scalable efficiency now produce the opposite: increasing inefficiency, as rigidly constructed programs face unanticipated changes in the market.

Equally importantly, we’re moving from a world of knowledge stocks, where competitive advantage resides in proprietary knowledge of lasting value, to a world of knowledge flows, where competitive advantage can only be attained by participating effectively in a larger and more diverse set of knowledge flows. In a world that’s changing more rapidly with growing uncertainty, knowledge stocks depreciate in value at an accelerating rate.

This suggests an alternative rationale for institutions. Rather than pursuing scalable efficiency, perhaps we need a new set of institutions that can drive scalable learning, helping participants to learn faster by working together.  While simple to state and intuitively appealing, this requires profound changes to our institutional landscape. 

Rather than relying on rigid push programs, we need to increasingly develop scalable pull platforms where people can draw out people and resources where they are needed and when they are needed, not just to perform pre-defined tasks, but to engage in creative problem-solving as unanticipated challenges arise. Interestingly, the authors of Race Against the Machine, cite a number of promising entrepreneurial initiatives that all turn out to be examples of scalable pull platforms, but they don’t step back to really develop what is different about these pull platforms or to explore their potential for accelerated learning and performance improvement.

These pull platforms have an interesting property.  They not only accommodate, but demand, the attributes of participants that are least susceptible to automation – imagination, creativity, genuine insight and emotional and moral intelligence. In fact, these pull platforms catalyze, cultivate and reward these attributes – the same attributes that are so suspect in today’s push driven institutions. In pull-driven institutions, participants are no longer fungible cost items but instead become fully visible as assets with the potential for virtually unlimited development.

One more point. These pull platforms are much more challenging to scale without digital technology infrastructure. At long last, we will have an institutional framework that requires us to race with the machine, rather than simply racing against the machine.

This emergence of pull-based institutions isn’t simply confined to the domain of corporations. It will pervade all institutional domains. For example, our educational system is a classic push driven environment – not surprising, given that its primary mission was to prepare individuals to enter the workforce of push driven corporations. Adopting this institutional re-framing in education points out the limitations of the authors’ recommendations on education – simply investing more money and working longer hours in a push driven educational environment will have only marginal impact at best.

The bottom line

Until and unless we re-frame the challenges we face, we’ll have little hope of developing effective programs of change. At its core, this isn’t a technological challenge, but an institutional challenge. We're dealing with a set of institutions that are increasingly inappropriate for the mounting pressure we face. At an even more fundamental level, it's a mindset challenge – it’s about our beliefs about the kinds of institutions that we need to assure our prosperity and well-being.

The good news is that if we can re-frame the challenge, the challenge becomes an opportunity. For the first time, we have an opportunity to build institutions that can help all of us to achieve more of our potential and increasingly differentiate us from machines, while harnessing their power to amplify our efforts. We have reached a point in our history where small moves, smartly made, can set very big things in motion.

Comments

[FMX]

Freedom creates the middle-class, reduces hardship and defines growing success for all; dirty statism and its beguiling relatives communitarianism and social rights are eliminating it.

[Todd Sampson]

Thanks John. Great post, as usual.

It seems like the tools that are emerging on the internet, and many of the examples in The Power of Pull, are available to everyone -- inside or outside the corporation. Passionate people are using these tools to create amazing things, from open source software using Github to new products funded through Kickstarter, without the need for the corporation. And since the tools being used are built at such a large scale, how could a corporation ever hope to offer something more to its team members? (Even when knowledge stocks were more valuable than today, how often did people search the corporate intranet before hitting Google?)

Have you seen any new examples of institutions being able to provide better access to learning, resources, and/or collaborative acceleration for individuals than they can access on their own?

[John Maloney]

Guess this is informative for some people, but it is just (yawn) a retread of concepts discovered ‘bout 15-20 years ago. It usually takes 2.5 generations (50 years) to retire such key concepts as Coase's transaction-cost economics (TCE). It's well underway. It not because of lofty books but because it sucks and doesn't work anymore. The knowledge-based view (KBV) is on the rise.

Why can't it happen faster? It is because of overreaching govt regulation, unions and indifference to innovation for starters.

What is a more perfect example of reducing "work to highly specified and standardized instructions" than a labor union contract the size of the Manhattan phone book? It's classic push. What imbeciles say, 'pay teachers more' to get MORE push crap that fails students, parents and society w/confidence? Good grief.

It is high time to get our 'pushy' deficit-based, obese government apparatus out of the way.

For example, welfare, a classic govt push, expands the very problem it is supposedly intended to fix: poverty. How dim are people?

Take healthcare. Do people really want US healthcare reduced to a push system of “highly specified and standardized instructions” from a central govt? Are people fucking insane? What is WRONG with people?

Pull IS great an powerful, but govt is in the way.

Liberty creates the middle class, relieves poverty and achieves growing prosperity for all; filthy statism and its beguiling cousins communitarianism and social justice are killing it.

[Dave_Ideate]

Hi John - Great post and timely as I just watched Andrew MacAfee's related TED presentation yesterday. I did walk away, enjoying the analysis, but found the conclusion a bit pollyannish.

I think we should all be a bit concerned about letting the systems 'run the show' without human intervention - that can lead to runaway situations which machines chase optimization in a way that is destructive - like automatic trading sell-offs.

We shouldn't forget that as good as the algorithms are and will get, they are 'predictive' based on assumptions and can lead to self-fulfilling prophecies that reinforce biases of algorithm designers. There's also the Black-Swans you point out and unknown-unknowns - we can't assume too much.

If we can replace journalists with software agents that can write 'perfect' articles, sure we can replace CEOs, CFOs, etc with imperfect records and bloated salaries.... very inefficient!

This takes us to a pretty dark place. At it's core this thinking is Anti-Social. If, in our quest for reduce transaction costs to increase access to goods and services for people (a social good), we accept increasingly high-levels of unemployment (reducing buying power and status of people), this is going to disrupt the social compact.

Unless we are moving to a Utopian candy mountain, unlikely, I agree that we need to re-think and re-organize the enterprise. We need to shift from hierarchical/reductionist system thinking to networked/collaborative systems.

I wrote an article on this in a tech journal last year - http://www.infoq.com/articles/social-lean-agile. It cites your work on the "Shift Index".

Best Regards,
Dave

[Jonathan Dean]

Hi John...Thank you. I always enjoy your excellent posts and books very much.

A question, though (and I have read Power of Pull, but don't recall the answer clearly); you write:
"In pull-driven institutions, participants are no longer fungible cost items but instead become fully visible as assets with the potential for virtually unlimited development."

My question is why is this? Why are pull-driven institutions not just a different set of processes being performed, with employees performing fundamentally repetitive actions?

Very much appreciate any insight and help with this.

[twitter.com/terrigriffith]

John, as always, great analysis and insight. I'm optimistic that the time is right for the ideas you offer to take root on a broad scale. In conversations with people ranging from my sailing buddies to academics from technology institutes in Germany, there is agreement that work is becoming more flexible and with a greater demand for personal innovation.

People are seeing that human, technology, & institutions provide the building blocks and that we must push forward with transparency, responsibility, and growth (I'm trial ballooning my book proposal here). ...and I'm seeing entrepreneurs begin to build the tools to support these new forms of work. We have the tools and broad motivation - in the past we may have only had the motivation from the grass roots -- the time seems right.

[Luis_AL]

John,

It is a great article, thanks for sharing

The question I would have is, how do we transform existing push institutions ? I understand building "pull" institutions, but changing existing ones is extremely hard. As you write: "what about all those wonderful things that the authors indicate will never likely be automated – imagination, creativity, genuine insight and emotional and moral intelligence? These attributes have no place in the push driven institutions we have built. They are ruthlessly rooted out wherever they rear their ugly heads"

Actually, the corporate function that should be responsible for fostering imagination and creativity (HR), is today the owner of the classic managerial mindset in most corporations. And even though, individually, many HR executives would like to move in the direction you point out, moving the system is just to hard.

I argued some years ago (http://luisalberolasblog.blogspot.fr/2009/12/beyond-enterprise-20-age-of-builders.html) that the E2.0 movement was an opportunity as it allowed HR to frame a strategy to develop the firm employees from "users" to "contributors" to "builders". But I still do not see that happening at scale.

Maybe you have some examples of push firms transforming themselves into pull firms ?

[BrianVellmure]

John,

Great writeup. Thanks for moving the conversation forward here and on other channels.

- Brian

[Jaykinghorn]

In general, I agree that the issue is institutional, though I would dig even deeper and say the problem, ultimately, is one of mindset. For generations, we've been conditioned to think linearly, with predictable changes and growth curves. Technology and the internet, however, are progressing exponentially, often in chaotic fits and spurts. The gap between these the linear and exponential mindsets, though slight at first, is widening. As a result, our current institutions, models and structural concepts are stretched to the breaking point.

As you articulate very well, there is a growing disconnect between the institutional architectures we've developed since the industrial age and what is needed today. I appreciate thinkers, like yourself, who've identified these trends and can articulate them to help spread the message. Unfortunately, these are radically new concepts to most individuals, particularly those most affected by these changes, and I've seen little substantive discussion about this in the popular media today. Are you seeing business leaders embracing the pull mindset or it is still an edge concept?

Thanks for the thought-provoking blog post!