At the end of last year, I posted on the relevance of Web 2.0 technologies for the enterprise. Over the past couple of months, this theme has received a lot more attention, prompted in large part by the writings of Dion Hinchcliffe and by the discussions at two industry gatherings – SPARK/MIX 06 and Software 2006. (For those who have not been following my blog, I attempted my own definition of Web 2.0 here.)
Some of Dion's key postings on this topic are "Running a business on Web-based software", "Web 2.0 for the enterprise: Where the action is?" and "Web 2.0 for the enterprise?". For discussion of the SPARK/MIX 06 event, see Dion Hinchcliffe here and here, David Hill, Dan Farber, and Phil Wainewright here and here. Also, one of the highlights of the MIX 06 event was an exchange between Bill Gates and Tim O'Reilly and the transcript is available here. For commentary on Software 2006 and the Software 2006 Industry Report prepared by McKinsey & Co. and the Sand Hill Group, see Dion Hinchcliffe, Dan Farber and Ross Mayfield here, here and here.
The discussions in these forums raise one key issue: what is the relationship between Web 2.0 and Service Oriented Architectures (SOA)? Dion in particular has been wrestling with this topic, including his postings on "The SOA with reach: Web Oriented Architecture", "Web 2.0 The Global SOA" and "Is Web 2.0 the GLOBAL SOA?"
As I indicated in my previous posting, a cultural chasm separates these two technology communities, despite the fact that they both rely heavily on the same foundational standard - XML. The evangelists for SOA tend to dismiss Web 2.0 technologies as light-weight “toys” not suitable for the “real” work of enterprises. The champions of Web 2.0 technologies, on the other hand, make fun of the “bloated” standards and architectural drawings generated by enterprise architects, skeptically asking whether SOAs will ever do real work.
This cultural gap is highly dysfunctional and IMHO precludes extraordinary opportunities to harness the potential of these two complementary technology sets. Perhaps because of this cultural gap, we lack a crisp articulation of the relative merits of these two technology sets for the enterprise.
We can make a lot of progress by looking at two key tasks addressed by these technologies: connection and composition. In the early days of Web services (the early standards that became the precursor to both Web 2.0 and SOAs), evangelists painted exciting visions describing the opportunity for enterprises to dynamically compose exciting new applications from modular services. In practice, though, the early production deployments of Web services within the enterprise have focused on a much more prosaic task: connecting large, unwieldy legacy applications together so that businesses can get more value from the data and application functionality already in place.
Connection of resources
When you talk to SOA proponents today, you will hear a lot about connecting applications and databases, but not a lot about connecting people together and helping to support their interactions with each other. In contrast, Web 2.0 advocates put a lot more emphasis on the opportunity to connect people together and to support their collaborative efforts. Web 2.0 certainly also addresses issues of connecting applications and data, but Web 2.0 is distinctive in the social dimension that it explicitly addresses.
The next wave of innovation by enterprises will depend on the ability to connect people together more effectively, especially at the edge of enterprises, and provide them with tools to support collaborative creation. In this context, Web 2.0 technologies like wikis will play a key role in driving value creation in the enterprise. As Dion Hinchcliffe has written, the architects and software engineers that dominate enterprise IT departments disconnect in discussions of Web 2.0 technologies because of the strong social aspect addressed by these technologies. SOA proponents ignore these technologies at their peril.
Andrew McAfee has just published “Enterprise 2.0: The Dawn of Emergent Collaboration” (purchase required) in Sloan Management Review highlighting the role of Web 2.0 technologies in connecting people within the enterprise. It focuses on the potential role of Web 2.0 technologies in knowledge management:
Enterprise 2.0 technologies have the potential to let an intranet become what an Internet already is: an online platform with a constantly changing structure built by distributed, autonomous and largely self-interested peers. On this platform, authoring creates content; links and tags knit it together; and search, extensions, tags and signals make emergent structures and patterns in the content visible, and help people stay on top of it all.
It’s a great article, but it focuses too much on knowledge capture and not enough on the use of these tools for collaborative knowledge creation. Focusing these tools on knowledge creation is likely to be the way that these tools overcome the skepticism expressed by Nick Carr. For those interested, McAfee responds to Carr at his blog and he has also developed a Harvard Business School case study on Wikis at Dresdner Kleinwort Wasserstein.
Composition of platforms
Both Web 2.0 and SOA technologies re-conceive software as services. Perhaps even more importantly, they view services as platforms. Rather than viewing services as standalone offers designed to be consumed exactly as written, both sets of technologies start with the vision that the role of any service is ultimately to become the building block for even more services that will be built on top of the original service.
Amazon provides an early, and very limited, example of this opportunity. By developing an affiliate program and offering a book buying service that can be embedded into other web sites, Amazon has been able to significantly expand its reach and create a much more robust platform for driving e-commerce activity.
Both sets of technologies share the same vision, but they are deeply skeptical of each other in terms of the approach used to accomplish this vision. Web 2.0 champions dismiss SOAs as much too rigid and slow moving in terms of building platforms for cumulative creation. Here’s the irony. SOAs initially generated significant interest within the enterprise because they appeared to offer a much more flexible and rapid way to build new application functionality relative to traditional enterprise application architectures.
What happened? SOAs were hijacked by an alliance of CIOs and IT consulting firms, each with their own reason for extending the effort required to deploy SOAs.
CIOs have become more and more risk averse for a variety of reasons. As a result, they are very nervous about the deployment of new technologies within the enterprise (and even more so across enterprises). Their natural instinct is to resist the rapid deployment of new technologies that might disrupt operations in unforeseen ways.
Many IT consulting firms, on the other hand, are economically dependent on projects that require a lot of consultants to work with clients for extended periods of time. As a result, they have a natural incentive to emphasize the complexity and issues associated with SOA definition and deployment.
The growing appeal of Web 2.0 technologies in part stems from this hijacking of SOAs. Line executives within the enterprise are experiencing mounting frustration over the escalating hype around SOAs, the growing spending over SOA design initiatives and the relatively limited business impact achieved by SOA deployments. In contrast, Web 2.0 initiatives are leading to a proliferation of mashups (one form of composition), as described by Dion Hinchcliffe in "The Web 2.0 Mashup Ecosystem Ramps Up" and "Some Predictions for the Coming 'Mashosphere' "
Breaking the logjam
What is required to break this SOA logjam? Two things. First, Web 2.0 technologists need to work on connecting directly with line executives of large enterprises without trying to go through the IT departments. Second, they should avoid the temptation to present grand visions of new architectures and concentrate instead on starting points where these technologies can deliver near-term business impact. (This should not be too hard since by nature Web 2.0 technologists are bootstrappers and hackers.)
Adoption will be guaranteed if they can show that tangible savings can be generated in a six to twelve month period with modest investment. When they engage these line executives, they should focus on the distinctive value of connecting people and deploying service platforms that support rapid incremental composition of new application functionality. To make these discussions tangible, they can point to the growing array of Web 2.0 software that is usable by the enterprise (see the list compiled by Jeff Nolan at SAP, with a hat tip to Dion Hinchcliffe).
Does this mean SOAs are DOA? Not at all. SOAs still provide a valuable foundation to support the sustained relationships required for distributed creation. But these SOAs need to be deployed in a much more incremental and pragmatic way. Perhaps a little competition from Web 2.0 technologies will help to break the logjam and force both IT departments and IT consultants to adapt their culture and operations to growing business pressure for accelerated impact and learning.
As JSB and I discuss in much more detail in The Only Sustainable Edge, the convergence of SOAs, virtualization architectures and Web 2.0 social software will drive the next wave of value creation within and across enterprises. The convergence will not unfold smoothly, as much of the current debate confirms, but it will take place - there is too much at stake and each of these technology arenas offers something distinctive in supporting next generation business platforms.