For a LONG time, I’ve been talking about how all interfaces can be defined in terms of IFaPs (Identifiers, Formats, and Protocols). My canonical example of an interface composed of IFaPs is of course the Web: URL (I), HTML (F), and HTTP (P). All three intersect in a particular instance of HTML, say my blog’s home page. The HTML for my blog’s home page is filled with URLs, HTML tags, and even HTTP "verbs" (though these are quite rare, mostly in an HTML form or embedded JavaScript).

Then along came REST and with it the concept of HATEOAS: Hypermedia As The Engine of Application State. And everyone, myself included, spent a lot of time trying to grok it and explain it to others. We’re still trying. One way I try to explain it is by highlighting that HATEOAS requires that each server response must contain not only the requested data — but also control information (in the forms of specially tagged URLs) describing the next set of permitted interactions with the server. It is this additional control information (at a bare minimum just some links to more data) that turns mere media into hypermedia.

Now along comes Jim Webber with a much better (dare I say brilliant) way of explaining HATEOAS and hypermedia: "Hypermedia Describes Protocols!" (See slide 26.) At first this might seem counterintuitive, since I said that HTTP is the Protocol and HTML is the Format in the WWW. But URLs, HTML, and HTTP are just generic description languages for describing domain-specific identifiers, formats, and protocols. Thus, think of a web of specific HTML pages as a domain-specific protocol. Jim Webber uses the example of ordering a Starbuck’s coffee. (What’s important is that each hypermedia DSL is composed using the generic languages of URL, HTML, and HTTP.)

This notion of bringing together identifiers, formats and verbs to describe a protocol is not new. One of the best descriptions of this was in the WS-BPEL 1.1 spec:

In thinking about the data handling aspects of business protocols it is instructive to consider the analogy with network communication protocols. Network protocols define the shape and content of the protocol envelopes that flow on the wire, and the protocol behavior they describe is driven solely by the data in these envelopes. In other words, there is a clear physical separation between protocol-relevant data and "payload" data. The separation is far less clear cut in business protocols because the protocol-relevant data tends to be embedded in other application data.

So if WS-BPEL was already thinking about mixing protocol data with "payload" data, what’s so new about HATEOAS? The fundamental difference is that WS-BPEL is based on the concept of providing an entire static protocol description up front once and for all — and providing it out of band. But HATEOAS is based on the notion of progressive description (don’t bother Googling the term, I coined it; and not to be confused with progressive disclosure). More and more of the description of the protocol is provided to the client (in band in the protocol itself) as the client executes its part of the protocol. I guess another good term might be JIT Protocol Description (Just In Time). Another good term might be "self-describing protocol". So now when explaining HATEOAS, instead of saying "each server response must contain control information" (huh?), I can say "each server response progressively self-describes the current protocol."

Now there are pros and cons to static/complete vs dynamic/progressive protocol descriptions. How can I program a client to execute its part of a protocol if I don’t have a full description of it up front? But if I encode the complete static description of the protocol into my client up front, how can I change the protocol dynamically?

Love to hear others’ thoughts. I’m going to think about this some more. That’s why I love epiphanies — they make you think about things in new ways.

While I agree with many of the sentiments behind Anne’s declarations that “SOA is dead”, I disagree with her way forward: “long live services”.

It is services thinking, as conventionally understood, that led to the mess in which we find ourselves: fragmentation caused by entity-specific (service) interfaces. I’d say instead, “long live the web.” I’m shocked that Anne’s blog post does not even mention the web!

I agree when Anne says, “it requires redesign of the application portfolio. And it requires a massive shift in the way IT operates.” But the disruptive redesign required is to make IT more Web-like — both in the architecture of software and in the way the ITO operates.

The most “spectacular gains” we have are those of Google, Amazon, and even Salesforce. What they have in common is an embrace of the Web, including web architecture, web community, and web business models. To paraphrase Anne’s blog post:

“Web-orientation is a prerequisite for rapid integration of data and business processes; it enables situational development models, such as mashups; and it’s the foundational architecture for SaaS and cloud computing.”

We all know about generic interface operations, eg HTTP’s GET, PUT, POST, DELETE, et al. And we are beginning to see more examples of generic formats (or generic schemas if you prefer). For example, ATOM and derivatives like GData.

But its harder to find compelling examples of generic identifiers that are web-oriented, ie, defined as URLs. Well, I just found at great example at the BBC!

| View | Upload your own

See slides 35 and on.

The basic idea is that MusicBrainz uses a GUID-based set of identifiers to uniquely identify artists, albums, and songs, like this URL for Led Zeppelin. So instead of inventing YAUN (Yet Another URL Namespace), BBC wisely decided to reuse MusicBrainz — as have others (last.fm).

While I’m not sure I like the machine-generated, non-human friendly GUID approach, I do love the idea of id sharing across organizations. Such sharing is the essence of the network effect.

For more information see Web Scale Identifiers, Automatically Linking Artists, Permanent Web IDs, It’s All About the URL, and BBC/Music/Brainz. The BBC definitely gets WOA and network effects.

Because of such consensus, the note can put forward Gartner positions such as:

• Interfaces based on WS-* specifications should be constrained by WOA, especially the generic interface constraints.
• More often than not, the WS-* protocol toolkit is unconsciously misused to create needlessly specialized interfaces.
• Application neutrality should be the principal goal of an interface, and implementation neutrality should be a secondary goal.

While I can’t share the entire note with the blogosphere, I can share a couple of highlights — first and foremost the official Gartner definition of Web-Oriented Architecture:

WOA is an architectural substyle of SOA that integrates systems and users via a web of globally linked hypermedia based on the architecture of the Web. This architecture emphasizes generality of interfaces (UIs and APIs) to achieve global network effects through five fundamental generic interface constraints:

1. Identification of resources
2. Manipulation of resources through representations
3. Self-descriptive messages
4. Hypermedia as the engine of application state
5. Application neutrality

Those of you familiar with Roy T. Fielding’s REST Thesis will no doubt recognize that WOA’s five generic interface constraints are an extension of Roy’s four uniform interface constraints. The one additional constraint, application neutrality, is implicit in the thesis, but we think it is so fundamentally important that we made it a "first class" constraint.

What is application neutrality? Here is a brief excerpt from the 13-page report:

The primary problem with the specifications known as WS-* (such as SOAP, WSDL and UDDI) is that their principal emphasis is on implementation neutrality. All the specifications focus on generalizing away the details of specialized middleware technologies, so that services can be accessed using any one of those technologies. Although this is not an unworthy goal (especially for vendors of specialized middleware technologies), it shifts the focus from the generic interface constraint of application neutrality.

Application neutrality should be the principal goal of an interface, because it is precisely this characteristic that enables shareability (a fundamental SOA principle). In other words, interface designers’ primary goal should be generic, application-neutral interfaces, which generalize away application-specific details.

The key to shared use (reuse) is a generic, application-neutral protocol, such as the Atom Publishing Protocol (APP) or Google’s GData Protocol. Conversely, the more application-specific a protocol is, the less shareable it is. With sufficient generality, the most powerful kind of reuse becomes possible: serendipitous reuse. So important is this kind of reuse that Tim Berners-Lee and Roy T. Fielding have highlighted it an essential characteristic of the Web:

• "Unexpected reuse is the value of the Web" (Tim Berners-Lee)
• "Engineer for serendipity" (Roy T. Fielding)

In terms of Gartner’s hourglass model for identifiers, formats and protocol operations (IFaPs), application neutrality makes the top of the hourglass wider; implementation neutrality makes the bottom wider. A wide top is more important than a wide bottom. In other words, generic application protocols (application neutrality) at the top of the hourglass are more important in creating powerful network effects than portable implementation protocols (implementation neutrality) at the bottom of the hourglass.

…

The key to generic interface design, WOA-style, is nested generality: gradually specializing generic interfaces in small increments. Whatever level of generalization one begins at — whether it is at the APP-envelope level or the SOAP-envelope level — it is a mistake to embed application-specific schemas and identifiers in either envelope.

Don’t just build on generic interfaces; build up generic interfaces that are only slightly less generic than those built on.

One last point highlighted in the note: let’s not get into a battle over names. If you don’t like the name WOA, call it REST, or ROA, or Web Architecture, or Fred. "The goal is to focus on the key generic interface constraints that unite these concepts, not debate the nuanced differences among them."

We’d love to hear your feedback on this key set of concepts –especially application neutrality — whatever name you choose to call it!

The Customer State Machine

The Barista’s State Machine

The answer is the uniform interface constraint (UIC), which you can also think of as the “generality constraint”. Since an interface is composed of identifiers, formats, and protocol operations, the (UIC) mandates that:

1. Identifiers be as generic as possible
2. Schemas be as generic as possible
3. Protocol operations be as generic as possible

for the type of application being architected.

Everyone who’s looked at REST now seems to get the generic protocol operations part: GET, PUT, POST, DELETE, etc. And some are now getting the generic formats part: Atom Publishing Protocol, GData.

But very few are focusing on the generic identifiers part yet — at least in terms of explicitly RESTful thinking. The closest thing to generic identifier work in enterprises is master data rationalization. Gartner defines master data as consistent and uniform identifiers and attributes for key business entities used across many business processes (at least that’s the gist of our definition). To make the master data connection to REST explicit, “consistent and uniform” identifiers across many processes translates into generic identifiers, or if you prefer Roy’s terminology uniform identifiers.

So REST doesn’t so much solve the data semantics problem; rather, properly understood, REST’s UIC puts the data semantics problem, ie mandating uniform, generic semantics instead of ad hoc, specialized data semantics, front and center. REST is the only architectural style I know of that makes interface generalization the primary imperative. On the other hand,  WS-* implicitly or explicitly encourages interface specialization. Steve Vinoski has a great article on the evils of interface specialization: Serendipitous Reuse.

Hope that helps clarify.

After such conversations, I typical send a follow up email with some good examples of “linked data” and some pointers on versioning IFaPs — two of the most critical aspects of a successful information architecture IMO. Here is one such example…

Here are some links to the Linked Data initiatives I mentioned:

1. http://en.wikipedia.org/wiki/Linked_Data
2. http://en.wikipedia.org/wiki/Open_Data
3. http://esw.w3.org/topic/SweoIG/TaskForces/CommunityProjects/LinkingOpenData

 I forgot to mention one other pioneering effort in “linked data”: the use of XBRL by the FDIC et al and the SEC. XBRL.org just published the US GAAP Taxonomies in XBRL — a major achievement: http://xbrl.us/pages/us-gaap.aspx   . The links on this page lead to a nice “taxonomy browser” that makes it (relatively) easy to navigate these complex taxonomies. 

Finally, we only mentioned it in passing, but extensibility/versioning strategies for XML and other Web languages (e.g., HTML) is another poorly understood topic. I think people read the X in XML (meaning eXtensible) and believe (mistakenly) that XML is extensible by default. It is NOT. Here are some useful references into the leading thinking on this issue:

1. http://www.w3.org/2001/tag/doc/versioning-compatibility-strategies
2. http://www.w3.org/2001/tag/doc/versioning
3. http://www.w3.org/2001/tag/doc/versioning-xml
4. http://www.pacificspirit.com/Authoring/Compatibility