Cloudstreams: The Next Cloud Integration Challenge

I’m about to introduce a term to describe a solid trend in cloud computing integration – Cloudstreams.

It will come as no surprise to those of you who have spoken to me that this trend lays in the domain of cloud services brokerage. I have said, and will continue to say, that cloud services brokerage represents the single biggest revenue growth opportunity in cloud computing and that it is built on markets totaling near one trillion dollars in IT spend. That’s all well and good.  But, when we get down to the details, we find that the biggest impact for actual customers will be found in a combination of four categories of brokerage – integration, customization, governance (including security), and aggregation.

Cloudstreams focuses on the integration, governance, and security impact points. Trust me, the definition is coming. And, no, I am not referring to the company Novosco who uses Cloudstream to describe many of its services (although it is related – free plug, guys).  

An odd thing is that the companies who provide the brokerage enabling technologies to do integration, governance and security at the appliance level often have lots of trouble differentiating their products and messages from one another. Go check out companies like Apigee (formerly Sonoa), Layer 7, Vordel, Intel (Expressway), Mashery, and IBM Data Power (If I left your company off the list, don’t scream, just call me and say – “I’m in!”).

What you will find is a dizzying array of terminology for basically the same things. They talk about SOA gateways and XML appliances and providing security or management for SOA and now for the cloud. The products are delivered as on-premises appliances, software, or even cloud services. We at Gartner even cover these products on our SOA governance technologies magic quadrant (now being updated for the cloud too).

The reason for all these different terms is that the customers these companies serve all talk about their problems in different ways, even though they mostly face the same issues. These customers want to talk from internal systems to external services (SOA or cloud) and they want to manage, secure, integrate, and generally enhance the access they get to those services. The problem is, they all talk at the purely technical level when specifying their need; and, the vendors come back with similar technical language to address those needs. One company might say they need application programming interface (API) management; another needs XML acceleration, service governance, interface tracking and versioning, a SOA appliance, caching, or policies for Kerberos authentication. All these needs are valid and most are provided from multiple vendors. So, unless you get down into the details, it is hard to understand why you would choose one of these vendors over another.

So, here comes Cloudstreams. The cloud has made the need for integrating between services (someone told me, “if you’re over 30 you call it an ‘API’, and if you are under 30 you call it a ‘service’”) more evident than ever. Companies want to connect from on-premises apps to cloud services and from cloud services to cloud services. And, all of these connections need to be secure and governed for performance. In short, what they want are flexible, well-defined, integrations of services at the API level using policies to orchestrate the data, messages, and invocations associated with those services. That is a Cloudstream.

Well, it’s actually more than that. A Cloudstream is a packaged integration template that provides a description of everything necessary to govern, secure, and manage the interaction between two services at the API level. It requires an appliance (Humor me. Call it a cloud broker appliance) to act as a gateway between services that is delivered in hardware, software, or managed (cloud) service form. Cloudstreams can be opened and maintained by XML appliances, SOA appliances or gateways, or any intermediary technology that can broker cloud and SOA services.

Now, you might be asking yourself, if we have all those names, why do we need a new term at all? The answer is simple. Complexity kills. Simplicity is the order of the day for cloud computing.

What I mean is that its time to up-level the discussion of integrating APIs between services (SOA or cloud). Instead of talking about needing to authenticate between this API and that one, let’s talk about opening and closing streams of information flowing to and from SaaS applications. Let’s talk about opening message streams from our organization to the cloud and back. Let’s talk about self-service configurations of Cloudstreams  in a simple tool that lets people set up what the data is, what the policies are, and what the key metrics are for performance and security. This is the only way we will ever get to a consistent way to describe the interactions between cloud services. This is about interoperability for the rest of us, not just the engineering geniuses in the basement.

And in case you think I made all this up in my basement, which is actually a theater, I ask you to look at a company like Layer 7 who offers all the necessary tools for providing these templates minus the self-service UI. In case you think I am being naïve, take a look at Mashery and then tell me that anything they do cannot be codified in one configuration file with relatively high level assertions and policies guiding the entire integration interaction. And if that is not enough for you or you think it’s too simple, go talk to GXS, who has created a trading network of over 33000 reusable templates for doing exactly this kind of thing in B2B scenarios and complex business solutions (not all cloud). Cloudstreams are not only feasible; they are only a marketing message and a configuration tool away for some vendors.  

I’ve spent a lot of time looking at technologies for governing interactions between services. From SOA to the cloud, this issue always seems to be at the top of the minds of people interested in using APIs to communicate between systems. It will continue to be so. Cloudstreams offers a unifying direction for all those efforts and will be used in my research about cloud services brokerages if it begins to gain more traction. As we work out the next market forecasting exercises, the next round of new Magic Quadrants, and the evaluations of brokerage enabling technologies for the cloud, you will see this term pop up over and over again.

The idea of packaging up cloud integrations (CIs), as Cloudstreams, is one which opens the gates (pun intended) to many streams of opportunities for the vendors who do this type of intermediation. As long as they remember that the cloud (and by association, SOA) should be about abstracting away from the technical details of how you interact with services and providing a way to think of those interactions as part of the business use of a solution, there will be a place for Cloudstreams. Let’s focus on the resulting integration that is part of the solution, not just on the appliance or the specific technology assertions that have to be upheld. The IT organization can focus on the details of authentication, caching, single sign on, identity, and a host of other fun fiddly-bits while still delivering a part of the solution that business users need. Remember when we got integration adapters and ODBC database drivers? Cloudstreams are packaged integrating processes that can be pulled off the shelf and modified as needed.

Imagine one day when your business comes to you and says “hey, we just bought a SaaS app and didn’t tell you! Now we want you to get data out of it and back into our internal ERP!” Your response could easily be, “oh, we have a Cloudstream for that SaaS app!” or, “Well, here is our menu of Cloudstreams. Just match your app to the right Cloudstream and you’re good to go!”

Cloudstreams. Remember it. The concept will live. It won’t hurt if the name catches on.