Blog post

Cloudstreams: The Next Cloud Integration Challenge

By Daryl Plummer | November 08, 2010 | 6 Comments

Windows 7VirtualizationVendor ContractsStrategic PlanningSocial SitesSocial NetworkingSocial ComputingSOA GovernanceSOAService OrientationIT GovernanceIAMGoogleERPEmerging TrendsEmerging PhenomenaEACloud

I’m about to introduce a term to describe a solid trend in cloud computing integration – Cloudstreams.

It will come as no surprise to those of you who have spoken to me that this trend lays in the domain of cloud services brokerage. I have said, and will continue to say, that cloud services brokerage represents the single biggest revenue growth opportunity in cloud computing and that it is built on markets totaling near one trillion dollars in IT spend. That’s all well and good.  But, when we get down to the details, we find that the biggest impact for actual customers will be found in a combination of four categories of brokerage – integration, customization, governance (including security), and aggregation.

Cloudstreams focuses on the integration, governance, and security impact points. Trust me, the definition is coming. And, no, I am not referring to the company Novosco who uses Cloudstream to describe many of its services (although it is related – free plug, guys).  

An odd thing is that the companies who provide the brokerage enabling technologies to do integration, governance and security at the appliance level often have lots of trouble differentiating their products and messages from one another. Go check out companies like Apigee (formerly Sonoa), Layer 7, Vordel, Intel (Expressway), Mashery, and IBM Data Power (If I left your company off the list, don’t scream, just call me and say – “I’m in!”).

What you will find is a dizzying array of terminology for basically the same things. They talk about SOA gateways and XML appliances and providing security or management for SOA and now for the cloud. The products are delivered as on-premises appliances, software, or even cloud services. We at Gartner even cover these products on our SOA governance technologies magic quadrant (now being updated for the cloud too).

The reason for all these different terms is that the customers these companies serve all talk about their problems in different ways, even though they mostly face the same issues. These customers want to talk from internal systems to external services (SOA or cloud) and they want to manage, secure, integrate, and generally enhance the access they get to those services. The problem is, they all talk at the purely technical level when specifying their need; and, the vendors come back with similar technical language to address those needs. One company might say they need application programming interface (API) management; another needs XML acceleration, service governance, interface tracking and versioning, a SOA appliance, caching, or policies for Kerberos authentication. All these needs are valid and most are provided from multiple vendors. So, unless you get down into the details, it is hard to understand why you would choose one of these vendors over another.

So, here comes Cloudstreams. The cloud has made the need for integrating between services (someone told me, “if you’re over 30 you call it an ‘API’, and if you are under 30 you call it a ‘service’”) more evident than ever. Companies want to connect from on-premises apps to cloud services and from cloud services to cloud services. And, all of these connections need to be secure and governed for performance. In short, what they want are flexible, well-defined, integrations of services at the API level using policies to orchestrate the data, messages, and invocations associated with those services. That is a Cloudstream.

Well, it’s actually more than that. A Cloudstream is a packaged integration template that provides a description of everything necessary to govern, secure, and manage the interaction between two services at the API level. It requires an appliance (Humor me. Call it a cloud broker appliance) to act as a gateway between services that is delivered in hardware, software, or managed (cloud) service form. Cloudstreams can be opened and maintained by XML appliances, SOA appliances or gateways, or any intermediary technology that can broker cloud and SOA services.

Now, you might be asking yourself, if we have all those names, why do we need a new term at all? The answer is simple. Complexity kills. Simplicity is the order of the day for cloud computing.

What I mean is that its time to up-level the discussion of integrating APIs between services (SOA or cloud). Instead of talking about needing to authenticate between this API and that one, let’s talk about opening and closing streams of information flowing to and from SaaS applications. Let’s talk about opening message streams from our organization to the cloud and back. Let’s talk about self-service configurations of Cloudstreams  in a simple tool that lets people set up what the data is, what the policies are, and what the key metrics are for performance and security. This is the only way we will ever get to a consistent way to describe the interactions between cloud services. This is about interoperability for the rest of us, not just the engineering geniuses in the basement.

And in case you think I made all this up in my basement, which is actually a theater, I ask you to look at a company like Layer 7 who offers all the necessary tools for providing these templates minus the self-service UI. In case you think I am being naïve, take a look at Mashery and then tell me that anything they do cannot be codified in one configuration file with relatively high level assertions and policies guiding the entire integration interaction. And if that is not enough for you or you think it’s too simple, go talk to GXS, who has created a trading network of over 33000 reusable templates for doing exactly this kind of thing in B2B scenarios and complex business solutions (not all cloud). Cloudstreams are not only feasible; they are only a marketing message and a configuration tool away for some vendors.  

I’ve spent a lot of time looking at technologies for governing interactions between services. From SOA to the cloud, this issue always seems to be at the top of the minds of people interested in using APIs to communicate between systems. It will continue to be so. Cloudstreams offers a unifying direction for all those efforts and will be used in my research about cloud services brokerages if it begins to gain more traction. As we work out the next market forecasting exercises, the next round of new Magic Quadrants, and the evaluations of brokerage enabling technologies for the cloud, you will see this term pop up over and over again.

The idea of packaging up cloud integrations (CIs), as Cloudstreams, is one which opens the gates (pun intended) to many streams of opportunities for the vendors who do this type of intermediation. As long as they remember that the cloud (and by association, SOA) should be about abstracting away from the technical details of how you interact with services and providing a way to think of those interactions as part of the business use of a solution, there will be a place for Cloudstreams. Let’s focus on the resulting integration that is part of the solution, not just on the appliance or the specific technology assertions that have to be upheld. The IT organization can focus on the details of authentication, caching, single sign on, identity, and a host of other fun fiddly-bits while still delivering a part of the solution that business users need. Remember when we got integration adapters and ODBC database drivers? Cloudstreams are packaged integrating processes that can be pulled off the shelf and modified as needed.

Imagine one day when your business comes to you and says “hey, we just bought a SaaS app and didn’t tell you! Now we want you to get data out of it and back into our internal ERP!” Your response could easily be, “oh, we have a Cloudstream for that SaaS app!” or, “Well, here is our menu of Cloudstreams. Just match your app to the right Cloudstream and you’re good to go!”

Cloudstreams. Remember it. The concept will live. It won’t hurt if the name catches on.

The Gartner Blog Network provides an opportunity for Gartner analysts to test ideas and move research forward. Because the content posted by Gartner analysts on this site does not undergo our standard editorial review, all comments or opinions expressed hereunder are those of the individual contributors and do not represent the views of Gartner, Inc. or its management.

Comments are closed


  • Like it Daryl.

    We have just delivered, to a large financial customer, the capability for an old legacy application to consume the “cloudstream” of Salesforce.

    The marketing team create campaigns in Salesforce for customers and now those campaigns pop-up in real-time when a customer interaction occurs from the legacy (at the users desktop). It would otherwise have been 5+ years before this legacy application could have participated in Cloudstreams (IT resource, time and money). However we gave the legacy app, the API it didn’t have and cloudstream to legacy became an instant reality.

  • dave watson says:

    like it very much, i think you talk “cloud streams” but they are really just activity streams that you will filter, correlate, aggregate and broker from the cloud. the source of the events can be cloud or on-premise and the target could be cloud or on-premise apps. we are doing a lot of work with salesforce chatter implementations that require exactly this model for managing the activity streams of business events. i wrote more about it here:

  • Sounds cool Francis. Now for leverage.

  • Daryl I am not convinced that one needs to have a separate appliance of service gateway or brokerage to deliver improved governance in any metered service integration. – A Blueprint for Cloud Service Metering & Monitoring

    I would prefer to see work done on standardization of contractual service exchanges. Here is a starting point for such discussions.

    Metering in the Cloud: Visualizations Part 1

    Note that we see ABC & Metering providing an universal solution that can be applied at high level service interactions as well as within the various containers and runtimes at the programming language level. This is markable different that proposed solutions above. A bigger vision with a much greater (potential) impact on the software industry.

  • Rajeev Gupta says:


    I am \In\ – checkout

    \Cloudstream\ – a nice word to the new trend. It was good reading the article.

    Having worked with GXS as an archetect, and now built an offering for Integration-as-a-Service –, I can say that I have seen this space evolve in time – both in technology, delivery and branding.

    One thing I have seen is the growing need of pre-built templates for integration. With increased use of standardized webservices/REST, well published \API’s\, its easier to bring in pre-built, quick to launch applications. I feel that we will continue to see \new complete products\ come out which hides the details of integration, and provides end users an experience.

    If you look at social integrations – from tweetfeeds to xobni, all of them do it – its the experience that counts. We have started to do this with our CRM to Accounting and now even have a users purchasing experience around buying templates or processes, and not just offering just an integration platform.

    It would be interesting to see new modes of offering integration for SaaS as products.

    Will certainly continue to follow you blogs.

    Rajeev Gupta
    CEO, DBSync

  • Interesting post Daryl – we definitely see this space growing rapidly and 3scale ( provides a powerful set of management tools for service endpoints (including the self-service UI in fact).

    A service relationship is a business relationship – often a critical business relationship – and needs to be managed effectively. Specific technology requirements are part of that. However, as you point out business relationships are paramount which is why our enterprise tools provide business / key account manager dashboards to manage relationships with partners users of any outward facing service.

    Fair comment that as vendors we’re often guilty of using overly technical language – hopefully this is changing, as the area matures.