Cameron Haight

Earlier this week, Amazon’s CTO Werner Vogels announced the availability of spot- (or market-) based Amazon EC2 instance pricing.  In essence, a customer would bid a price and as long as the bid exceeds the spot price (and they only pay the spot price), their instances would run.  Once the spot price exceeded the bid price, the instances would terminate thus necessitating that the applications running in the Amazon cloud be able to frequently save their state.  A spot pricing history would be available via the Amazon AWS management console to presumably help with bidding.  This is something that is close to what I predicted in an earlier blog post where I discussed how cloud providers might deal with excess capacity.  Is the theorized future’s exchange now not that far off into the future? 

But this leads us to another area of discussion – how will IT operations be able to leverage this?  We’re still dealing with a great deal of uncertainty about how much services cost to run and support within the current enterprise infrastructure.  Now we have a mechanism to potentially set our price, but upon what basis will we know if our bid is actually “profitable” for us or not?  As I speculated in my earlier post, I see an enhanced role for information gathering and analytical tools to help IT determine optimal pricing in the cloud world.  Is an operations-centric “Bloomberg” terminal providing cost and other relevant cloud service provider information (such as QoS) in our collective futures?  

Yes – Aperations, not apparitions as Google tried to correct me when I checked to see if the term had been used previously although this blog post may be a bit spooky to some (had to thrown in the seasonal tie – sorry).  I’ve been mulling this concept for awhile and the thinking begins along the following line – we all know the story about applications getting more and more complex due to the growing number of interdependencies as well as the increasing degree of loose coupling and dynamic binding found in newer application architectures.  Yet we persist with an organizational alignment that in my mind doesn’t act to counter these attributes, i.e., the familiar pattern of development (and application support), test (and staging) and then of course, operations.

Years ago when I covered application management and saw the rise of J2EE-based applications, I was initially surprised that my client inquiries for tools and best practices with regards to Java management were coming from development and application support teams.  It was déjà vu all over again with SOA. Today, of course, we have REST, EDA (Event-driven Architecture), CoDA (Context Delivery Architecture) and other application paradigms that just seem to add fuel to the proverbial management challenge fire.  And why were application developers calling and not traditional IT operations?  Well, the latter weren’t sufficiently skilled in terms of support at the time, and this can still be true today – especially for these emerging application types. 

So the thought is this … quit fighting natural selection.  Developers (or application owners) know more about the application than others within IT so they should be the rightful management owners as well.  I’m not talking about traditional level 3 style activity, but ALL management.  I know the argument against this which is that an expensive resource like a developer should focus on their core competency and only be engaged with problems at the last possible moment, but I ask you the reader, how successful has this posture been?  We throw applications over the wall to operations and somehow we hope that those less knowledgeable about the architecture will sometime in the future be able to management it effectively.  This is not a criticism of IT operations capabilities because they are already extremely busy having to deal with substantial infrastructure change in the form of virtualization, cloud computing, etc.

And speaking of cloud computing, for some cloud and Internet services providers this wouldn’t seem so unusual as some already require heavy instrumentation of their applications and operate under the mantra of “you built it, you manage it  (see On Designing and Deploying Internet-Scale Services).”  They (as does Gartner) believe also that most operations-related issues have their origin in pre-production (note: the research paper by James Hamilton has many other valuable lessons in it that I’ll return to periodically).  For me, the question isn’t so much should application development assume management ownership, but “how low should they go?”  The recent acquisition of SpringSource by VMware I think will further blur the line between infrastructure and applications adding even more impetus to a re-evaluation of how we’re currently structured within IT. So, let the debate begin.

…. as I was saying (back in March) … okay, sorry, I’ve been less than a bit regular on posting, but once more I go into the breech.  Why the play on words?  One of the things that I have focused on in this blog as well as in my career at Gartner is innovation (or the lack thereof) in the operations management realm.  So when something appears at least somewhat unconventional, I’d like to point it out. The folks at GroundWork Open Source are going to include what they call a “Seurat View” in version 6.0 of their GroundWork Monitor (see here).  My wife was an art major, so I happened to know that Georges-Pierre Seurat was a French neo-impressionist artist credited with a painting technique known as pointillism (think painting with dots).  It takes advantage of how we see and process images to essentially enrich the picture (that part I didn’t remember – thank you Wikipedia).  Now, what GroundWork has done is not Seurat’s technique per se, but they’re onto clearly one of the problems in today’s increasingly virtual environments and that is how can we display information in a meaningful way for a large number of managed objects?  Traditional tree- and topology-style views just don’t seem appropriate for a dynamic, virtual environment, let along one that may soon expand into the cloud world.  Another initiative that looks interesting is PixlCloud. They’re building a SaaS-based visualization service.  Could a future management tool take advantage of something like this?  I don’t know the founder, Raffael Marty, but I have pointed out before some of the visualization techniques that he’s been involved with in terms of security visualization. If anyone else has any pointers to interesting visualization techniques that might be appropriate for the management space, please feel free to let me know.  

I guess we will see.  Mantissa Corporation announced last week their intention to provide support for Windows desktops on IBM mainframes.  This is, of course, does not require a total stretch of the imagination as Linux has run on the mainframe for years.  But this is, I believe, a first for Windows, and especially, Windows-based desktops.  Little is known of how this will be done … and the potential impact this will have on mainframe resources. 

[Danger: I'm about to show my age] I remember years ago when I was at IBM trying to enable a port of the DEC VT-100 protocol to an IBM 3090 class mainframe for one of my clients (we were trying to dislodge those pesky, from our perspective, DEC/SNA gateways).  Can you say performance problem?  We did the right thing and ultimately canceled the project but I’m sure some of the sales folks looked wistfully upon the lost mainframe cycles that might have been sold.  I share this though as an example that there are certain environments that don’t blend well with mainframe architectures.  

But for the moment, let’s assume the overhead is manageable and that there are no issues with remote protocols, etc.  Is the mainframe the right place to do this?  While still expensive in nominal terms, the latest z10 IBM server and associated specialty engines (IFL, zAAP and zIIP) seem to be changing many traditional notions about mainframe costs (and capabilities).  I seem to recall studies suggesting that power consumption may also be increasingly a winning argument for mainframes.  And of course when it comes to security, the mainframe is often the bar that other systems alternatives aim for.

So, there may be a plausible case for this – at least among existing mainframe customers.  Still I am concerned about one thing that in my traditional virtualization coverage I heavily focus upon – support (and related to this, complexity).  It’s no secret that skills availability for mainframe support is starting to become an issue as many of these traditional administrators near retirement age.  And of course, even with skilled operators the mainframe can be complex to manage – although managing a distributed UNIX or Windows server farm isn’t necessarily for the faint of heart either.  All-in-all, it’s an interesting announcement and we’ll have to continue to monitor Mantissa’s progress. 

Good comments by my colleague Tom Bittman on the recent Citrix and Microsoft news.  What’s interesting to me though is the now strategic emphasis on management technology by Citrix.  And of course Microsoft continues to invest in System Center and VMware in vCenter and other management add-ons.  As hypervisor platform commoditization grows, where will these vendors head?  Certainly one arc is management.  But the last time I checked there were some already (very) large players in this space.  Admittedly the activities of the “Big Four” (BMC, CA, HP and IBM) in the virtualization arena are still works-in-progress, but they (and other management suppliers) also see the opportunity. The virtualization providers and traditional management firms of course are all trying to find ways to work together, but how long will it be before everyone wants to own much of the same turf?  Like automation and workflow orchestration?  Or capacity planning (what we are now calling ITRP or IT Resource Planning at Gartner)? What about provisioning? There are multiple levels of these functions so large scale direct conflict is still likely in the future. But then we have to ask is this likely to repeat itself all over again in the cloud computing world? The potentially good news for management consumers is that as a result we may see not only lower costs, but maybe, just maybe, more innovation as a consequence of heightened competition.  

From WWW to WWC?  I first came across this term here in Reuven Cohen’s Cloud Computing Interoperability Forum and its proposed Unified Cloud Interface or UCI (I’ll focus more on the issues of standardization in a subsequent post because it’s of importance to my management focus). Cohen used this term in relation to SOASTA’s CloudTest offering, and specifically I believe, to this posting by SOASTA’s Tom Lounibos describing what they call “global load and performance tests.”  It’s an interesting capability that bears its own further review, but again I’m drawn to the term World Wide Cloud and it’s potential ramifications.  In today’s WWW-based world, I can use URLs to seamlessly consume content across multiple web providers.  The term World Wide Cloud, to me at least, presages something similar but this time the “content” or consumed resources are obviously not traditional objects, but compute and storage resources.  This gets a little bit closer to that concept using it appears the OpenNebula virtualization platform. Now the question is – how do you manage all of this?  Still, as Spock would say, fascinating.

As we’ve observed from the major virtualization platform providers, we are now starting to see more cost and TCO calculators for cloud computing.  Amazon has had one for awhile.  Here’s another that compares Amazon to an on premises solution.  This tool appears to be a generic SaaS versus on premise evaluator.  Rackspace/Mosso seems to have developed a calculator comparing their Cloud Files enabled with Limelight’s CDN with Amazon’s S3 and CloudFront.  I’m sure more of these will arise.  Key is of course to a) fully know of any underlying assumptions and b) ensure that in any comparative analysis, an apples-to-apples approach is being used.  I’ll be on the hunt for more of these but feel free to let me know if you stumble across any others.  

I just came across an interesting paper on a VMware project called PARDA.  The acronym stands for Proportional Allocation of Resources for Distributed Storage Access and represents an attempt to potentially deal with increasing virtualization I/O concerns using a "proportional-share resource scheduler that can provide service differentiation for I/O like VMware already provides for CPU and Memory."  Interesting charts that show that the target is to address latency and not necessarily bandwidth.  While many clients that I speak to suggest that they focus most on memory in terms of costs, they also cite that from a performance standpoint they often concentrate primarily on storage I/O so this could very interesting.  It does though seem to potentially create an interesting scenario for virtualization consumers, i.e.,., do I use a common VM-oriented I/O scheduler that is seemingly independent of the back-end storage, or do I use array-based service controlling mechanisms that may be independent of the underlying VM technology platform? 

Thought I’d pass along an interesting link.  Microsoft has been doing some work on analyzing IT operations costs.  They have a URL www.spotlightoncost.com where you can access a study (note: requires registration) on operations best practices (and their cost impacts) against a set of six server types (i.e., email, print servers, etc.).  While the cost savings of specific operational processes may be open to debate, it does provide interesting insight into what practices are being adopted and benchmarks for the number of servers per FTE.  With respect to the process or practice adoption, the results seem to somewhat mirror Gartner’s view that as an industry we still have a ways to go in terms of operations maturity.  And from a virtualization perspective, that distance may be even longer. 

Sorry for the lapse in blogging … no excuse other than to promise to do better from now on out.  I thought that I’d start out by telling those that care what I’m focusing on this year.  I’m still covering virtualization management … in fact, this is one of the reasons for my blogging "vacation" as the market, in contrast to perhaps much of what we read regarding the overall environment still seems vibrant and is keeping me very busy.  There is a large installed base of server virtualization technology among which there is still only relatively limited penetration of management tools – but I expect this to change especially as many end users of virtualization see management tooling (and processes) as a means to both improve control and reduce costs (the latter obviously important these days).  I’m continuing to look at how virtualization itself impacts the cost equation – I’ve already written some research to date and more is in progress.  Cloud computing is of course the new, new thing but it seems to be something that is perhaps gaining steam as well at least partially due to the potential economics involved.  I’m undertaking a review of how this emerging environment will impact both enterprise IT operations (because it will even if you think it won’t) as well as cataloging the new management entrants.  Of course the major management players also are positioning themselves to play a role and I’ll be including them in my research too.  So, all in all, its been a busy first month or so … and for now anyway, promises to be an interesting year.  If there is anything related to these issues that you’d like to see more comments on, please feel free to let me know.