October 18-21 finds me in Orlando, Florida for Gartner Symposium/ITXpo 2010, where I’m delivering several important sessions:

• Monday, 10/18 – Gartner Keynote: New Realities, Rules, and Opportunities (with various distinguished colleagues)
• Tuesday, 10/19 – Debate: Shared-Hardware vs. Shared-Everything Multitenancy (with Lydia Leong and Ray Valdes)
• Tuesday, 10/19 – Web Application Development in 2015: Shifting Sands
• Tuesday, 10/19 – Vendor Roulette: Predicting the Next M&A Moves (with Ray Valdes and David Mitchell Smith)
• Thursday, 10/21 – Platform as a Service: The Strategic Center of Cloud Computing Architecture (with Yefim Natis)

The following week, October 25-27 finds me in Japan at Gartner Tokyo Symposium, where I get to deliver some very interesting, broad talks on cloud and emerging technologies:

• Monday, 10/25 – Top 10 Strategic Technology Trends for 2011
• Tuesday, 10/26 – Emerging Trends and Technologies: Now that You Have the Bandwidth
• Wednesday, 10/27 – The Cloud Computing Scenario

After some much needed R&R I will be landing for Mashery’s the Business of APIs conference in downtown San Francisco. Just a little over a week later, I am chairing the Application Development track at Gartner’s Application Architecture, Development & Integration conference in Los Angeles November 15-17, where again I get to meet with the folks and present some interesting pitches:

• Monday, 11/15 – End-User Roundtable: Best Practices in Cutting Development Cost
• Monday, 11/15 – Rapid Fire: If We Build Cloud APIs, Do We (Still) Need Cloud Integration (with Benoit Lheureux)
• Tuesday, 11/16 – Rapid Fire: Apptrepreneurship: Your Next Great Software Delivery Model?
• Wednesday, 11/17 – Web & Cloud Development 2015: Prepare for Shifting Sands

That rounds out my Gartner conference travel schedule and while I’m excited about the great material and big audiences I expect to see, I am a little bummed that I’ll be forced to miss some of the premier AD conferences of the season, including Adobe MAX and Microsoft PDC. Fortunately my consolation prize is attendance at DreamForce in early December, which should be an interesting opportunity to see what’s up with VMforce, among other things!

I hope to see some of you at these events!

As a CEAP, vFabric provides a variety of vetted components – Spring for AD, GemFire for XTP, Rabbit for messaging, etc. Of course the underlying lingua franca is Java, a language ubiquitous in the enterprise. Interested cloud platform providers, such as carriers – will no doubt take a close look at the vFabric offering as a basis for PaaS competitiveness.

The other story is one that I’ve been following for a while – the potential shakeout in PaaS. I pointed out last year that the SpringSource acquisition put VMware in position to dictate some terms in the cloud application platform space. Not everyone believed, but here we are with VMware moving up the stack (albeit not with CloudFoundry, but hey, nobody’s perfect).

The next question – how far can VMware go in its “encirclement of Microsoft” strategy? VMforce and Google App Engine for Business are a great start, but the PaaS shakeout is just getting started. Smaller providers must catch Maritz mania for the VMware strategy to be fully realized.

It will be interesting to see how things play out. For now I plan to enjoy the next couple of days of VMworld and report back.

Note to Microsoft: Eclipse is an IDE not a language.

Etsy, a Web proprietor of handmade goods, didn’t have a public API. That didn’t stop the “Etsy Underground” from hand-rolling one.

Etsy did have an API for internal use only. While it wasn’t promoted, it also wasn’t well-hidden from the public. Some enterprising developers figured that out, and reverse-engineered the API for their own nefarious purposes. This resulted in third party applications, code libraries, and widgets all leveraging Etsy’s site and content.

By the time Etsy got wind of what was happening in a new developer ecosystem, it was too late to steer the direction of the community directly. They could either shut off the community’s ability to innovate, or add fuel to the fire. The company chose the latter, and launched an open API which replicated all of the features the Underground had come to know and love, and took it further.

The result? According to Etsy, the API now gets 9 million invocations a month, and has spawned 17 applications, including 5 for the iPhone. One site in particular, craftcult.com, is a leading driver of traffic to Etsy’s site.

Handmade goods are interesting, but what about your business? Chances are, your data is already being scraped by somebody; maybe you can make it easier for them, and in so doing foster a community that spreads your brand.

The urge to control access is great, but what if you can become a bigger broker in the conversation by sharing instead? Clearly this won’t work for all businesses – sometimes the content is just too valuable to share, and loses its value to the enterprise once it spreads.

5.7 million API calls yesterday. The 19th to yesterday (5 days) accounts for 61% of our total so far this month.

The day before, @bestbuyremix reported 5.3 million API calls.

I spent half a day last week at the Business of APIs Conference, where I had the opportunity to meet with folks from Best Buy, Hoover’s, Etsy, CBS, the New York Times, and more who are involved in open API programs.

It is pretty clear to me that APIs are going to become an invaluable – and common – tool in the enterprise’s public Web presence toolbox. When we finally hit “critical mass” – more retailers with public WOA APIs than without – will API traffic become a new barometer for predicting Christmas shopping economic outcomes?

UPDATE – Proof that an API community must be closely measured and managed. Your key partners might need help at a critical moment, as according to @bestbuyremix:

We had to increase the API rate limit for @miloshopping because of all of their press so far today! NYT article: http://bit.ly/073VfE9

Happy Turkey Day Everyone!

This packaging looks, well, cheap. When I first saw it, I was reminded of the classic “private label” beverage of choice in many 80s TV shows:

The person who brought up the “new look” of Wal-Mart private label goods found this cheap look unappealing, and believed that consumers would not respond to it.

Savers First

Without going into too much detail, I’ll start by saying that I believe that consumers are retrenching, and that the mentality of “conspicuous consumption” is gone for at least a generation. Too many kids and young adults in this country and others have seen their parents get wiped out by the economic downturn. They will become savers first.

Many white label products are just as good as their name-brand counterparts. In fact, many are identical, rolling off of the same factory floor. I admit I’m no expert but I’m unable to tell a difference between Great Value pasta and more expensive brands. The same is true for many other commodity items. I buy on price first.

I Beg to Differ(entiate)

At a high level, every company must decide on one of two “generic strategies.” Either be the cheapest (“cost leadership”) or provide more perceived value to the customer (“differentiation”). The customer’s perception of value can be influenced with more features, marketing, convenience, etc. The idea is that these value drivers will provide more of a reason for customers to pay a higher price.

Wal-Mart doesn’t make any bones about it – it is a cost leader. I don’t shop at Wal-Mart because I love the piped-in audio or the ambient lighting – I shop there because it’s CHEAP, plain and simple. I don’t care what the packaging looks like. More to the point, if the packaging looks cheap, it accentuates Wal-Mart’s cost leadership position. In fact, it starts to make cost leadership itself into a potential differentiator.

Imagine this scenario: Wal-Mart could charge $1 for a box of Ziti, which sits next to other brands that cost $1.20 or more. In the past, Wal-Mart’s Great Value packaging looked more in line with that of other brands (it wasn’t cheap-looking). Now Wal-Mart’s packaging looks cheap. It is designed to make the product look even cheaper than it is. Wal-Mart could raise its prices by a nickel a box, and customers might not even notice.

For a more in-depth discussion of generic strategies, competitive advantage, and corporate strategy I strongly recommend Gordon Walker’s book. It was perfect for a strategy novice like me, and it is a regular reference on my shelf.

How does this relate to Technology?

IT product and service companies have the same strategic choice to make – cost leadership or differentiation?

Most companies choose the latter, because differentiation offers more room for innovation – marketing campaigns, cool new features and delivery models, smarter consultants, etc. Cost leadership is about relentlessly pushing down fixed and marginal costs to provide the cheapest alternative. It’s really hard and you’re never safe – there’s always someone else who’s willing to do it a little cheaper if they can figure out how.

In our recent special reports on Cloud Application Platforms, we noted differentiators again and again – but rarely suggested considering a vendor based on price. It is not in the interest of vendors to compete on price, and enterprise CIOs would do well to remember it.

I think most decent developers do this, and some try to stretch by learning brand new unrelated tools (even easier if you’re a consultant moving from project to project). IMHO if you bother reading developer blogs at all you’re already several cuts above the average developer.

A Peek Behind the Curtain

At Gartner we like to slice and dice everything and everyone into patterns, quadrants, matrices, etc. I’m not exposing any state secrets when I say that our business (at a very simplistic level) is divided roughly in half between producers of technology (vendors) and consumers of technology (end users). We get to find out what kind of horrible pain our end users are suffering and advise them on remedies. We also get to find out what vendors are planning to do, and advise them on strategy. Sometimes it feels like cheating.

As you might expect, by and large technology producers are much stronger at nuts and bolts than end users are. After all, for end users technology is a means to an end. For technology producers it is their core competency and competitive differentiator.

The Enterprise Web Developer

So, in my past life I was one of those enterprise Web developer guys who doesn’t really know anything compared to the engineers at Google and Microsoft. On the other hand I was much more in touch with the reality of doing business where technology is an enabler rather than a differentiator. The reality is that the best enterprise developers need to be slightly ahead of the pack technologically, and also good at the business side. You could say that from the perspective of the indivudal developer as an enterprise, use of technology becomes a differentiator for them.

If you want to be a highly-valuable (and hopefully highly-compensated) enterprise Web AD guy, you have to do a few things, probably not limited to the following (which are not in any specific order):

1. Be really good at the technology that you work with at your company.

2. Have a sense of what is coming down the road with technology that will affect your enterprise and work proactively to use it.

3. Be really good at talking in business terms to internal customers who really don’t care that you just built a cool widget using only 3 lines of jQuery. Be equally good at understanding business speak from the end users.

Do those basic things and you will be in a great position to make good bucks and maintain some job security and job selection. Why am I telling you all this?

Enterprise Web AD is Nearing an Inflection Point

The growing importance of the Web is pushing more transactions online. All businesses are under tremendous economic pressure.  The explosion in elastically-priced Cloud application platform technology is creating a new ecosystem for doing business on the Web, one in which costs are lower, time to market is better, and developer productivity is higher.

Five or ten years from now, most Web applications will not be developed for “traditional” app servers like Java EE, .NET, ColdFusion, etc (although possibly these app servers will be re-engineered to become elastic; hard to be sure at this point). Furthermore, the amount of data developers are used to working with in a relational DB is going to change dramatically – you will be working with ginormous data sets and SQL just won’t cut it anymore. Most likely the tools you are working with today will change dramatically. You might use different tools altogether or the ones you’re using might morph into new ones.

I can’t predict which platforms will win and lose. Its too early for that. However if you want to continue to be a highly-valuable, highly-compensated developer I have a few pieces of advice for you (also not in any specific order):

1. Get familiar with non-relational horizontally-scalable databases. (Ex: CouchDB, Google BigTable, Amazon SimpleDB) I am not a database guru and we have guys for that here at Gartner. But I can tell you that this stuff is different enough to be alien to most developers, even the good ones.

2. Get familiar with platforms and languages that support highly parallelizable applications. (Ex: Erlang, Clojure, Hadoop, anything that uses MapReduce internally; maybe even break out the old LISP) At a minimum bust out some recursive programming techniques in the platforms you already work on.

3. Get familiar with composition methodologies. Give up “Not Invented Here” (I know its hard – developers like building things). The best Cloud application platforms – at least for building business applications – will offer high productivity boosts with generically reusable components. If you don’t get on board with composition you won’t be able to wire together applications quickly from reusable pieces, and the crummy cut and paste code monkey sitting next to you will whip you.

In the late 90s, the Internet boom created a lot of new jobs and obviated a lot of old ones. The simplicity of the Web removed the need for as many desktop applications. We’re about to have another such shift. The best enterprise developers stay ahead of the curve. Its time to skill up for Cloud AD.

According to my strategy professor, Gordon Walker, “the trigger point [for shakeout to begin] is when one or more firms achieve a level of productivity that neither weaker rivals nor potential entrants can match. Thus, the shift in entry and exit rates is ultimately caused by successful and sustainable growth strategies. Only the presence of one or more firms whose dynamic capabilities create dominant, defendable market positions can deter entrants and force weak competitors to leave the industry.” (page 137)

Shake it on, Baby

Now, I can tell you, that to be sure, there is no shortage of entrants into the cloud application platform industry. Force.com may have started things off a few years ago, but today there are at least 40 companies, and probably more than that, who play in the space. An upcoming Gartner report profiles many of these companies. I don’t think we’re in the shakeout period yet; however, I put better-than-even money on the idea that we’re in the last stages of expansion before a shakeout gets started. I submit as evidence the following:

1. CogHead, a leading (but small) technology company in the Cloud application platforms space, went bust last December. Its assets were ingested by the large, enterprise-flavored SAP. I am not sure what SAP plans to do with those assets, but I don’t believe they bought CogHead’s stuff to put it in a museum.

2. Microsoft Azure (and .NET Services), which is kind of a hybrid approach somewhere between Amazon EC2 (cloud system infrastructure) and Google App Engine (APaaS), will hit general availability sometime soon. This will mark the first big-time enterprise software player to have their own Cloud application platform offering.

3. VMWare has jumped into the fray with its planned acquisition of SpringSource – whose CloudFoundry offering constitutes a Cloud application platform has to be the main reason the company shelled out over 400 million, for what is a tiny open-source company (admittedly, Spring, around which the core business of SpringSource is built, is a great and widely-adopted framework for Java programmers). It will be interesting to see how VMWare plans to incorporate the SpringSource and Hyperic assets into its portfolio, and what (if any) other complementary acquisitions they will make. I’m not sure what to expect, but if I were a small Cloud application platform startup counting on a neutral VMWare container within which to run my “shared-hardware” multi-tenant platform, I’d be a little bit more worried than I was a few weeks ago.

Twist and Shout

The future for APaaS and other types of Cloud application platforms looks very bright to me. I’m particularly bullish on APaaS because of the productivity benefits available to developers using a combination of development of new code and composition of business-oriented services and components offered by the provider. I don’t think there’s a big enough market for all of the companies producing Cloud application platform software to wind up as “winners”, though. I also think that the time is drawing close where the other “mega-vendors” – IBM, Oracle, and SAP – must either launch their own Cloud application platforms or acquire one, or risk being left behind. Keep your eyes open for acceleration in the demise of Cloud application platform companies, and deceleration among new entrants.

I’ve been so busy with writing deadlines that for the last month I’ve had no time to update this blog. I hate to write things that aren’t “meaningful” to me, and I’m not able to generate meaningful blog posts with the same frequency of some of my peers. So, I ask that you bear with me as I try to get back into the swing of things.

Upcoming Speaking Engagements

I’ll be delivering a pair of presentations at Symposium this year – a great honor for a new, wet-behind-the-ears Gartner analyst. The first, “Citizen Developers: When End-Users Surpass the IT Department” is a Maverick (read: controversial within Gartner) pitch about the waves of end-user development we’ve seen over the years and the big one that is on the horizon at the nexus of Cloud Computing, Enterprise Mash-ups, Digital Natives, and Modern Development Tools. The second, “Buckle Your Seatbelts: Web and Cloud AD Convergence Ahead” speaks to the coming enterprise shift toward Web AD in the Cloud, and makes a few predictions. I’ll also be participating in a debate with Tom Bittman, called “Private vs. Public Cloud Computing: Does Control Trump Scale?”, in which we’ll explore the implications of private, public, and hybrid approaches to enterprise use of cloud computing. I’d be lying if I said I wasn’t a little terrified about speaking at Symposium, but I’m also excited to have this great opportunity.

Speaking of … speaking, tomorrow is a local analyst briefing in Dallas, where I’ll be giving a couple of 20-minute “rapid fire” sessions. One is on Enterprise RIA, and the other is a cut-down version of my upcoming Symposium pitch on Web & Cloud AD convergence. Should be interesting and it gives me an opportunity to work in some audience feedback before my final writing deadline for the Symposium presentation.

There Is No Easy Button

Before I joined Gartner, I worked as a developer, architect, business analyst, team leader, and sometimes jack-of-all trades IT guy (for example I set up a PCBoard BBS in 1999 to provide a way for insurance agents without internet access to upload quotes via modem; I was the only guy who still knew how to configure and run a BBS), etc. at various points in my career. The mantra “there is no silver bullet” was commonly repeated – sometimes in exasperation – in all of these roles.

Good, Fast, Cheap – Pick Two

I don’t believe that we’re suddenly going to find an easy button – in the abstract. Someone, somewhere, has to do heavy lifting in any software development endeavor to build a high-quality, high-availability, highly-reusable Web architecture. Developing these things is a complex process, with a lot of moving parts. However, we know from experience that tools make problems easier to solve. For example, imaging trying to build a modern Web application using just CGI, C++, and sweat, and compare that to what you get out of the box with .NET, ColdFusion, Rails, etc. I realize each of these tools has various advantages and disadvantages, but they are marginal differences compared to where we were ten years ago.

Still, software development costs and IT costs in general keep growing faster than the rate of inflation. For example, Gartner predicts that in 2013, organizations will drop almost $4b on IT, compared to just over $3b in 2007. Yet, some folks think that IT doesn’t matter. So why do we spend more and more on it?

IT keeps getting more complicated, requiring more investment just to keep up with competitors. Companies able to manage the complexity with strategies such as SOA can leverage their underlying investments to generate additional value, sometimes even creating brand new product offerings, repurposing existing systems as disruptive innovation.

In a perfectly architected software world, IT wouldn’t matter, because all competitors would have similar capabilities and success would come down to other parts of the corporate strategy. “Best practices” – which by definition proscribe noncompetitive processes, which are the same as everybody else’s – would dominate. How many organizations are successful in implementing best practices? How many even agree on what these are? I submit that IT does matter, and that companies who learn how to screw down the cost drivers while simultaneously enhancing value drivers to satisfy customer needs have a competitive advantage in their industries.

Pay It Forward

So, how does this all add up to the Cloud saving the world? My (admittedly clumsy) interpretation of Tainter is that as the world grows more complex, the only chance we have to head off the disintegration of modern society under the weight of complexity comes through technological leaps in the form of disruptive innovation. The hype around the Cloud provides some justification for the idea that it is disruptive. Yefim Natis and I (mostly Yefim) developed a research note in June that describes what we see as the Killer App – Application Platform-as-a-Service (APaaS) – on the horizon that will result in accelerated disruption.

APaaS simplifies the development of Web applications and the management of infrastructure, abstracting the complexity behind the walls of a provider, such as Google (App Engine), Salesforce (Force.com), Microsoft (upcoming Windows Azure .NET Services). The adoption of APaaS by a wide range of enterprises will make parts of IT matter less to most enterprises (although it will be much more important for the APaaS providers themselves), and it will enable enterprises to focus on what truly differentiates them – business processes. APaaS isn’t an easy button, but a leap in Web development tools that stashes “best practices” behind the facade of a service provider, and it will allow enterprises to save enormous amounts of money in the long run, money that will be paid forward to shareholders and then reinvested in other areas of the economy.

Could this new level of simplicity in complexity be the disruptive innovation that saves the world – or at least gives us a bit more time?

I just want you to know, that I do feel guilt for my lack of recent blog posts. I try to commit enough time to my published writing, client conversations, and informal activities (such as this blog), and sometimes I fall down on the job. There’s no excuse.

You Start Me Up

I’m wrapping up a weekend in Seattle. I’ve had a great week meeting with a variety of Gartner clients; in addition I’ve spent time digesting the spectrum of technology news releases and opinions published over the last several weeks. It’s a big spectrum and I’m wont to indigestion.

So, I’m a Gen-X’er. I’ve spent some fair amount of my adult life spewing bile on all things Microsoft. It’s kind of a badge of honor for my generation; after all, we saw the movie Anti-Trust, and appreciate Slashdot’s Borg Gates and all that. </sarcasm>

Anyway, right or wrong, there’s a lot of years and perspective between Win95 and Windows 7. I’m not here specifically to talk about Windows 7 (nor am I qualified to do so), by the way. But, I want to recognize a reality that I believe is encroaching on all of us today..

We’ve (the royal “We”) spent a long time in apology or attack of Microsoft technologies. These technologies succeed in spite of our complaints.

In the mid-nineties, I was part of a generation that witnessed incredible excitement and interest in Microsoft technologies. At the time, MS was a “relatively small to medium sized” enterprise with around 17,000 employees.

Where Do You Want To Go Today?

Today, Microsoft is a Goliath. Big, transnational, transcendending technological boundaries in a range of disciplines, from Azure to Windows 7. (Sorry I couldn’t come up with a credible Z)

Many would see this Goliath as a ripe target for disruption. I can sympathize with this view; the big kid on the block never sees the brick aimed at the base of his neck.

On the other hand, Microsoft has been taking crap from people for just about my entire adult life. What makes people think that dealing with crap isn’t a skill? Why do you believe that Microsoft can’t get smart?

In the mid-nineties, none of us Gen-X’ers was trained to hate Bill Gates and the Microsoft; but we learned. A new generation of technologically apolitical youngsters is about to graduate college looking for work and inspiration. Why do you believe Microsoft won’t capture their dreams?

What are the odds we’re about to see a reinvigoration of interest in Microsoft? Is the renaissance at hand? Discuss amongst yourselves.