Andreas Bitterer

Caution: Long post, but lots to say…

Not often, a vendor reacts with a blog posting to the publication of a Gartner document. Press releases, yes, and a lot, kinda like “Company XYZ announces that a leading analyst firm has placed it in the leaders quadrant of the ABC Magic Quadrant…” Rarely, though, do we see a posting directly referring to a magic quadrant, particularly from a vendor that wasn’t included. (Draw your own conclusions.)

Yves de Montcheuil, VP of marketing at Talend, did just that on his blog, where he commented on the latest data integration magic quadrant. I never met Yves, only talked to him on the phone a few times, and I have likewise a lot of respect for him, just like he wrote he has for the Gartner analysts who authored the Magic Quadrant for Data Integration Tools. In his post, Yves makes a few statements that call for a response, as they indicate a distorted perspective on the authors’ view and some other comments are just plain wrong.

No surprise, Gartner’s analyses are still very conservative

If this means that we are not jumping on every new idea and announce it as the silver bullet, then yes, we are conservative. It does not mean, though, that we are turning a blind eye on new developments. Quite the contrary, in fact, as demonstrated by the large number of tiny and often virtually unknown vendors that are featured as a “Cool Vendor”.

Their analysts use mostly their rearview mirror, to look at what happened behind them, whereas they should have a radar to see what’s happening around them and ahead of them.

This is rather meaningless to me, because it’s rather irrelevant what was “happening behind me“, or is “happening ahead of me“. Of course, we are evaluating each vendor’s past, present, and potential future. As an author of any magic quadrant, what the analyst is looking at, and that is clearly communicated to every vendor that is potentially included, are not only things such as product capablities,  but also revenues, sales, marketing, support, alliances, or their financial position. I’m sure, even Yves would agree that those criteria are more backwards looking, as they describe a track record, which, by definition, is always looking at the past. A “future track record” does not exist, with radar or without. And “future revenues” are fairy tales.

… the Magic Quadrant reflects past adoption of certain technologies by large accounts in the US, who are customers of Gartner.

Wrong on all accounts. The MQ is not an adoption map. It has nothing to do with the size of the company. The MQ does not have a US focus. And finally, surveyed companies do not have to be a Gartner client.

Updated every 18 to 24 months and reflecting the long cycles of traditional vendors, who used to take years before their could achieve a significant position on a market

Wrong again. This MQ is updated every year, and Yves should know that. And the MQ is definitely not reflecting sales or product cycles from anybody.

This quadrant includes a combination of dying technologies which have been acquired over and over again (ETI, Open Text’s Genio…), loading utilities (Syncsort, Pervasive, Sybase’s Solonde…) and real enterprise solutions (Informatica, IBM’s DataStage). One component is missing: open source - of course.

I may be repeating myself here, but for every MQ there is a pre-defined set of inclusion criteria, and every vendor that meets those criteria, will be included. Period. Whether they have been acquired a gazillion times (not sure, why that matters), sell “dying technology” (as Yves puts it, well, it’s his job as marketier to knock the competition) or sell products that are made from clay. It does not matter, as long as they match the criteria. And open source vendors are not “missing, of course”, but because not one of them can meet those set criteria (to date). We are even changing the inclusion criteria to make it possible at all for open source vendors to be evaluated. If we would stick with “minimum license revenue of 20M USD” (I picked an arbitrary figure!) as one of the inclusion criteria, it’s hard to see how any open source data integration tools vendor would make the MQ any time soon, as they typically don’t license the software. Because this wouldn’t be fair to open source vendors, we use a figure for “minimum total revenue” instead (only for open source vendors, that generate revenues mostly from service subscriptions), so they have at least a chance of being included in the MQ. If the vendor does not make those figures, they are still out. Nobody gets included for good will.

Some would say that open source vendors cannot afford to pay Gartner (I personally don’t think it makes a difference). This may be true for some vendors. But in our case, Talend is a commercial vendor with strong resources and could afford a contract with Gartner.

Ah, the old mis-perception. Being included in any MQ (not just for data integration tools) is in no way depending on being a Gartner client. I’m happy to see that you don’t think it makes a difference. Because it doesn’t.

But why? To hear that “open source is immature (probability 0.9) and will become mature in 5 to 20 years (probability 0.8)”?

That is funny. But Yves made that up, of course. Interesting though is, that Yves makes the connection between Talend’s exclusion in the MQ and general open source maturity. I think that’s a bit of a stretch, as there are many technology areas that are considered mature (operating system, web servers, app servers, DBMS, to name a few), but data integration is just not there yet, according to our data from user surveys. Yves himself has provided reference accounts of Talend users for our open source data integration survey.

No thanks. We know, and our clients know, that open source has changed a lot over the past years and has become a true alternative for the enterprise (probability 1.0). Maybe even Gartner will realize this one day (probability 0.2)!

We are not disagreeing at all, that “open source has changed a lot”, but that’s not the point. We are also realizing that open source tools (of many kinds) are used in small, medium and large enterprises. All of these things are definitely taken into consideration when we evaluate open source, as we update the MQ again next year. Equally, we will be looking how the rest of the data integration market has developed, as open source does not exist in a vacuum. Criteria may change, too.  Until then, I hope this clarifies the 2008 version of the data integration MQ, even for Yves. Probability … uh.. never mind. We stopped using those probabilities quite some time ago, anyway.

Of course, there is no point locking the barn door now, since the horse has bolted. I’m talking about those billions of dollars and euros being spent to save banks. In a nutshell, what caused the market meltdown were banks selling products and approving loans that came with an enormous risk, and unfortunately, that risk became reality. Sure, there were lots of people out there that should have never been given a loan, because they didn’t understand what an adjustable rate mortgage (ARM) is. But, of course, bankers pushed those loans down people’s throats to get a commission and generate revenue. And here is where I think the industry is missing something. Who is liable for the dangerous products the banker sold?

If you buy a coffee maker, or a toaster, or a lawn mower, pretty much anything, the manufacturer is liable for any injuries that product may have caused. As far as I can see, this is always focused on “bodily injury”… why? If someone loses the house, car, or job, because the “bank product” just imploded, isn’t that another injury caused by negligence?

“This plastic bag is not a toy.” or “Do not place animals into the microwave.” Duh. Everybody understands that. (At least I hope so). Where is the disclaimer on those “banking products” that hardly anybody (including the bankers themselves) understands. Where does it say “this derivative has a risk score of 89 (out of 100)” on a scale that says “you could lose that money”? Would people be more careful if they knew what they were getting into? I think so. But they just took the banker’s word for it. No data was made available to back up those claims.

Where was the bank’s risk management? Oh right, it was focused on the customer. If there was a product liability for banks, I’m sure the bankers would have been a little more hesitant with those loans. The data was all there, but nobody wanted to look. I’m hoping that the lawmakers in those governments in the US, the EU, Japan, Korea, and others, that are bailing out the financial institutions at the moment, are considering to extend the current liability law into the financial world.

If I’m reading this description from Wikipedia with banking products in mind, that would make a good start.

“The most fundamental rationale for strict liability is to force producers to internalize the external costs they impose on society. By placing liability for all injuries caused by a product on its manufacturer, the manufacturer is forced to take into account, when deciding whether and how much to produce the product, the harm caused by it. If this internalized harm is so great that the manufacturer cannot profit from producing the product, it will discontinue the product, or sell it only at a higher price to consumers who value it especially highly (in economic terms, modify its activity level). In this way, strict liability provides a mechanism for ensuring that the societal good of products in the marketplace outweighs their societal harm.”

Disclaimer: I’m no financial specialist, just an observer.

I must have attended a few dozen BI vendor conferences large and small over the years and sat through many hundred BI briefings. And I have to admit, that my expectations have largely decreased over time. Maybe it’s because the market is considered mature, maybe because I have heard every possible marketing claim, maybe I’ve been around the block a few times. I don’t know. What I do know is that I’m missing something.

Sure, the BI market has enjoyed consistent growth, has seen a lot of action on the M&A front, technology has advanced significantly (I remember when gigabytes were considered wild), and yet we are still discussing same old business intelligence. I keep hearing vendors announce that the next version of their tool will be able to address that untapped market within their customer base, growing penetration beyond those 10-15% that are using BI today. I heard this 5 years ago already, but what has changed since then? Not much.

So the tools have a nicer UI, can scale up and scale out, are interactive, and can be mashed, and so on. And yet, BI licenses are not really flying off the shelves because of all the new cool stuff that is available. So the vendors started buying up and down the stack, into performance management, data integration, master data management, data quality, etc.  Technology advanced towards in-memory analytics, data warehouses grew into petabyte range (not many yet, but still quite the accomplishment), Web 2.0 technology allows to build mashups from Google Earth and heatmaps and alerts and visualize the whole thing on an iPhone. What does it all help if people think what they want are “reports”? Ugh.

Here’s the thing. We have done Integration (or at least we are underway): IBM integrates Cognos, Applix, Ascential, DWL, Unicorn,.. SAP integrates Business Objects, Pilot, Firstlogic, Fuzzy, Outlooksoft, Cartesis,… Microsoft integrates Stratature, Datallegro, Zoomix,… Oracle integrates all the rest, Siebel, PeopleSoft, Hyperion, Sunopsis, BEA.

We have done Innovation. Compared to the old days of “management reporting”, new available technology is to die for, from wildly scalable data warehouse appliances, to predictive modeling and mobile BI, all based on SOA, then delivered through software-as-a-service, or through open source license models.

What’s missing is Inspiration. It does look like potential buyers are caught in the “reporting web”, users seem to be largely oblivious to the current developments, and so the great potential value of BI is simply missed because of a lack of BI exposure. Of course, it doesn’t help if attendees at said conferences are bombarded with tech talk about the next great feature. But nobody enables the potential BI users to think outside the “reporting box” and answer the question “what would happen if BI could do that?”

I, for one, can’t say that I’m truly inspired by the messages from the BI vendors. But that’s just me.

With yesterday’s announcement of the Oracle Exadata Storage Appliance, Oracle essentially became a hardware vendor… well, sort of. With the help of their friends across Highway 101, Oracle leverages HP hardware, apparently servers and storage, Infiniband, its own 11g DBMS plus ASM and wraps it into a massively parallel database grid. So far so good.

Exadata… cute. Because Terabytes on Teradata are already out there. Petabytes on Petadata (“Pet a data?”) isn’t visionary enough, so we need another factor 1000, with Exabytes on Exadata. Now this is far out enough. Can you “axe a data”? OK, never mind.

Why all this? I’m assuming that Oracle executives visited many data centers in their careers and never ever saw an Oracle logo on those server racks running their very own DBMS. IBM servers, yes, HP, Dell, Teradata, EMC, all there. No Oracle. That must have been a big sting.

Oracle was already very successful getting the Oracle logo out into the world. No major airport that I go through without an onslaught of banners with Oracle is #1, Oracle serves 19 of the top 20 companies in whatever industry, Oracle, Oracle. Paris Charles de Gaulle airport has turned red because of Oracle advertising.

Next step: data center. Well, Oracle was hardly going to start manufacturing any motherboards, chipsets, or disk drives themselves, since the world wasn’t really waiting for another hardware newbie behemoth, when the whole market is consolidating at the same time. Of course, watching even Microsoft starting to acquire hardware assets (I mean, beyond the Microsoft Mouse) with the Datallegro takeover must have accelerated the decision making process to provide an Oracle-branded box with suitable capabilities, and the 11g DBMS comes to mind. So I imagine a phone call from Redwood Shores to Cupertino, kinda like “Hey, Mark, we’ve been friends for a long time, certainly after you had left Teradata <chuckle>, could we get some of your blades and storage racks and wrap them into a nice bundle with our database, so we both can take over the data warehouse market?” Something to that effect and I am obviously exaggerating somewhat.

Hewlett Packard (the hardware unit, mind you!) smells a big opportunity, as most data warehouses these days run on Oracle, so why not have them run on an HP box? Quickly, a few other components are assembled to make it a nice appliance and as painless to manage for the potential user, a name is found (rumor has it, that Larry Ellison himself is heavily involved in all branding activities), welcome, Exadata, the new enemy of Teradata (and IBM).

Wait… wasn’t HP trying to sell its own NeoView box? What was that again? Oh yes, another highly scalable data warehouse “machine”. (They don’t want us to call NeoView an “appliance”). And wasn’t that targeted as direct competition to Teradata as well? By the looks of it, Oracle just stepped on HP toes, and HP let it happen. Funny, how market dynamics sometimes work. I’m sure, the clever marketiers in both companies have all arguments ready, why this new competition is no problem on either side: “It’s good for the customer, more choices, strategic relationship, yada, yada, yada.” However, when it comes to “our revenue” or “no revenue”, I can guarantee that both Oracle and HP favor the first option, which turns the friendship into plain competition. As we say in German: “Beim Geld hört die Freundschaft auf”. Means something like „Friendship ends where money begins“.

We’ll be watching and reporting on developments. Until then, welcome to hardware land, Oracle.

Sometimes it helps talking to a different set of clients to get a reality check. In my world, where all that’s needed is broadband and an airport (I’m exaggerating a little here) it is really helpful to get a new perspective on how our clients are grappling with day-to-day issues.

I recently met with a representative from the education department of a large African city. They wanted to do BI, getting reports on students, classes, teachers, grades, and so on. Now, I learned to be more careful with my advice in places like Africa, as some parts of the infrastructure aren’t comparable with European standards. So I didn’t recommend putting in a big honking data warehouse and some funky BI platform, as I knew (a) the budgets wouldn’t be there, (b) skills would be an issue, and (c) data volumes wouldn’t warrant such an investment.

So I started slowly and asked what kind of reports they’d be expecting to create, what applications and data sources they’d be tapping, how much data, how many potential users, and that sort of thing.

The lady from the education department said they wanted, for example, to get some “teacher reports” and what teachers were more successful, as indicated by better grades of their students. Easy enough I thought. Boy, was I wrong, as there was one problem: There wasn’t any data.

Apparently, the department wanted to know “which teacher is actually physically in the classroom?” or “what topics are those teachers actually discussing in class?”. Kinda different set of problem, compared to the run-of-the-mill inquiries on BI. So I suggested having some employees do regular rounds in the schools to monitor attendance, and create some data by recording their observations in … uh … Excel, at least for the time being. Never thought I would do such a thing. Good reality check.

Wherever I go to present on BI issues, such as technology, M&A, best practices, organizational issues, market development or else, one of my questions to the audience is “Who has a BI strategy?” and the typical response is fairly consistent. In most European or North American audiences, only about 2% of surveyed participants admit they have a BI strategy. Now, I don’t know whether those are any good, complete, or actionable, but it’s interesting nevertheless.

I asked the same question yesterday afternoon at the Gartner Future of IT conference in Sao Paulo, and, what do you know, more than 10% claimed to have a BI strategy. Still not the number I’d like to see, but significantly higher than in Chicago, Cologne, Copenhagen or Cape Town. I gotta have a closer look at those strategies.