Open – For Business – At the ASF

The Apache Software Foundation is about to celebrate an anniversary, and its extraordinary contribution to the economic refactoring of software stacks seems to be gaining more momentum with every passing year. After three Gartner Data and Analytics events on 3 continents with thousands of attendees in the past 4 weeks, I find myself more impressed than ever by the pervasive interest in and influence of open source software. I had several dozen one-on-one meetings with attendees (many, but not all, Gartner clients), and its appeal and impact on data management was reinforced again and again. Donald Feinberg and I noted in  State of the Open-Source DBMS Market, 2018 that

7.6% of DBMS revenue was attributable to OSDBMS-based offerings; a growth rate of 50% over the previous year in a broad market that grew 7.7%. This growth followed on the heels of a doubling in its size during the previous year.

The impact on revenue, significant as it is, understates the broader significance of open source offerings on data management technologies. In data quality, data preparation, data integration and other areas, open source software is broadly used, and much more so than revenue suggests – because it is so often available at no cost. In large part, this is attributable to the work of the Apache Software Foundation, which this week celebrates its 19^th year.

The 350 projects currently in the ASF lineup have been absorbed into the portfolios and development plans of vendors from the very largest to tiny startups and have been crucial in defining the direction of key technologies from web serving to networks and storage as well as DBMS. In my own work, I’ve been following the pervasive impact of Apache Hadoop and related projects for several years now, and nothing other than the cloud has had more impact on DBMS.

The commercialization of these technologies has taken many forms, from embracing, contributing and crediting, to outright uncredited appropriation, to attempts to “own” the branding – which the ASF rightly fends off fiercely. In the case of Hadoop, several failed attempts to brand collections of stack components by industry behemoths, including Intel and IBM, entered and exited rather quickly. Successful packagers are busy repositioning themselves. But Hadoop itself is everywhere, despite the new marketing messages of distribution vendors that leave the name out of their headlines. At the Gartner Summits, the surviving distributors did not feature the name on their booth signs, but attendees packed my Hadoop sessions, and one-on-one meetings typically featured Hadoop – and Apache. “Open source is a mandate,” said one client. We avoid the commercial DBMS vendors as much as possible.” In some cases, this ambition is a bit overoptimistic – relying on open source without commercial support is still not for the fainthearted. But over 3000 committers contributed code to Apache projects in the past 12 months, and organizations from Netflix to the US Federal Aviation Agency rely on its code.

My colleague Mark Driver notes the changes in attitudes about open source software in Use Service-Level Requirements to Drive Decisions Between Commercial and Self-Support for Open-Source Software:

Service and support strategies in the open-source world are a continuum, starting with an individual, then their employer, then expanding to the larger community, and finally ending with a rapidly expanding market of commercial vendors.

This is why the role of the ASF remains so important. By providing a vehicle for developers to work “in the open,” while keeping the playing field level in many respects, the ASF has enabled the rapid development and pervasive spread of key layers that everyone benefits from. And it’s good to see so many vendors stepping up by sponsoring an organization whose output they eagerly embrace for their own profit – though it’s still disappointing to see how many of the top 15 DBMS vendors are still not among the 48 corporate sponsors. Amazon, Cloudera, Hortonworks, IBM, MapR, and Microsoft have stepped up. It would be nice to see them joined by CA Technologies, EnterpriseDB, Fujitsu,  Intersystems, Oracle, Progress Software, SAP, Software AG, and Teradata. Most of these vendors contribute to some open source projects inside or outside Apache. The ASF also deserves their support to continue its mission – and they and their customers will benefit.