Setting the Record Straight – HTAP & OPDBMS

There has been much written in the industry about the convergence of transactions and analytics in database management systems (DBMS). There are many names for this convergence from multiple sources. Many of these sources have given proper credit for the concept; however, some have not, allowing readers to believe they originated the idea. I have no issue with anyone with the desire to create their own names or acronyms and to write about this. I do have an issue when they want readers to believe they originated the idea. One perfect example of how to do it right is Hybrid Operational / Analytic Processing (HOAP) from 451 Research. They created that name and acronym, and in their research, they credited Gartner, Inc. with the original concept (thank you 451 Research).

Gartner introduced Hybrid Transaction / Analytical Processing (HTAP) first in a Hype Cycle, “Hype Cycle for In-Memory Computing Technology, 2013” dated July 2013. In that document, we defined HTAP as: “An in-memory computing (IMC)-enabled hybrid transaction/analytical processing (HTAP) architecture leverages IMC techniques and technologies to enable analytical processing on the same (in-memory) data store that performs transaction processing”. Since that time, a search of Gartner’s body of research produces 186 specific documents pertaining to or about HTAP. Since our introduction of HTAP, we defined two forms of HTAP – Point-of-Decision HTAP & In-Process HTAP. Point-of-Decision HTAP is an architecture using in-memory computing (IMC) techniques and technologies to enable concurrent analytical and transaction processing on the same in-memory data store. In-Process HTAP is an application architecture whereby, in the context of a given application, analytical and transaction processing techniques are weaved together as needed to accomplish the business task.

Another misconception is that HTAP is a separate market, product or function within a product; this not true. HTAP is an architecture used for specific application use cases. You can purchase HTAP enabled applications but you cannot purchase HTAP as a product or “turn-on” HTAP in a product. DBMS products capable of supporting HTAP applications will have features such as builtin analytics and will normally be in-memory computing (IMC) enabled. Point-of-Decision HTAP is possible with DBMS products not supporting IMC but of course the application or transaction latency will be longer. In-Process HTAP really does require IMC as the latency requirements of the process or transaction is necessary higher and although “stepping out” of the process to perform analytics is possible, the SLAs of the transaction will normally not be met.

Finally, I want to set the record straight about operational database management systems (OPDBMS). The portion of the DBMS market involved with transactions has been called On-Line Transaction Processing (OLTP) for greater than 40 years, since the release of the IBM CICS software in 1969. Today, OLTP no longer describes transactions as all transactions are on-line. Further, we do have new types of transactions using new types of data. For example, IoT brings lighter-weight, streaming transactions that may or not be OLTP. Hence, in May, 2013, my colleague Merv Adrian and I published the first Gartner research renaming OLTP to OPDBMS – “The OLTP DBMS Market Becomes the Operational DBMS Market“. We believe this extends the OLTP market to these new types or data and transactions.

There is more change coming to this very fragmented market called Database Management Systems! Today we see eight or 10 major use cases, and 100’s if not 1000’s of DBMS products. There are relational, graph, document, XML, wide-column, key-value, time-series, columnar, in-memory, object-oriented, pick, prerelational and more types of DBMS products. Something has to give! Stay tuned here, soon.