Hybrid transaction/analytical processing (HTAP) is a term created by Gartner Inc., an information technology research and advisory company, in its early 2014 research report Hybrid Transaction/Analytical Processing Will Foster Opportunities for Dramatic Business Innovation.[1] As defined by Gartner:
Hybrid transaction/analytical processing (HTAP) is an emerging application architecture that "breaks the wall" between transaction processing and analytics. It enables more informed and "in business real time" decision making.[2][1]
In more recent reports Gartner has begun referring to HTAP as "augmented transactions."[3] Another analyst firm Forrester Research calls the same concept "Translytical"[4] while 451 Group calls it "Hybrid operational and analytical processing" or HOAP.[5]
Background edit
In the 1960s, computer use in the business sector began with payroll transactions and later included tasks in areas such as accounting and billing. At that time, users entered data, and the system processed it at a later time. Further development of instantaneous data processing, or online transaction processing (OLTP), led to widespread OLTP use in government and business-sector information systems.[6]
Online analytical processing (OLAP) covers the analytical processing involved in creating, synthesizing, and managing data. With greater data demands among businesses,[citation needed] OLAP also has evolved. To meet the needs of applications, both technologies are dependent on their own systems and distinct architectures.[7][6] As a result of the complexity in the information architecture and infrastructure of both OLTP and OLAP systems, data analysis is delayed.[7][need quotation to verify]
HTAP advantages and challenges edit
There are various interpretations of HTAP other than Gartner's original definition; an "emerging architecture". These interpretations suggest different advantages, one being a database functionality. Recent advances in research, hardware, OLTP and OLAP capabilities, in-memory and cloud native database technologies,[8] scalable transactional management and products enable transactional processing and analytics, or HTAP, to operate on the same database.[7][9][6]
However, Gartner's most recent reports suggest broader advantages than a single unified database can offer. Traditional application architectures separated transactional and analytical systems. Digital business, and the need to respond to business moments, means that using "after the fact" analysis is no longer adequate. Business moments are transient opportunities that must be exploited in real time. If an organization is unable to recognize and/or respond quickly to a business moment by taking fast and well-informed decisions, then some other organization will, resulting in a missed opportunity (or a new business threat). HTAP allows advanced analytics to be run in real time on "in flight" transaction data, providing an architecture that empowers users to respond more effectively to business moments.[10]
The main technical challenges for an HTAP database are how to be efficient both for operational (many small transactions with a high fraction of updates) and analytical workloads (large and complex queries traversing large number of rows) on the same database system and how to prevent the interference of the analytical queries over the operational workload. This kind of operational workload is also commonly referred to as Operational Analytical Processing.
HTAP solves the issue of analytic latency in several ways, including eliminating the need for multiple copies of the same data and the requirement for data to be offloaded from operational databases to data warehouses via ETL processes.[7][9]
Most applications of HTAP are enabled by in-memory technologies that can process a high volume of transactions and offer features such as forecasting and simulations. New HTAP technologies use scalable transactional processing, and do not need to rely on keeping the whole database in-memory. HTAP has the potential to change the way organizations do business by offering immediate business decision-making capabilities based on live and sophisticated analytics of large volumes of data. Government and business leaders can be informed of real-time issues, outcomes, and trends that necessitate action, such as in the areas of public safety, risk management, and fraud detection.[7][11]
Some challenges for HTAP include limited industry experience and skills, as well as undefined best practices.[7]
In 2020, the first paper in the industry was published by the team of PingCAP describing the practical implementation of a distributed Hybrid Transactional/Analytical Processing (HTAP) database: TiDB: A Raft-based HTAP Database.[12]
References edit
- ^ a b "Hybrid Transaction/Analytical Processing Will Foster Opportunities for Dramatic Business Innovation". Gartner. 28 January 2014. Retrieved 4 October 2021.
- ^ "Market Guide for HTAP-Enabling In-Memory Computing Technologies". www.gartner.com. Retrieved 15 April 2017.
- ^ "Hype Cycle for Data Management, 2019". Gartner.
- ^ "Forrester". Forrester.
- ^ https://mariadb.com/wp-content/uploads/2020/09/enterprises-turn-to-hoap-for-oltp-workloads_analyst-report_1088.pdf [bare URL PDF]
- ^ a b c Bog, Anja. Benchmarking Transaction and Analytical Processing Systems: The Creation of a Mixed Workload Benchmark and Its Application Springer-Verlage Berlin Heidelberg. 2014
- ^ a b c d e f Pezzini, Massimo; Feinberg, Donald; Rayner, Nigel; Edjlali, Roxane. "Hybrid Transaction/Analytical Processing Will Foster Opportunities for Dramatic Business Innovation." Gartner. 28 January 2014
- ^ "Azure Analytics: Clarity in an instant". azure.microsoft.com. 19 May 2020. Retrieved 20 June 2020.
- ^ a b Wolpe, Toby. "SQL and NoSQL? Fine, but how does the hybrid database fit in?" ZDNet. 12 May 2014
- ^ "How to Enable Digital Business Innovation via Hybrid Transaction/Analytical Processing". www.gartner.com. Retrieved 15 April 2017.
- ^ Baer, Tony. "Fast Data hits the Big Data fast lane." ZDNet. 16 April 2012
- ^ "TiDB: A Raft-based HTAP Database" (PDF). Proceedings of the VLDB Endowment. 13 (12): 3072. doi:10.14778/3415478.3415535. ISSN 2150-8097. S2CID 221666363.