About cookies on this site Our websites require some cookies to function properly (required). In addition, other cookies may be used with your consent to analyze site usage, improve the user experience and for advertising. For more information, please review your options. By visiting our website, you agree to our processing of information as described in IBM’sprivacy statement. To provide a smooth navigation, your cookie preferences will be shared across the IBM web domains listed here.
Publication
ICDCS 2016
Conference paper
DL-Store: A Distributed Hybrid OLTP and OLAP Data Processing Engine
Abstract
There has been a recent push in the database community towards supporting real-time analytical queries (OLAP) while sustaining a large volume of fine-grained updates (OLTP). Supporting these types of workloads require both an efficient data storage layer as well as a distributed architecture. In this demo, we address the latter point with our Distributed Lineage-based Data Store (DL-Store), which is a distributed data processing engine. DL-Store is built on top of L-Store, which is a lineage-based storage architecture designed to handle mixed OLTP and OLAP workloads, and provides scalability and elasticity by supporting multiple L-Store nodes. To maintain the desired consistency semantics, DL-Store employs a distributed transaction handler component which can horizontally scaled by provisioning additional transaction manager nodes. We leverage partitioning in the record space of the transactions to minimize communication across transaction managers while ensuring consistent execution. The demo shows our implementation of DL-Store over Apache Spark using a variety of use cases.