S-Store: A Big-Velocity Database System

By John Meehan, PhD Candidate, Brown University

At the 2014 Intel Science and Technology Center for Big Data annual Research Retreat, Nesime Tatbul of Intel Labs and MIT provided an update on S-Store.

Background

Recently, the architectures of transactional processing DBMSs and real-time stream processing engines have been converging in a number of ways. Both systems operate primarily in memory, offer low-latency processing on batched inputs, and utilize scalable, shared-nothing clusters. Although they are similar, each system brings its own functionality to the table; OLTP demands coordination and safety of short atomic computations, while stream processing primarily addresses data-driven workflows in real-time.

Purpose

S-Store, a new, big-velocity database system, integrates these two disparate systems. It extends the main-memory, low-latency, distributed OLTP system H-Store by adding streaming constructs such as stream tables, windows, and triggers, allowing it to react to ever-changing data.  It introduces a novel ACID streaming transaction model in which a workflow is defined as a sequence of transactions to be performed on an input batch of tuples.

These transaction executions cannot be arbitrarily ordered, and must balance the guarantee of expected workflow results with allowing OLTP transactions as frequent access to shared data as possible. S-Store also takes advantage of the fault-tolerance methods of both OLTP and streaming systems, providing both full command-logging and upstream backup.

Demonstration of how S-Store simultaneously handles both the OLTP and real-time data management aspects of a hypothetical bike-sharing system, enabling its operators to monitor usage in real-time to manage demand. 

Current Status

Initial results show that the streaming components of S-Store not only increase its functionality, but also improve its performance over H-Store on a variety of streaming workloads.

Future research directions of S-Store are very promising, including extensions to the transaction model and the dynamic load distribution of streaming workloads while continuing to protect data integrity.

Additional Resources and Reading:

S-Store project web site

Paper: “S-Store: A Streaming NewSQL System for Big Velocity Applications.” Proceedings of the VLDB Endowment, Vol. 7, No. 13

S-Store: Real-Time Analytics Meets Transaction Processing.” Intel Science and Technology Center for Big Data blog, February 10, 2014

 

This entry was posted in Analytics, Big Data Applications, ISTC for Big Data Blog, Streaming Big Data and tagged , , , , , , . Bookmark the permalink.

Leave a Reply

Your email address will not be published. Required fields are marked *


× one = 7