Anti-Caching and Non-Volatile Memory for Transactional DBMS

By Andy Pavlo of Brown University and Mike Stonebraker of MIT CSAIL

The traditional wisdom for building disk-based database management systems (DBMSs) is to organize blocks of data on disk, with a main memory block cache. In order to improve performance given high disk latency, systems must employ dynamic record-level locking, an Aries-style write-ahead log, and a multithreaded architecture in order to allow multiple transactions to access the DBMS at the same time.

Previous work has shown that all of this overhead is a significant hindrance to scaling the performance of a transactional DBMS. Thus, to achieve better performance for modern transaction processing workloads, new main memory DBMSs have been proposed. Recent implementations of these systems are able to execute ACID transactions 25-50x faster than DBMSs that employ the traditional techniques used to mask the latency of disks. These systems, however, only support databases that fit in main memory, which is a substantial limitation for many applications.

At the recent ISTC for Big Data Research Retreat in California, Andy Pavlo of Brown University and Mike Stonebraker of MIT CSAIL reported on two emerging research projects on overcoming this limitation in main memory database systems. This work was discussed in the context of the H-Store distributed DBMS, part of the emerging class of NewSQL systems. The H-Store project is a collaboration between the database research groups at BrownMIT, and Yale.


First, Andy Pavlo presented a radical new architecture for main memory DBMSs that we call anti-caching. In this scenario, when memory is exhausted, the DBMS gathers the “coldest” tuples and writes them to disk with minimal translation from their main memory format, thereby freeing up space for more recently accessed tuples. As such, the “hotter” data resides in main memory, while the colder data resides on disk in the anti-cache portion of the system. Unlike in a traditional DBMS architecture, tuples do not reside in both places; each tuple is either in memory or in a disk block, but never in both places at the same time. In this new architecture, main memory, rather than disk, becomes the primary storage device.

A prototype of the anti-caching design has been implemented in H-Store and has been shown to outperform a well-known, open-source traditional database system by up to 10x for a database that is twice as large as the size of available memory.

Non-Volatile Memory

Mike Stonebraker then discussed new research on using experimental non-volatile memory (NVM) hardware in H-Store. Currently, H-Store uses disks to store snapshots of the in-memory database as well as for writing a command log (i.e., logical log) of the transactions that were executed. Upon a node failure, the system loads the last checkpoint that was taken and then replays the command log to get the database back to the state of where it was before the crash. All of this machinery is necessary because H-Store is a main memory-oriented system.

Starting this summer, we will begin researching how to best integrate NVM into a transactional DBMS like H-Store. Our initial work will include using NVM as a block-addressable device as a replacement for flash. The more interesting research will be in how to best use NVM byte-addressable replacement for DRAM. We will investigate techniques and data structures on a variety of workloads to better understand the latency and wear-leveling properties of this new hardware.

You can follow Andy Pavlo on Twitter.

This entry was posted in Big Data Architecture, Computer Architecture, DBMS, ISTC for Big Data Blog and tagged , , . Bookmark the permalink.

Leave A Reply

Your email address will not be published. Required fields are marked *

eight + 1 =