Solving the “One Concurrency Control Does Not Fit All” Problem for OLTP Databases

By Dixin Tang and Aaron J. Elmore, University of Chicago In this post, we present a new transactional database system that adaptively changes data organization and concurrency control protocols in response to workload changes. With the increasing memory sizes of … Continue reading

Posted in Big Data Architecture, Data Management, DBMS, ISTC for Big Data Blog | Tagged , , , , , | Leave a comment

ISTC Releases Open Source Code for BigDAWG Polystore System

By Dr. Tim Mattson, Intel and Dr. Vijay Gadepally and Kyle O’Brien, MIT Lincoln Laboratory Today, the ISTC for Big Data released the first version of BigDAWG, our polystore system for simplifying integration and analytics of disparate data at scale. BigDAWG is … Continue reading

Posted in Analytics, Big Data Architecture, Data Management, Databases and Analytics, DBMS, ISTC for Big Data Blog, Polystores, Query Engines, Storage | Tagged , , , , , , , , | Leave a comment

ISTC Researchers Present Work at NEDB Day 2017

ISTC for Big Data principal investigators, researchers and their students presented work at North East Database Day 2017, held at MIT’s Stata Center in Cambridge, Mass., on January 27, 2017. Microsoft and Facebook sponsored the event. The 9th Annual North East … Continue reading

Posted in Big Data Architecture, Data Management, Databases and Analytics, DBMS, ISTC for Big Data Blog, Storage, Streaming Big Data, Visualizing Big Data | Tagged , , , , , , , , , , | Leave a comment

NVMRocks: RocksDB on Non-Volatile Memory Systems

By Jianhong Li (CMU), Andrew Pavlo (CMU), and Siying Dong (Facebook) Non-volatile memory (NVM) has been a game-changing memory technology. In contrast to traditional block-based durable storage devices, it provides low latency comparable to DRAM and byte-addressability. Although NVM is … Continue reading

Posted in Big Data Architecture, DBMS, ISTC for Big Data Blog, Storage | Tagged , , , , , | 2 Comments

Write-Behind Logging

By Joy Arulraj, Matthew Perron, and Andrew Pavlo, Carnegie Mellon University In a joint collaboration between Carnegie Mellon University and Intel Labs, we explore the changes required in the logging and recovery algorithms in non-volatile memory database management systems (DBMSs). The results of this work … Continue reading

Posted in Big Data Architecture, Data Management, DBMS, ISTC for Big Data Blog, Math and Algorithms, Storage | Tagged , , , , , , , | Leave a comment

Interactive Search and Exploration over Large Multidimensional Data

by Alexander Kalinin, Ugur Cetintemel and Stan Zdonik of Brown University In the Big Data era, professionals across scientific areas need efficient, interactive ad-hoc data analysis. Ideally, they need generic and reusable systems tools for interactive search, exploration and mining … Continue reading

Posted in Analytics, Big Data Applications, Data Management, Databases and Analytics, DBMS, ISTC for Big Data Blog | Tagged , , , , , | Leave a comment

Genomics Data, Analytics and the Future of Climate Change

By Vijay Gadepally, MIT CSAIL, in collaboration with the Chisholm Laboratory at MIT Meet Prochlorococcus marinus, a marine cyanobacterium that’s intricately linked to the global carbon cycle, widely present in seawater, and possibly holds secrets to future climate change. These … Continue reading

Posted in Big Data Applications, Big Data Architecture, Data Management, Databases and Analytics, DBMS, Graph Computation, ISTC for Big Data Blog, Polystores, Streaming Big Data, Tools for Big Data, Visualizing Big Data | Tagged , , , , , , , , | Leave a comment

Rethinking Streaming: Correct State Matters!

by Nesime Tatbul, Intel Labs and MIT CSAIL; Kristin Tufte, Portland State University; and Stan Zdonik, Brown University Stream processing has largely been thought of as real-time analytics. Data enters the system as streams and analytic functions (aggregates) are computed on the … Continue reading

Posted in Big Data Applications, DBMS, ISTC for Big Data Blog, Streaming Big Data | Tagged , , , , , , , , , | Leave a comment

Larger-than-Memory Data Management on Modern Storage Hardware for In-Memory OLTP Database Systems

By Lin Ma, Carnegie Mellon University; Joy Arulraj, Carnegie Mellon University; Sam Zhao, Brown University; Andrew Pavlo, Carnegie Mellon University; Subramanya R. Dulloor, Intel Labs; Michael J. Giardino, Georgia Institute of Technology; Jeff Parkhurst, Jason L. Gardner, Kshitij Doshi, Intel Labs; and Col. Stanley Zdonik, Brown … Continue reading

Posted in Big Data Architecture, Data Management, DBMS, ISTC for Big Data Blog, Storage | Tagged , , , , , | Leave a comment

TicToc: Time Traveling Optimistic Concurrency Control

By Xiangyao Yu, MIT CSAIL; Andrew Pavlo, Carnegie Mellon; Daniel Sanchez and Srinivas Devadas, MIT CSAIL The TicToc algorithm enables scalable and high-performing concurrency control for future multi- and many-core systems. Large-scale, highly parallel transaction processing systems can be built with TicToc. We … Continue reading

Posted in Big Data Architecture, Data Management, DBMS, ISTC for Big Data Blog, Math and Algorithms | Tagged , , , , | Leave a comment