High-Performance Computing Meets Databases

By Bill Howe, Ph.D., University of Washington eScience Institute

The large-scale data systems associated with high performance computing (HPC) applications ― e.g., parallel file systems ― are becoming increasingly “smart,” augmenting basic open(), seek() and read() interfaces with sophisticated indexing techniques and advanced metadata services.  The techniques and strategies developed in the database community ― logical and physical data independence, algebraic optimization, simple data parallel languages, and data models as first-class citizens― are applicable to problems faced by the HPC community as we enter the exascale age.

These were some of the reasons we established the High Performance Computing Data Base (HPCDB) workshop  in 2011: to bring together researchers in the database/data management community (who usually attend SIGMOD, VLDB and ICDE) with researchers from the high-performance computing community (who usually attend SuperComputing).

The second annual HPCDB workshop will be held November 11 at SuperComputing 12 in Salt Lake City.  This year we received a number of great position papers and have lined up a panel to discuss requirements for exascale IO.

Work at the intersection of the HPC and database/data management disciplines needs to be “application pull” rather than “technology push.” That’s why SuperComputing 12 is a natural place for this workshop (in addition to being a great host and sponsor).  In particular, we tried to shape the workshop agenda to address the applications important to the HPC community.  We asked each participant to specifically address how his work could be deployed in leadership-class computing platforms or address typical HPC applications ― simulation and visualization in the sciences, in particular.

A key goal is to try and push both communities out of their “comfort zones.”  We’re looking forward to a productive workshop and spirited discussion.

You can read more about our thinking and agenda at http://hpcdb.org  and follow the event on Twitter @HPCDB.   Hope to see you there or online.



This entry was posted in Computer Architecture, DBMS, High-Performance Computing, ISTC for Big Data Blog and tagged , , , . Bookmark the permalink.

Leave A Reply

Your email address will not be published. Required fields are marked *

3 × = nine