PF-OLA: a high-performance framework for parallel online aggregation

Online aggregation provides estimates to the final result of a computation during the actual processing. The user can stop the computation as soon as the estimate is accurate enough, typically early in the execution. This allows for the interactive data exploration of the largest datasets.In this paper we introduce the first framework for parallel online aggregation in which the estimation virtually does not incur any overhead on top of the actual execution. We define a generic interface to express any estimation model that abstracts completely the execution details. We design a novel estimator specifically targeted at parallel online aggregation. When executed by the framework over a massive 8 TB TPC-H instance, the estimator provides accurate confidence bounds early in the execution even when the cardinality of the final result is seven orders of magnitude smaller than the dataset size and without incurring overhead.

[1]  Fei Xu,et al.  The DBO database system , 2008, SIGMOD Conference.

[2]  Christopher Ré,et al.  Towards a unified architecture for in-RDBMS analytics , 2012, SIGMOD Conference.

[3]  Beng Chin Ooi,et al.  Continuous sampling for online aggregation over multiple queries , 2010, SIGMOD Conference.

[4]  Jeffrey F. Naughton,et al.  A scalable hash ripple join algorithm , 2002, SIGMOD '02.

[5]  Suman Nath,et al.  PR-join: a non-blocking join achieving higher early result rate with statistical guarantees , 2010, SIGMOD Conference.

[6]  Carlo Zaniolo,et al.  Early Accurate Results for Advanced Analytics on MapReduce , 2012, Proc. VLDB Endow..

[7]  A. Winsor.  Sampling techniques. , 2000, Nursing times.

[8]  Florin Rusu,et al.  GLADE: a scalable framework for efficient analytics , 2012, OPSR.

[9]  Fei Xu,et al.  Confidence bounds for sampling-based group by estimates , 2008, TODS.

[10]  Sara Cohen,et al.  User-defined aggregate functions: bridging theory and practice , 2006, SIGMOD Conference.

[11]  Chris Jermaine,et al.  The Sort-Merge-Shrink join , 2006, TODS.

[12]  Florin Rusu,et al.  Parallel online aggregation in action , 2013, SSDBM.

[13]  Peter J. Haas,et al.  Large-sample and deterministic confidence intervals for online aggregation , 1997, Proceedings. Ninth International Conference on Scientific and Statistical Database Management (Cat. No.97TB100150).

[14]  Ion Stoica,et al.  Blink and It's Done: Interactive Queries on Very Large Data , 2012, Proc. VLDB Endow..

[15]  Frank Olken,et al.  Random Sampling from Databases , 1993 .

[16]  Chris Jermaine,et al.  Online Estimation For Subset-Based SQL Queries , 2005, VLDB.

[17]  Carlo Zaniolo,et al.  Using SQL to Build New Aggregates and Extenders for Object- Relational Systems , 2000, VLDB.

[18]  Helen J. Wang,et al.  Online aggregation , 1997, SIGMOD '97.

[19]  Joseph M. Hellerstein,et al.  CONTROL: continuous output and navigation technology with refinement on-line , 1998, SIGMOD '98.

[20]  Beng Chin Ooi,et al.  Distributed Online Aggregation , 2009, Proc. VLDB Endow..

[21]  Minos N. Garofalakis,et al.  Approximate Query Processing: Taming the TeraBytes , 2001, VLDB.

[22]  Joseph M. Hellerstein,et al.  MapReduce Online , 2010, NSDI.

[23]  Chris Jermaine,et al.  Scalable approximate query processing with the DBO engine , 2008, TODS.

[24]  Michael Stonebraker,et al.  The POSTGRES Data Model , 1987, Research Foundations in Object-Oriented and Semantic Database Systems.

[25]  Fei Xu,et al.  Turbo-Charging Estimate Convergence in DBO , 2009, Proc. VLDB Endow..

[26]  Peter J. Haas,et al.  Ripple joins for online aggregation , 1999, SIGMOD '99.

[27]  Florin Rusu,et al.  Sampling Estimators for Parallel Online Aggregation , 2013, BNCOD.

[28]  Subramanian Arumugam,et al.  The DataPath system: a data-centric analytic processing engine for large data warehouses , 2010, SIGMOD Conference.

[29]  Chris Jermaine,et al.  Online aggregation for large MapReduce jobs , 2011, Proc. VLDB Endow..

[30]  Yu Cheng,et al.  GLADE: big data analytics made easy , 2012, SIGMOD Conference.

[31]  Chris Jermaine,et al.  A Bayesian Method for Guessing the Extreme Values in a Data Set , 2007, VLDB.