Dynamic memory allocation strategies for parallel query execution

In the decision support queries which manipulate large data volumes, it is frequent that a query constituted by several joins can not be computed completely in memory. In this paper, we propose three strategies allowing to assign the memory of a shared-nothing parallel architecture to operation clones of a query. The performance evaluation of the three strategies shows that the strategies which favor the operation clones using a lot of memory obtain a better response time than the strategy which favors the clones using little memory. The main contribution of this paper is to take into account the available memory sizes on every processor and to avoid allotting the same processor to two operation clones that must run in parallel.

[1]  Christos Faloutsos,et al.  Flexible buffer allocation based on marginal gains , 1991, SIGMOD '91.

[2]  Abraham Silberschatz,et al.  Efficient and accurate cost models for parallel query optimization (extended abstract) , 1996, PODS.

[3]  Philip S. Yu,et al.  Scheduling and processor allocation for parallel execution of multijoin queries , 1992, [1992] Eighth International Conference on Data Engineering.

[4]  David J. DeWitt,et al.  Dynamic Memory Allocation for Multiple-Query Workloads , 1993, VLDB.

[5]  David J. DeWitt,et al.  A performance evaluation of four parallel join algorithms in a shared-nothing multiprocessor environment , 1989, SIGMOD '89.

[6]  Patrick Valduriez,et al.  Memory-adaptive scheduling for large query execution , 1998, CIKM '98.

[7]  Hesham El-Rewini,et al.  Dynamic Data Reallocation for Skew Management in Shared-Nothing Parallel Databases , 1997, Distributed and Parallel Databases.

[8]  Abraham Silberschatz,et al.  Efficient and Acurate Cost Models for Parallel Query Optimization. , 1996, ACM SIGACT-SIGMOD-SIGART Symposium on Principles of Database Systems.

[9]  David J. DeWitt,et al.  Tradeoffs in Processing Complex Join Queries via Hashing in Multiprocessor Database Machines , 1990, VLDB.

[10]  Minos N. Garofalakis,et al.  Parallel Query Scheduling and Optimization with Time- and Space-Shared Resources , 1997, VLDB.

[11]  David J. DeWitt,et al.  Data placement in shared-nothing parallel database systems , 1997, The VLDB Journal.

[12]  Kian-Lee Tan,et al.  Multi-Join Optimization for Symmetric Multiprocessors , 1993, VLDB.

[13]  William L. Maxwell,et al.  Theory of scheduling , 1967 .

[14]  Philip S. Yu,et al.  Using Segmented Right-Deep Trees for the Execution of Pipelined Hash Joins , 1992, VLDB.

[15]  Abdelkader Hameurlain,et al.  Hybrid Simultaneous Scheduling and Mapping in SQL Multi-query Parallelization , 1999, DEXA.

[16]  David J. DeWitt,et al.  Memory allocation strategies for complex decision support queries , 1998, CIKM '98.

[17]  Abdelkader Hameurlain,et al.  Scheduling and mapping for parallel execution of extended SQL queries , 1995, CIKM '95.