Memory allocation strategies for complex decision support queries

Complex decision support queries, as exempliied by the TPC-D benchmark, make heavy use of join, aggregation and sorting operations. These operations can be very memory intensive and it is unlikely (particularly in a multiuser environment) that a database system will always have enough memory to completely satisfy the demands of all concurrently running operators. In this paper we propose, analyze and evaluate four diierent algorithms that distribute available memory among operators in ways that satisfy all operator requirements and concurrent schedulability constraints. One of these algorithms, based on linear programming, is also optimal in that it is guaranteed to produce the best query response time. We also show how these algorithms can be extended to work on shared-nothing parallel database systems. All the proposed strategies are completely general and can be used with any extended relational operations that show a linear improvement in performance with increased memory availability.

[1]  David J. DeWitt,et al.  Managing Intra-operator Parallelism in Parallel Database Systems , 1995, VLDB.

[2]  Minos N. Garofalakis,et al.  Parallel Query Scheduling and Optimization with Time- and Space-Shared Resources , 1997, VLDB.

[3]  Patricia G. Selinger,et al.  Access path selection in a relational database management system , 1979, SIGMOD '79.

[4]  Hansjörg Zeller,et al.  An Adaptive Hash Join Algorithm for Multiuser Environments , 1990, VLDB.

[5]  Goetz Graefe,et al.  Memory-Contention Responsive Hash Joins , 1994, VLDB.

[6]  Surajit Chaudhuri,et al.  An overview of data warehousing and OLAP technology , 1997, SGMD.

[7]  David J. DeWitt,et al.  Dynamic Memory Allocation for Multiple-Query Workloads , 1993, VLDB.

[8]  Donald D. Chamberlin,et al.  Access Path Selection in a Relational Database Management System , 1989 .

[9]  Erhard Rahm,et al.  Dynamic Multi-Resource Load Balancing in Parallel Database Systems , 1995, VLDB.

[10]  David J. DeWitt,et al.  Building a scaleable geo-spatial DBMS: technology, implementation, and evaluation , 1997, SIGMOD '97.

[11]  Jaideep Srivastava,et al.  Optimizing multi-joint queries in parallel relational databases , 1993, [1993] Proceedings of the Second International Conference on Parallel and Distributed Information Systems.

[12]  Patrick Valduriez,et al.  Open issues in parallel query optimization , 1996, SGMD.

[13]  Philip S. Yu,et al.  Using Segmented Right-Deep Trees for the Execution of Pipelined Hash Joins , 1992, VLDB.

[14]  Miron Livny,et al.  Managing memory for real-time queries , 1994, SIGMOD '94.

[15]  David J. DeWitt,et al.  Tradeoffs in Processing Complex Join Queries via Hashing in Multiprocessor Database Machines , 1990, VLDB.