Optimal memory management strategies for query processing

The principle of program locality states that a program references only a small subset of its address space, the locality set, during an extended phase of execution. The reference load of a program can thus be specified by a sequence of locality sets and the duration of their corresponding phases. We introduce a generalized memory management model which uses this set representation of program reference loads, and use it to study memory management problems in various types of operating environment. The model accommodates the use of fixed- and variable-size main-memory buffers as well as the overlapping of locality-set processing and secondary-storage accesses (preparing). We devise four optimal memory management strategies which minimize secondary-storage accesses and main-memory utilization. The PSETMIN and SETMIN strategies minimize the number of page fetches for fixed-size buffers with and without prepaging, respectively. For variable-size buffers, we devise the PSETVMIN and SETVMIN strategies, which minimize the VMIN cost with and without prepaging, respectively. These optimal memory management strategies can be beneficially used in applications such as font caching in computer typesetting and query optimization in relational database systems. We show that our memory management model is a generalization of the font caching model of Fuchs and Knuth. For their application to query optimization, we develop a novel query processing scheme, in which multiattribute indices are used for decomposing a query into an equivalent set of subqueries. These subqueries operate on small subsets of the data referenced by the original query, and can be sequenced in an appropriate order to minimize query processing cost. We term the problem of finding such optimal sequences the locality-set sequencing (LSS) problem. Two of its variations using variable-size buffers are shown to be NP-hard. We relate the LSS problem to well-known combinatorial problems such as the traveling salesman problem and the quadratic assignment problem. We show that a reflection property of the SETMIN and SETVMIN strategies can be exploited to reduce by one half the running time of exhaustive algorithms for the LSS problem. Lastly, we present a heuristic algorithm that further reduces the running time for solving the LSS problem.