Magic Functions: A Technique to Optimize Extended Datalog Recursive Programs

Several methods have been proposed to compile recursive Datalog programs. The most well-known perform a rewriting of rules using MAGIC or PROBLEM predicates in order to push selections before recursion. Rewritten rule systems are generally complex and difficult to translate into optimized relational algebra programs. Moreover, they often generate too many results; thus, the query must be applied to the generated results to eliminate non relevant answers. In this paper, after a survey of the existing compilation techniques which points out their limitations, we develop the magic function method iutroduced in [GardariuDeMaindreville86]. It is based on an understanding of the query as a function which maps columns of a relation to other columns. A query against recursive rules is then translated into a fixpoint functional equation. The resolution of this fixpoint equation using Tarski’s theorem leads to efficient computation of the query answer. In particular, the derived algorithms push selections through recursion, because selections appear as function arguments. They generate only relevant answers to a given query, without redundant data computation. The purpose of this paper is the introduction of a generalized method to obtain and resolve the fixpoint functional equation. The method is general enough to handle non-binary rules, cyclic rules and function symbols. The main advantages of the method are : (1) It directly generates an optimized relational algebra program. (2) It performs a symbolic precomputation which permits rule redundancy elimination. (3) It fully supports function symbols and range queries.