From datalog rules to efficient programs with time and space guarantees

This article describes a method for transforming any given set of Datalog rules into an efficient specialized implementation with guaranteed worst-case time and space complexities, and for computing the complexities from the rules. The running time is optimal in the sense that only useful combinations of facts that lead to all hypotheses of a rule being simultaneously true are considered, and each such combination is considered exactly once in constant time. The associated space usage may sometimes be reduced using scheduling optimizations to eliminate some summands in the space usage formula. The transformation is based on a general method for algorithm design that exploits fixed-point computation, incremental maintenance of invariants, and combinations of indexed and linked data structures. We apply the method to a number of analysis problems, some with improved algorithm complexities and all with greatly improved algorithm understanding and greatly simplified complexity analysis.

[1]  Tom Schrijvers,et al.  Analyses, Optimizations and Extensions of Constraint Handling Rules: Ph.D. Summary , 2005, ICLP.

[2]  I. V. Ramakrishnan,et al.  Term Indexing , 1995, Lecture Notes in Computer Science.

[3]  Yanhong A. Liu,et al.  Querying Complex Graphs , 2006, PADL.

[4]  K. A. Ross,et al.  Tabled Evaluation with Delaying for General Logic Programs , 1996 .

[5]  Robert Paige.  Formal differentiation: A program synthesis technique , 1981 .

[6]  Serge Abiteboul,et al.  Regular path queries with constraints , 1997, J. Comput. Syst. Sci..

[7]  Ronald L. Rivest,et al.  Certificate Chain Discovery in SPKI/SDSI , 2002, J. Comput. Secur..

[8]  Peter J. Stuckey,et al.  Optimizing Compilation of Constraint Handling Rules , 2001, ICLP.

[9]  Elnar Hajiyev,et al.  Semantics of static pointcuts in aspectJ , 2007, POPL '07.

[10]  Yanhong A. Liu,et al.  Efficiency by Incrementalization: An Introduction , 2000, High. Order Symb. Comput..

[11]  Yanhong A. Liu,et al.  Dynamic Programming via Static Incrementalization , 2003, High. Order Symb. Comput..

[12]  Yanhong A. Liu,et al.  Efficient type inference for secure information flow , 2006, PLAS '06.

[13]  Eric Van Wyk,et al.  Universal Regular Path Queries , 2003, High. Order Symb. Comput..

[14]  Yanhong A. Liu,et al.  Parametric regular path queries , 2004, PLDI '04.

[15]  Robert Paige,et al.  Finite Differencing of Computable Expressions , 1982, TOPL.

[16]  Michael Kifer,et al.  On compile-time query optimization in deductive databases by means of static filtering , 1990, TODS.

[17]  Lars Ole Andersen,et al.  Program Analysis and Specialization for the C Programming Language , 2005 .

[18]  Edmond Schonberg,et al.  Programming with Sets: An Introduction to SETL , 1986 .

[19]  Yanhong A. Liu,et al.  Improved Algorithm Complexities for Linear Temporal Logic Model Checking of Pushdown Systems , 2006, VMCAI.

[20]  Yanhong A. Liu,et al.  Dynamic Programming via Static Incrementalization , 1999, High. Order Symb. Comput..

[21]  Konstantinos Sagonas,et al.  XSB as a deductive database , 1994, SIGMOD '94.

[22]  Yanhong A. Liu,et al.  Generating Specialized Rules and Programs for Demand-Driven Analysis , 2008, AMAST.

[23]  Patrick Cousot,et al.  Abstract interpretation: a unified lattice model for static analysis of programs by construction or approximation of fixpoints , 1977, POPL.

[24]  Monica S. Lam,et al.  Cloning-based context-sensitive pointer alias analysis using binary decision diagrams , 2004, PLDI '04.

[25]  Raghu Ramakrishnan,et al.  Review - Magic Sets and Other Strange Ways to Implement Logic Programs , 1999, ACM SIGMOD Digit. Rev..

[26]  Georg Gottlob,et al.  Conjunctive queries over trees , 2004, JACM.

[27]  David Maier,et al.  Magic sets and other strange ways to implement logic programs (extended abstract) , 1985, PODS '86.

[28]  Benjamin Livshits,et al.  Context-sensitive program analysis as database queries , 2005, PODS.

[29]  David A. McAllester.  On the complexity analysis of static analyses , 1999, JACM.

[30]  Robert Paige,et al.  Program Derivation by Fixed Point Computation , 1989, Sci. Comput. Program..

[31]  Michael Leuschel,et al.  Logic Program Specialisation , 1998, Partial Evaluation.

[32]  Georg Gottlob,et al.  Complexity and expressive power of logic programming , 2001, CSUR.

[33]  Georg Gottlob,et al.  Conjunctive queries over trees , 2006, J. ACM.

[34]  Quanzhong Li,et al.  Indexing and Querying XML Data for Regular Path Expressions , 2001, VLDB.

[35]  E. L. Lozinskii,et al.  A Framework for an Efficient Implementation of Deductive Databases ∗ , 1986 .

[36]  Robert Paige,et al.  Real-time Simulation of a Set Machine on a Ram , 1989 .

[37]  Torben Æ. Mogensen,et al.  Partial Evaluation. Practice and Theory: DIKU 1998 International Summer School, Copenhagen, Denmark, June 29 - July 10, 1998 , 1998 .

[38]  Phokion G. Kolaitis,et al.  On the expressive power of datalog: tools and a case study , 1990, J. Comput. Syst. Sci..

[39]  John W. Lloyd,et al.  Partial Evaluation in Logic Programming , 1991, J. Log. Program..

[40]  Robert Paige,et al.  Programming with Invariants , 1986, IEEE Software.

[41]  Yanhong A. Liu,et al.  Strengthening invariants for efficient computation , 2001, Sci. Comput. Program..

[42]  Yanhong A. Liu,et al.  Program optimization using indexed and recursive data structures , 2002, PEPM '02.

[43]  Olivier Tardieu,et al.  Ultra-fast aliasing analysis using CLA: a million lines of C code in a second , 2001, PLDI '01.

[44]  Jeffrey F. Naughton,et al.  Bottom-Up Evaluation of Logic Programs , 1991, Computational Logic - Essays in Honor of Alan Robinson.

[45]  Yanhong A. Liu,et al.  Solving Regular Tree Grammar Based Constraints , 2001, SAS.

[46]  Mads Rosendahl,et al.  Automatic complexity analysis , 1989, FPCA.

[47]  John Beidler,et al.  Data Structures and Algorithms , 1996, Wiley Encyclopedia of Computer Science and Engineering.

[48]  Stephan Merz,et al.  Model Checking , 2000 .

[49]  Yanhong A. Liu,et al.  Efficient trust management policy analysis from rules , 2007, PPDP '07.

[50]  Harald Ganzinger,et al.  Logical Algorithms , 2002, ICLP.

[51]  Javier Esparza,et al.  Efficient Algorithms for Model Checking Pushdown Systems , 2000, CAV.

[52]  Matthias Jarke,et al.  Logic Programming and Databases , 1984, Expert Database Workshop.

[53]  Yanhong A. Liu,et al.  From datalog rules to efficient programs with time and space guarantees , 2003, PPDP '03.

[54]  Butler W. Lampson,et al.  SPKI Certificate Theory , 1999, RFC.

[55]  Joxan Jaffar,et al.  Set Constraints and Set-Based Analysis , 1994, PPCP.

[56]  Michael J. Maher,et al.  Constraint Logic Programming: A Survey , 1994, J. Log. Program..

[57]  Philip A. Bernstein,et al.  Computational problems related to the design of normal form relational schemas , 1979, TODS.

[58]  Thomas W. Reps,et al.  Program analysis via graph reachability , 1997, Inf. Softw. Technol..

[59]  Yanhong A. Liu,et al.  Solving Regular Path Queries , 2002, MPC.

[60]  Moshe Y. Vardi.  The complexity of relational query languages (Extended Abstract) , 1982, STOC '82.

[61]  Yanhong A. Liu,et al.  Static caching for incremental computation , 1998, TOPL.

[62]  Harald Ganzinger,et al.  A New Meta-complexity Theorem for Bottom-Up Logic Programs , 2001, IJCAR.

[63]  Peter J. Stuckey,et al.  Optimizing compilation of constraint handling rules in HAL , 2004, Theory and Practice of Logic Programming.

[64]  Georg Gottlob,et al.  On the complexity of single-rule datalog queries , 2003, Inf. Comput..

[65]  Georg Gottlob,et al.  On the Complexity of Single-Rule Datalog Queries , 1999, LPAR.

[66]  Somesh Jha,et al.  Model checking SPKI/SDSI , 2004, J. Comput. Secur..

[67]  Alexander Aiken,et al.  Introduction to Set Constraint-Based Program Analysis , 1999, Sci. Comput. Program..

[68]  Serge Abiteboul,et al.  Querying Semi-Structured Data , 1997, Encyclopedia of Database Systems.

[69]  Diego Calvanese,et al.  Answering regular path queries using views , 2000, Proceedings of 16th International Conference on Data Engineering (Cat. No.00CB37073).

[70]  Fritz Henglein,et al.  Type Analysis and Data Structure Selection , 1991 .

[71]  Ninghui Li,et al.  DATALOG with Constraints: A Foundation for Trust Management Languages , 2003, PADL.

[72]  Serge Abiteboul,et al.  Foundations of Databases , 1994 .

[73]  Hisao Tamaki,et al.  OLD Resolution with Tabulation , 1986, ICLP.