Existential Heap Abstraction Entailment Is Undecidable

In this paper we study constraints for specifying properties of data structures consisting of linked objects allocated in the heap. Motivated by heap summary graphs in role analysis and shape analysis we introduce the notion of regular graph constraints. A regular graph constraint is a graph representing the heap summary; a heap satisfies a constraint if and only if the heap can be homomorphically mapped to the summary. Regular graph constraints form a very simple and natural fragment of the existential monadic second-order logic over graphs. One of the key problems in a compositional static analysis is proving that procedure preconditions are satisfied at every call site. For role analysis, precondition checking requires determining the validity of implication, i.e., entailment of regular graph constraints. The central result of this paper is the undecidability of regular graph constraint entailment. The undecidability of the entailment problem is surprising because of the simplicity of regular graph constraints: in particular, the satisfiability of regular graph constraints is decidable. Our undecidability result implies that there is no complete algorithm for statically checking procedure preconditions or postconditions, simplifying static analysis results, or checking that given analysis results are correct. While incomplete conservative algorithms for regular graph constraint entailment checking are possible, we argue that heap specification languages should avoid second-order existential quantification in favor of explicitly specifying a criterion for summarizing objects.

[1]  Dawson R. Engler,et al.  Checking system rules using system-specific, programmer-written compiler extensions , 2000, OSDI.

[2]  R. Wilhelm,et al.  Parametri Shape Analysis via 3-Valued Logi , 1999 .

[3]  Alexander Aiken,et al.  Flow-sensitive type qualifiers , 2002, PLDI '02.

[4]  David Walker,et al.  Alias Types for Recursive Data Structures , 2000, Types in Compilation.

[5]  Thomas Reps,et al.  A decidable logic for linked data structures , 1999 .

[6]  Viktor Kuncak,et al.  Typestate Checking and Regular Graph Constraints , 2004, ArXiv.

[7]  Mark N. Wegman,et al.  Analysis of pointers and structures (with retrospective) , 1990 .

[8]  Yuri Gurevich,et al.  The Classical Decision Problem , 1997, Perspectives in Mathematical Logic.

[9]  Ronald Fagin,et al.  On Monadic NP vs. Monadic co-NP , 1995, Inf. Comput..

[10]  G. Ramalingam,et al.  The undecidability of aliasing , 1994, TOPL.

[11]  J. Dekker,et al.  Some theorems on classes of recursively enumerable sets , 1958 .

[12]  Mark N. Wegman,et al.  Analysis of pointers and structures , 1990, SIGP.

[13]  Reinhard Wilhelm,et al.  Solving shape-analysis problems in languages with destructive updating , 1998, TOPL.

[14]  Laurie J. Hendren,et al.  Is it a tree, a DAG, or a cyclic graph? A shape analysis for heap-directed pointers in C , 1996, POPL '96.

[15]  H. Rice.  Classes of recursively enumerable sets and their decision problems , 1953 .

[16]  Michael Sipser,et al.  Introduction to the Theory of Computation , 1996, SIGA.

[17]  Bruno Courcelle,et al.  The Expression of Graph Properties and Graph Transformations in Monadic Second-Order Logic , 1997, Handbook of Graph Grammars.

[18]  Reinhard Wilhelm,et al.  Parametric shape analysis via 3-valued logic , 1999, POPL '99.

[19]  Michael I. Schwartzbach,et al.  The pointer assertion logic engine , 2000, PLDI '01.

[20]  Michael Benedikt,et al.  A Decidable Logic for Describing Linked Data Structures , 1999, ESOP.

[21]  P. Seymour,et al.  Surveys in combinatorics 1985: Graph minors – a survey , 1985 .

[22]  Luca Cardelli,et al.  Deciding validity in a spatial logic for trees , 2003, TLDI '03.

[23]  Daniel Jackson,et al.  Alloy: a lightweight object modelling notation , 2002, TSEM.

[24]  Sorin Lerner,et al.  ESP: path-sensitive program verification in polynomial time , 2002, PLDI '02.

[25]  David Walker,et al.  Alias Types , 2000, ESOP.

[26]  Wolfgang Thomas,et al.  Languages, Automata, and Logic , 1997, Handbook of Formal Languages.

[27]  Susan Horwitz,et al.  On the non-approximability of points-to analysis , 2002, Acta Informatica.

[28]  Flemming Nielson,et al.  Principles of Program Analysis , 1999, Springer Berlin Heidelberg.

[29]  Charles Gregory Nelson,et al.  Techniques for program verification , 1979 .

[30]  Viktor Kuncak,et al.  Role analysis , 2002, POPL '02.

[31]  Sophia Drossopoulou,et al.  Fickle : Dynamic Object Re-classification , 2001, ECOOP.

[32]  James R. Larus,et al.  Detecting conflicts between structure accesses , 1988, PLDI '88.

[33]  Hubert Comon,et al.  Tree automata techniques and applications , 1997 .

[34]  Greg Nelson,et al.  Extended static checking for Java , 2002, PLDI '02.

[35]  Robert E. Strom,et al.  Typestate: A programming language concept for enhancing software reliability , 1986, IEEE Transactions on Software Engineering.

[36]  Antonio Restivo,et al.  Two-Dimensional Languages , 1997, Handbook of Formal Languages.

[37]  Michael Sipser,et al.  Introduction to the Theory of Computation: Preliminary Edition , 1996 .

[38]  Neil Immerman,et al.  Descriptive Complexity , 1999, Graduate Texts in Computer Science.

[39]  Robert DeLine,et al.  Enforcing high-level protocols in low-level software , 2001, PLDI '01.

[40]  Pascal Fradet,et al.  Shape types , 1997, POPL '97.