Generalising tree traversals and tree transformations to DAGs: Exploiting sharing without the pain

We present a recursion scheme based on attribute grammars that can be transparently applied to trees and acyclic graphs. Our recursion scheme allows the programmer to implement a tree traversal or a tree transformation and then apply it to compact graph representations of trees instead. The resulting graph traversal or graph transformation avoids recomputation of intermediate results for shared nodes – even if intermediate results are used in different contexts. Consequently, this approach leads to asymptotic speedup proportional to the compression provided by the graph representation. In general, however, this sharing of intermediate results is not sound. Therefore, we complement our implementation of the recursion scheme with a number of correspondence theorems that ensure soundness for various classes of traversals. We illustrate the practical applicability of the implementation as well as the complementing theory with a number of examples.

[1]  Bruno C. d. S. Oliveira,et al.  Functional programming with structured graphs , 2012, ICFP.

[2]  Witold Charatonik,et al.  Automata on DAG representations of finite trees , 1999 .

[3]  Joost Engelfriet,et al.  Macro Tree Transducers , 1985, J. Comput. Syst. Sci..

[4]  Johan Jeuring,et al.  Generic programming with fixed points for mutually recursive datatypes , 2009, ICFP.

[5]  Bruno C. d. S. Oliveira,et al.  Abstract syntax graphs for domain specific languages , 2013, PEPM '13.

[6]  Adriaan Middelkoop,et al.  Inference of Program Properties with Attribute Grammars, Revisited , 2012 .

[7]  Emil Axelsson,et al.  Generalising Tree Traversals to DAGs: Exploiting Sharing without the Pain , 2015, PEPM.

[8]  Simon L. Peyton Jones,et al.  Closed type families with overlapping equations , 2014, POPL.

[9]  Richard S. Bird Using circular programs to eliminate multiple traversals of data , 2004, Acta Informatica.

[10]  Akio Fujiyoshi,et al.  Recognition of directed acyclic graphs by spanning tree automata , 2010, Theor. Comput. Sci..

[11]  João Saraiva,et al.  Functional Incremental Attribute Evaluation , 2000, CC.

[12]  S. Doaitse Swierstra,et al.  First-class Attribute Grammars , 2000, Informatica.

[13]  Görel Hedin,et al.  Circular reference attributed grammars - their evaluation and applications , 2003, Sci. Comput. Program..

[14]  Andy Gill,et al.  Type-safe observable sharing in Haskell , 2009, Haskell.

[15]  Benjamin C. Pierce,et al.  Local type inference , 1998, POPL '98.

[16]  Ayumi Shinohara,et al.  Functional programs as compressed data , 2012, PEPM '12.

[17]  Donald E. Knuth,et al.  Semantics of context-free languages , 1968, Mathematical systems theory.

[18]  Soichiro Hidaka,et al.  Structural recursion for querying ordered graphs , 2013, ICFP.

[19]  Dan Suciu,et al.  UnQL: a query language and algebra for semistructured data based on structural recursion , 2000, The VLDB Journal.

[20]  Mads Rosendahl,et al.  Abstract Interpretation Using Attribute Grammar , 1990, WAGA.

[21]  John Launchbury,et al.  Implicit parameters: dynamic scoping with static types , 2000, POPL '00.

[22]  Patrick Bahr,et al.  Programming macro tree transducers , 2013, WGP '13.

[23]  Joost Engelfriet,et al.  Bottom-up and top-down tree transformations— a comparison , 1975, Mathematical systems theory.

[24]  Mads Rosendahl,et al.  Abstract Interpretation using Attribute Grammars , 1990 .

[25]  Robert E. Tarjan,et al.  Variations on the Common Subexpression Problem , 1980, J. ACM.

[26]  Alfred V. Aho,et al.  Compilers: Principles, Techniques, and Tools , 1986, Addison-Wesley series in computer science / World student series edition.

[27]  Zoltán Horváth,et al.  Feldspar: A domain specific language for digital signal processing algorithms , 2010, Eighth ACM/IEEE International Conference on Formal Methods and Models for Codesign (MEMOCODE 2010).

[28]  Maarten M. Fokkinga,et al.  Functional Programming with Bananas, Lenses, Envelopes and Barbed Wire , 1991, FPCA.

[29]  S. Doaitse Swierstra,et al.  Abstract Interpretation of Functional Programs using an Attribute Grammar System , 2009, Electron. Notes Theor. Comput. Sci..

[30]  Simon L. Peyton Jones,et al.  Secrets of the Glasgow Haskell Compiler inliner , 2002, Journal of Functional Programming.

[31]  Emil Axelsson,et al.  Functional Programming Enabling Flexible Hardware Design at Low Levels of Abstraction , 2008 .

[32]  Torbjörn Ekman,et al.  The JastAdd system - modular extensible compiler construction , 2007, Sci. Comput. Program..

[33]  Anthony M. Sloane,et al.  A pure embedding of attribute grammars , 2013, Sci. Comput. Program..

[34]  Abhi Shelat,et al.  The smallest grammar problem , 2005, IEEE Transactions on Information Theory.

[35]  Bart Jacobs,et al.  Categorical Views on Computations on Trees (Extended Abstract) , 2007, ICALP.

[37]  Sorin Lerner,et al.  Composing dataflow analyses and transformations , 2002, POPL '02.

[38]  Martin Odersky,et al.  Lightweight modular staging: a pragmatic approach to runtime code generation and compiled DSLs , 2010, GPCE '10.

[39]  J. Gibbons,et al.  Modular Tree Automata , 2012 .

[40]  Patricia Johann,et al.  Foundations for structured programming with GADTs , 2008, POPL '08.

[41]  Sebastian Maneth,et al.  The complexity of tree automata and XPath on grammar-compressed trees , 2006, Theor. Comput. Sci..

[42]  Oleg Kiselyov Implementing Explicit and Finding Implicit Sharing in Embedded DSLs , 2011, DSL.

[43]  Sebastian Maneth,et al.  Tree Transducers and Tree Compressions , 2004, FoSSaCS.

[44]  Zoltán Fülöp,et al.  Syntax-Directed Semantics: Formal Models Based on Tree Transducers , 2011 .

[45]  Adam Chlipala Parametric higher-order abstract syntax for mechanized semantics , 2008, ICFP 2008.

[46]  Jeremy Gibbons Generic downwards accumulations , 2000, Sci. Comput. Program..

[47]  Gad M. Landau,et al.  Tree compression with top trees , 2015, Inf. Comput..

[48]  Tom Schrijvers,et al.  Understanding idiomatic traversals backwards and forwards , 2013, Haskell '13.

[49]  Sebastian Maneth,et al.  Efficient memory representation of XML document trees , 2008, Inf. Syst..

[50]  Donald R. Morrison,et al.  PATRICIA—Practical Algorithm To Retrieve Information Coded in Alphanumeric , 1968, J. ACM.

[51]  Sherif Sakr,et al.  XML compression techniques: A survey and comparison , 2009, J. Comput. Syst. Sci..

[52]  Dhananjay M. Dhamdhere,et al.  A generalized theory of bit vector data flow analysis , 1994, TOPL.

[53]  S. Doaitse Swierstra,et al.  Higher order attribute grammars , 1989, PLDI '89.

[54]  Görel Hedin,et al.  Reference Attributed Grammars , 2000, Informatica.

[55]  Michaël Rusinowitch,et al.  Closure properties and decision problems of dag automata , 2005, Inf. Process. Lett..

[56]  Thorsten Altenkirch,et al.  Foundations of Software Science and Computation Structures: 6th International Conference, FOSSACS 2003 Held as Part of the Joint European Conferences on Theory and Practice of Software, ETAPS 2003 Warsaw, Poland, April 7–11, 2003 Proceedings , 2003, Lecture Notes in Computer Science.

[57]  S. D. Swierstra,et al.  Using attribute grammars to derive efficient functional programs , 1986 .

[58]  M. Barr,et al.  Toposes, Triples and Theories , 1984 .

[59]  Bart Jacobs,et al.  Categorical views on computations on trees , 2007 .

[60]  Kevin Knight,et al.  DAGGER: A Toolkit for Automata on Directed Acyclic Graphs , 2012, FSMNLP.

[61]  S. Doaitse Swierstra,et al.  Attribute grammars fly first-class: how to do aspect oriented programming in Haskell , 2009, ICFP.

[62]  Giora Slutzki,et al.  Transductions of dags and trees , 2005, Mathematical systems theory.

[63]  Donald E. Knuth Semantics of context-free languages: Correction , 2005, Mathematical systems theory.

[64]  Görel Hedin,et al.  JavaRAG: a Java library for reference attribute grammars , 2015, MODULARITY.

[65]  Patrick Bahr Proving Correctness of Compilers Using Structured Graphs , 2014, FLOPS.

[66]  Dhananjay M. Dhamdhere,et al.  Bidirectional data flow analysis for type inferencing , 2003, Comput. Lang. Syst. Struct..