First Class Syntax, Semantics, and Their Composition

Ideally complexity is managed by composing a system out of quite a few, more or less independent, and much smaller descriptions of various aspects of the overall artefact. When describing (extensible) programming languages, attribute grammars have turned out to be an excellent tool for modular definition and integration of their different aspects. In this paper we show how to construct a programming language implementation by composing a collection of separately compiled attribute grammar fragments, each describing a separate aspect of the language. More specifically we describe how to use a coherent set of libraries and tools which together makes it possible to express this directly in Haskell, where the correctness of the composition is enforced through the Haskell type system's ability to represent attribute grammars as plain Haskell values and their interfaces as Haskell types makes this possible. Semantic objects thus constructed can be combined with parsers which are constructed on the fly out of a collection of grammar fragments, which are also represented by typed Haskell values. Again the type checker prevents unsound compositions. Using a very small example language and some simple extensions, we show how our techniques fit together towards the construction of extensible compilers out of a collection of pre-compiled, statically type-checked "language definition fragments".

[1]  Viljem Zumer,et al.  Incremental programming language development , 2005, Comput. Lang. Syst. Struct..

[2]  Bastiaan Heeren,et al.  Top quality type error Messages , 2005 .

[3]  Mark P. Jones,et al.  Typing Haskell in Haskell , 1999 .

[4]  S. Doaitse Swierstra,et al.  Typing Haskell with an Attribute Grammar , 2004, Advanced Functional Programming.

[5]  Uwe Kastens,et al.  Ordered attributed grammars , 1980, Acta Informatica.

[6]  Daniel Weise,et al.  Programmable syntax macros , 1993, PLDI '93.

[7]  Robert Hieb,et al.  Revised 5 Report on the Algorithmic Language , 1999 .

[8]  Eric Van Wyk,et al.  Silver: an Extensible Attribute Grammar System , 2008, Electron. Notes Theor. Comput. Sci..

[9]  Stephen R. Adams.  Modular grammars for programming language prototyping , 1991 .

[10]  S. Doaitse Swierstra,et al.  The Structure of the Essential Haskell Compiler, or Coping with Compiler Complexity , 2007, IFL.

[11]  Simon L. Peyton Jones,et al.  Aspect-Oriented Compilers , 1999, GCSE.

[12]  William Maddox.  Semantically-Sensitive Macroprocessing , 1989 .

[13]  Simon L. Peyton Jones,et al.  Scrap more boilerplate: reflection, zips, and generalised casts , 2004, ICFP '04.

[14]  Burt M. Leavenworth,et al.  Syntax macros and extended translation , 1966, CACM.

[15]  John C. Reynolds.  User-defined types and procedural data structures as complementary approaches to data abstraction , 1994 .

[16]  William M. Waite,et al.  Modularity and reusability in attribute grammars , 1994, Acta Informatica.

[17]  John Tang Boyland,et al.  Remote attribute grammars , 2005, JACM.

[18]  David Fisher,et al.  Static analysis for syntax objects , 2006, ICFP '06.

[19]  S. Doaitse Swierstra,et al.  Typing dynamic typing , 2002, ICFP '02.

[20]  Simon L. Peyton Jones,et al.  Template meta-programming for Haskell , 2002, Haskell '02.

[21]  S. Doaitse Swierstra,et al.  Combinator Parsing: A Short Tutorial , 2009, LerNet ALFA Summer School.

[22]  Alexandra Silva,et al.  Strong types for relational databases , 2006, Haskell '06.

[23]  Conor McBride.  Faking it: Simulating dependent types in Haskell , 2002, J. Funct. Program..

[24]  Alberto Pardo,et al.  Just do it while compiling!: fast extensible records in haskell , 2013, PEPM '13.

[25]  Harvey Abramson,et al.  Definite Clause Translation Grammars , 1984, SLP.

[26]  Torbjörn Ekman,et al.  The JastAdd system - modular extensible compiler construction , 2007, Sci. Comput. Program..

[27]  Mark P. Jones,et al.  Language and program design for functional dependencies , 2008, Haskell '08.

[28]  S. Doaitse Swierstra,et al.  Attribute grammars fly first-class: how to do aspect oriented programming in Haskell , 2009, ICFP.

[29]  S. Doaitse Swierstra,et al.  Parsing permutation phrases , 2004, J. Funct. Program..

[30]  Mark Johnson,et al.  Finite-state Approximation of Constraint-based Grammars using Left-corner Grammar Transforms , 1998, ACL.

[31]  Anthony M. Sloane,et al.  A Pure Object-Oriented Embedding of Attribute Grammars , 2010, LDTA.

[32]  Martin Bravenboer,et al.  Exercises in Free Syntax. Syntax Definition, Parsing, and Assimilation of Language Conglomerates , 2003 .

[33]  S. Doaitse Swierstra,et al.  Haskell, do you read me?: constructing and composing efficient top-down parsers at runtime , 2008, Haskell '08.

[34]  Richard S. Bird.  Using circular programs to eliminate multiple traversals of data , 2004, Acta Informatica.

[35]  Atze Dijkstra.  Stepping through Haskell , 2000 .

[36]  S. Doaitse Swierstra,et al.  Iterative type inference with attribute grammars , 2010, GPCE '10.

[37]  Simon L. Peyton Jones,et al.  Simple unification-based type inference for GADTs , 2006, ICFP '06.

[38]  Philip Wadler,et al.  Idioms are Oblivious, Arrows are Meticulous, Monads are Promiscuous , 2011, Electron. Notes Theor. Comput. Sci..

[39]  S. Doaitse Swierstra,et al.  Grammar fragments fly first-class , 2012, LDTA.

[40]  Eelco Visser,et al.  Stratego/XT 0.17. A language and toolset for program transformation , 2008, Sci. Comput. Program..

[41]  Ralf Lämmel,et al.  Strongly typed heterogeneous collections , 2004, Haskell '04.

[42]  S. Doaitse Swierstra,et al.  Attribute grammar macros , 2014, Sci. Comput. Program..

[43]  Harold Abelson,et al.  Revised5 report on the algorithmic language scheme , 1998, SIGP.

[44]  Mark P. Jones,et al.  Functional Programming with Overloading and Higher-Order Polymorphism , 1995, Advanced Functional Programming.

[45]  Marc A. de Kruijf.  Compiler Construction , 1996, Lecture Notes in Computer Science.

[46]  Wouter Swierstra,et al.  The power of Pi , 2008, ICFP 2008.

[47]  Simon L. Peyton Jones,et al.  Scrap your boilerplate: a practical design pattern for generic programming , 2003, TLDI '03.

[48]  David Fisher,et al.  Building language towers with Ziggurat , 2008, Journal of Functional Programming.

[49]  Ulf Norell.  Dependently typed programming in Agda , 2009, TLDI '09.

[50]  S. D. Swierstra,et al.  Using attribute grammars to derive efficient functional programs , 1986 .

[51]  R. Kent Dybvig,et al.  Revised5 Report on the Algorithmic Language Scheme , 1986, SIGP.

[52]  Ralf Hinze,et al.  Fun with phantom types , 2003 .

[53]  Peyton Jones,et al.  Haskell 98 language and libraries : the revised report , 2003 .

[54]  Peter D. Mosses,et al.  Denotational semantics , 1995, LICS 1995.

[55]  Eric Van Wyk,et al.  Forwarding in Attribute Grammars for Modular Language Design , 2002, CC.

[56]  Jurriaan Hage,et al.  Scripting the type inference process , 2003, ICFP '03.

[57]  S. Doaitse Swierstra,et al.  The architecture of the Utrecht Haskell compiler , 2009, Haskell.

[58]  Mark P. Jones,et al.  Type Classes with Functional Dependencies , 2000, ESOP.

[59]  Dominique Devriese,et al.  Explicitly Recursive Grammar Combinators - A Better Model for Shallow Parser DSLs , 2011, PADL.

[60]  S. Doaitse Swierstra,et al.  Compositional compiler construction: Oberon0 , 2015, Sci. Comput. Program..

[61]  Eelco Visser,et al.  Grammar Engineering Support for Precedence Rule Recovery and Compatibility Checking , 2007, LDTA@ETAPS.

[62]  Emir Pasalic,et al.  Meta-programming with Typed Object-Language Representations , 2004, GPCE.

[63]  Johan Jeuring,et al.  Generic programming with fixed points for mutually recursive datatypes , 2009, ICFP.

[64]  João Saraiva,et al.  Designing and Implementing Combinator Languages , 1998, Advanced Functional Programming.

[65]  Wouter Swierstra,et al.  Data types à la carte , 2008, Journal of Functional Programming.

[66]  Ross Paterson,et al.  A new notation for arrows , 2001, ICFP '01.

[67]  Simon Peyton Jones,et al.  Type classes: an exploration of the design space , 1997 .

[68]  Apostolos Syropoulos,et al.  Steps in Scala: The expression problem , 2010 .

[69]  S. Doaitse Swierstra,et al.  First-class Attribute Grammars , 2000, Informatica.

[70]  Andres Löh,et al.  Dependently Typed Grammars , 2010, MPC.

[71]  Simon L. Peyton Jones,et al.  Associated type synonyms , 2005, ICFP '05.

[72]  S. Doaitse Swierstra,et al.  UUAG meets AspectAG: how to make attribute grammars first-class , 2012, LDTA.

[73]  S. Doaitse Swierstra,et al.  Abstract Interpretation of Functional Programs using an Attribute Grammar System , 2009, Electron. Notes Theor. Comput. Sci..

[74]  Bruno C. d. S. Oliveira,et al.  Monads, zippers and views: virtualizing the monad stack , 2011, ICFP '11.

[75]  Geoffrey Mainland.  Why it's nice to be quoted: quasiquoting for haskell , 2007, Haskell '07.

[76]  Thomas Johnsson,et al.  Attribute grammars as a functional programming paradigm , 1987, FPCA.

[77]  S. Doaitse Swierstra,et al.  Typed Transformations of Typed Grammars: The Left Corner Transform , 2010, Electron. Notes Theor. Comput. Sci..

[78]  Anthony M. Sloane,et al.  Eli: a complete, flexible compiler construction system , 1992, CACM.

[79]  Thomas W. Reps,et al.  Remote attribute updating for language-based editors , 1986, POPL '86.

[80]  Conor McBride,et al.  Applicative programming with effects , 2008, J. Funct. Program..

[81]  John Hughes,et al.  Generalising monads to arrows , 2000, Sci. Comput. Program..

[82]  S. Doaitse Swierstra,et al.  Type-safe, self inspecting code , 2004, Haskell '04.

[83]  S. Doaitse Swierstra,et al.  Linear, bounded, functional pretty-printing , 2009, J. Funct. Program..

[84]  Benedict R. Gaster,et al.  A Polymorphic Type System for Extensible Records and Variants , 1996 .

[85]  S. Doaitse Swierstra,et al.  Typed transformations of typed abstract syntax , 2009, TLDI '09.