Pure and declarative syntax definition: paradise lost and regained

Syntax definitions are pervasive in modern software systems, and serve as the basis for language processing tools like parsers and compilers. Mainstream parser generators pose restrictions on syntax definitions that follow from their implementation algorithm. They hamper evolution, maintainability, and compositionality of syntax definitions. The pureness and declarativity of syntax definitions is lost. We analyze how these problems arise for different aspects of syntax definitions, discuss their consequences for language engineers, and show how the pure and declarative nature of syntax definitions can be regained.

[1]  Eelco Visser,et al.  Declarative, formal, and extensible syntax definition for aspectJ , 2006, OOPSLA '06.

[2]  Ralf Lämmel,et al.  Towards an engineering discipline for GRAMMARWARE Draft as of August 17 , 2003 , 2003 .

[3]  Murray Hill,et al.  Yacc: Yet Another Compiler-Compiler , 1978 .

[4]  Bryan Ford,et al.  Parsing expression grammars: a recognition-based syntactic foundation , 2004, POPL '04.

[5]  Paul Klint,et al.  The syntax definition formalism SDF—reference manual— , 1989, SIGP.

[6]  Alfred V. Aho,et al.  Deterministic parsing of ambiguous grammars , 1975, Commun. ACM.

[7]  Donald E. Knuth,et al.  On the Translation of Languages from Left to Right , 1965, Inf. Control..

[8]  Richard Edwin Stearns,et al.  Syntax-Directed Transduction , 1966, JACM.

[9]  Masaru Tomita,et al.  Efficient Parsing for Natural Language: A Fast Algorithm for Practical Systems , 1985 .

[10]  Peter M. Maurer,et al.  Generating test data with enhanced context-free grammars , 1990, IEEE Software.

[11]  P. Klint,et al.  Efficient annotated terms , 2000 .

[12]  Eelco Visser,et al.  Scannerless Generalized-LR Parsing , 1997 .

[13]  Eelco Visser,et al.  Grammar Engineering Support for Precedence Rule Recovery and Compatibility Checking , 2007, LDTA@ETAPS.

[14]  Daniel P. Friedman,et al.  Aspect-Oriented Programming is Quantification and Obliviousness , 2000 .

[15]  Eelco Visser,et al.  The spoofax language workbench: rules for declarative specification of languages and IDEs , 2010, OOPSLA.

[16]  Eelco Visser,et al.  Generation of formatters for context-free languages , 1996, TSEM.

[17]  Eelco Visser,et al.  Syntax definition for language prototyping , 1997 .

[18]  J. Rekers,et al.  Parser Generation for Interactive Environments , 1992 .

[19]  Eelco Visser,et al.  A Case Study in Optimizing Parsing Schemata by Disambiguation Filters , 1997, IWPT.

[20]  Charles B. Haley,et al.  Practical LR error recovery , 1979, SIGPLAN '79.

[21]  Bryan Ford,et al.  Packrat parsing:: simple, powerful, lazy, linear time, functional pearl , 2002, ICFP '02.

[22]  Akim Demaille,et al.  C-Transformers: a framework to write C program transformations , 2006, CROS.

[23]  Eelco Visser,et al.  Disambiguation Filters for Scannerless Generalized LR Parsers , 2002, CC.

[24]  Jean-Marie Favre.  Languages evolve too! Changing the software time scale , 2005, Eighth International Workshop on Principles of Software Evolution (IWPSE'05).

[25]  David G. Cantor,et al.  On The Ambiguity Problem of Backus Systems , 1962, JACM.

[26]  James R. Cordy,et al.  TXL: A Rapid Prototyping System for Programming Language Dialects , 1991, Comput. Lang..

[27]  Eelco Visser,et al.  Natural and Flexible Error Recovery for Generated Parsers , 2009, SLE.

[28]  F. L. Deremer,et al.  Practical translators for LR(k) languages , 1969 .

[29]  Eelco Visser,et al.  Providing rapid feedback in generated modular language environments: adding error recovery to scannerless generalized-LR parsing , 2009, OOPSLA 2009.

[30]  John Cocke,et al.  Programming languages and their compilers: Preliminary notes , 1969 .

[31]  Brian A. Malloy,et al.  Applying software engineering techniques to parser design: the development of a C # parser , 2002 .

[32]  Eelco Visser,et al.  Meta-programming with Concrete Object Syntax , 2002, GPCE.

[33]  Eelco Visser,et al.  The Spoofax language workbench , 2010, SPLASH/OOPSLA Companion.

[34]  Alfred V. Aho,et al.  A Minimum Distance Error-Correcting Parser for Context-Free Languages , 1972, SIAM J. Comput..

[35]  Russell W. Quong,et al.  LL and LR translators need k>1 lookahead , 1996, SIGP.

[36]  Daniel H. Younger,et al.  Recognition and Parsing of Context-Free Languages in Time n^3 , 1967, Inf. Control..

[37]  Eelco Visser,et al.  Using Filters for the Disambiguation of Context-free Grammars , 1994 .

[38]  Adrian Johnstone,et al.  GLL Parsing , 2010, LDTA.

[39]  Gordon V. Cormack,et al.  Scannerless NSLR(1) parsing of programming languages , 1989, PLDI '89.

[40]  Saul Gorn.  Detection of Generative Ambiguities in Context-Free Mechanical Languages , 1963, JACM.

[41]  Jeffrey D. Ullman,et al.  Parsing Algorithms with Backtrack , 1970, SWAT.

[42]  M. Jonge,et al.  A pretty-printer for every occasion , 2001 .

[43]  Adrian Johnstone,et al.  Evaluating GLR parsing algorithms , 2006, Sci. Comput. Program..

[44]  Joseph S. Ullian,et al.  Ambiguity in context free languages , 1966, JACM.

[45]  Charles N. Fischer,et al.  Crafting a Compiler , 1988 .

[46]  C L KatsLennart,et al.  Providing rapid feedback in generated modular language environments , 2009 .

[47]  Eelco Visser,et al.  Preventing injection attacks with syntax embeddings , 2007, GPCE '07.

[48]  Paul Klint,et al.  Efficient annotated terms , 2000, Softw. Pract. Exp..

[49]  Tadao Kasami,et al.  An Efficient Recognition and Syntax-Analysis Algorithm for Context-Free Languages , 1965 .

[50]  Jay Earley,et al.  An efficient context-free parsing algorithm , 1970, Commun. ACM.