Declarative specification of indentation rules: a tooling perspective on parsing and pretty-printing layout-sensitive languages

In layout-sensitive languages, the indentation of an expression or statement can influence how a program is parsed. While some of these languages (e.g., Haskell and Python) have been widely adopted, there is little support for software language engineers in building tools for layout-sensitive languages. As a result, parsers, pretty-printers, program analyses, and refactoring tools often need to be handwritten, which decreases the maintainability and extensibility of these tools. Even state-of-the-art language workbenches have little support for layout-sensitive languages, restricting the development and prototyping of such languages. In this paper, we introduce a novel approach to declarative specification of layout-sensitive languages using layout declarations. Layout declarations are high-level specifications of indentation rules that abstract from low-level technicalities. We show how to derive an efficient layout-sensitive generalized parser and a corresponding pretty-printer automatically from a language specification with layout declarations. We validate our approach in a case-study using a syntax definition for the Haskell programming language, investigating the performance of the generated parser and the correctness of the generated pretty-printer against 22191 Haskell files.

[1]  P. J. Landin,et al.  The next 700 programming languages , 1966, CACM.

[2]  Eelco Visser,et al.  An Algorithm for Layout Preservation in Refactoring Transformations , 2011, SLE.

[3]  David Walker,et al.  Semantics and algorithms for data-dependent grammars , 2010, POPL '10.

[4]  Michael J. Steindorfer,et al.  Towards Zero-Overhead Disambiguation of Deep Priority Conflicts , 2018, Art Sci. Eng. Program..

[5]  Guido van Rossum,et al.  The Python language reference manual: for Python version 3.2 , 2011 .

[6]  Anastasia Izmaylova,et al.  Operator precedence for data-dependent grammars , 2016, PEPM.

[7]  Fred L. Drake,et al.  The Python Language Reference Manual , 1999 .

[8]  Lisa F. Rubin.  Syntax-Directed Pretty Printing—A First Step Towards a Syntax-Directed Editor , 1983, IEEE Transactions on Software Engineering.

[9]  Michael D. Adams.  Principled parsing for indentation-sensitive languages: revisiting landin's offside rule , 2013, POPL.

[10]  M. Jonge,et al.  A pretty-printer for every occasion , 2001 .

[11]  Philip Wadler,et al.  A prettier printer , 2002 .

[12]  Huiqing Li,et al.  The Haskell Refactorer, HaRe, and its API , 2005, Electron. Notes Theor. Comput. Sci..

[13]  Nils Anders Danielsson.  Correct-by-construction pretty-printing , 2013, DTP '13.

[14]  Sebastian Erdweg,et al.  Layout-Sensitive Generalized Parsing , 2012, SLE.

[15]  M G J Van Den Brand,et al.  Prettyprinting without Losing Comments , 1993 .

[16]  Eelco Visser,et al.  Declarative specification of template-based textual editors , 2012, LDTA.

[17]  M G J Van Den Brand.  Generation of Language Independent Modular Prettyprinters , 1993 .

[18]  Eelco Visser,et al.  Pure and declarative syntax definition: paradise lost and regained , 2010, OOPSLA.

[19]  Eelco Visser,et al.  Generation of formatters for context-free languages , 1996, TSEM.

[20]  Eelco Visser,et al.  A Case Study in Optimizing Parsing Schemata by Disambiguation Filters , 1997, IWPT.

[21]  Eelco Visser,et al.  Scannerless Generalized-LR Parsing , 1997 .

[22]  Anastasia Izmaylova,et al.  One parser to rule them all , 2015, Onward!.