SugarJ: library-based syntactic language extensibility

Existing approaches to extend a programming language with syntactic sugar often leave a bitter taste, because they cannot be used with the same ease as the main extension mechanism of the programming language - libraries. Sugar libraries are a novel approach for syntactically extending a programming language within the language. A sugar library is like an ordinary library, but can, in addition, export syntactic sugar for using the library. Sugar libraries maintain the composability and scoping properties of ordinary libraries and are hence particularly well-suited for embedding a multitude of domain-specific languages into a host language. They also inherit self-applicability from libraries, which means that sugar libraries can provide syntactic extensions for the definition of other sugar libraries. To demonstrate the expressiveness and applicability of sugar libraries, we have developed SugarJ, a language on top of Java, SDF and Stratego, which supports syntactic extensibility. SugarJ employs a novel incremental parsing technique, which allows changing the syntax within a source file. We demonstrate SugarJ by five language extensions, including embeddings of XML and closures in Java, all available as sugar libraries. We illustrate the utility of self-applicability by embedding XML Schema, a metalanguage to define XML languages.

[1]  Robert Bruce Findler,et al.  Scribble: closing the book on ad hoc documentation tools , 2009, ICFP.

[2]  John N. Shutt.  Recursive Adaptable Grammars , 1999 .

[3]  Mario Kolberg,et al.  Feature interaction: a critical review and considered forecast , 2003, Comput. Networks.

[4]  Laurence Tratt,et al.  Domain specific language implementation via compile-time meta-programming , 2008, TOPL.

[5]  C. M. Sperberg-McQueen,et al.  Extensible Markup Language (XML) , 1997, World Wide Web J..

[6]  Torbjörn Ekman,et al.  The jastadd extensible java compiler , 2007, OOPSLA.

[7]  Michael R. Clarkson,et al.  Polyglot: An Extensible Compiler Framework for Java , 2003, CC.

[8]  Martin Lange,et al.  Analyzing Context-Free Grammars Using an Incremental SAT Solver , 2008, ICALP.

[9]  Martin P. Ward.  Language-Oriented Programming , 1994, Softw. Concepts Tools.

[10]  Tom Mens,et al.  Detecting Structural Refactoring Conflicts Using Critical Pair Analysis , 2005, SETra@ICGT.

[11]  Eelco Visser,et al.  Disambiguation Filters for Scannerless Generalized LR Parsers , 2002, CC.

[12]  Eric Van Wyk,et al.  Silver: An extensible attribute grammar system , 2008, Sci. Comput. Program..

[13]  Daniel Weise,et al.  Programmable syntax macros , 1993, PLDI '93.

[14]  Michael I. Schwartzbach,et al.  An introduction to XML and web technologies , 2005 .

[15]  Jonathan Bachrach,et al.  The Java syntactic extender (JSE) , 2001, OOPSLA '01.

[16]  Yannis Smaragdakis,et al.  JTS: tools for implementing domain-specific languages , 1998, Proceedings. Fifth International Conference on Software Reuse (Cat. No.98TB100203).

[17]  Sebastian Erdweg,et al.  Featherweight TeX and Parser Correctness , 2010, SLE.

[18]  M Mernik,et al.  When and how to develop domain-specific languages , 2005, CSUR.

[19]  Claus Brabrand,et al.  Growing languages with metamorphic syntax macros , 2000, PEPM '02.

[20]  Jonathan Rees,et al.  Macros that work , 1991, POPL '91.

[21]  Burt M. Leavenworth,et al.  Syntax macros and extended translation , 1966, CACM.

[22]  Eelco Visser,et al.  The spoofax language workbench: rules for declarative specification of languages and IDEs , 2010, OOPSLA.

[23]  Christian Hofer,et al.  Modular domain-specific language components in scala , 2010, GPCE '10.

[24]  Sam Tobin-Hochstadt,et al.  Languages as libraries , 2011, PLDI '11.

[25]  Gilad Bracha.  Pluggable Type Systems , 2004 .

[26]  Yoshihito Toyama,et al.  Proving Confluence of Term Rewriting Systems Automatically , 2009, RTA.

[27]  Eelco Visser,et al.  Growing a language environment with editor libraries , 2011, GPCE '11.

[28]  Charles Simonyi,et al.  The Death of Computer Languages, The Birth of Intentional Programming , 1995 .

[29]  Eelco Visser,et al.  Preventing injection attacks with syntax embeddings , 2007, GPCE '07.

[30]  Shriram Krishnamurthi,et al.  EDUCATIONAL PEARL: Automata via macros , 2006, Journal of Functional Programming.

[31]  Luca Cardelli,et al.  Program fragments, linking, and modularization , 1997, POPL '97.

[32]  Chris Seaton,et al.  A Programming Language Where the Syntax and Semantics Are Mutable at Runtime , 2007 .

[33]  Sylvain Schmitz,et al.  Conservative Ambiguity Detection in Context-Free Grammars , 2007, ICALP.

[34]  Paul J. Walmsley,et al.  XML Schema Part 0: Primer Second Edition , 2004 .

[35]  Eelco Visser,et al.  Concrete syntax for objects: domain-specific language embedding and assimilation without restrictions , 2004, OOPSLA '04.

[36]  Bruno C. d. S. Oliveira,et al.  Modular Visitor Components , 2009, ECOOP.

[37]  Eric Van Wyk,et al.  Attribute Grammar-Based Language Extensions for Java , 2007, ECOOP.

[38]  S. Dmitriev.  Language Oriented Programming: The Next Programming Paradigm , 2004 .

[39]  Shigeru Chiba,et al.  OpenJava: A Class-Based Macro System for Java , 1999, Reflection and Software Engineering.

[40]  Oleg Kiselyov.  Macros That Compose: Systematic Macro Programming , 2002, GPCE.

[41]  Paul Hudak,et al.  Modular domain specific languages and tools , 1998, Proceedings. Fifth International Conference on Software Reuse (Cat. No.98TB100203).

[42]  Eelco Visser,et al.  Stratego: A Language for Program Transformation Based on Rewriting Strategies , 2001, RTA.

[43]  Eelco Visser,et al.  Meta-programming with Concrete Object Syntax , 2002, GPCE.

[44]  Paul Klint,et al.  The syntax definition formalism SDF—reference manual— , 1989, SIGP.

[45]  Matthias Felleisen,et al.  Hygienic macro expansion , 1986, LFP '86.

[46]  Eric E. Allen,et al.  Growing a Syntax , 2008 .

[47]  Christian Hofer,et al.  Polymorphic embedding of dsls , 2008, GPCE '08.

[48]  Matthew Flatt.  Composable and compilable macros:: you want it when? , 2002, ICFP '02.

[49]  Oscar Nierstrasz,et al.  Embedding Languages without Breaking Tools , 2010, ECOOP.