Higher-order and Symbolic Computation Manuscript No. Scala-virtualized: Linguistic Reuse for Deep Embeddings

Scala-Virtualized extends the Scala language to better support hosting embedded DSLs. Scala is an expressive language that provides a flexible syntax, type-level computation using implicits, and other features that facilitate the development of embedded DSLs. However, many of these features work well only for shallow embeddings, i.e. DSLs which are implemented as plain libraries. Shallow embeddings automatically profit from features of the host language through linguistic reuse: any DSL expression is just as a regular Scala expression. But in many cases, directly executing DSL programs within the host language is not enough and deep embeddings are needed, which reify DSL programs into a data structure representation that can be analyzed, optimized, or further translated. For deep embeddings, linguistic reuse is no longer automatic.Scala-Virtualized defines many of the language’s built-in constructs as method calls, which enables DSLs to redefine the built-in semantics using familiar language mechanisms like overloading and overriding. This in turn enables an easier progression from shallow to deep embeddings, as core language constructs such as conditionals or pattern matching can be redefined to build a reified representation of the operation itself.While this facility brings shallow, syntactic, reuse to deep embeddings, we also present examples of what we call deep linguistic reuse: combining shallow and deep components in a single DSL in such a way that certain features are fully implemented in the shallow embedding part and do not need to be reified at the deep embedding level.

[1]  Kei Davis,et al.  ROSE: An Optimizing Transformation System for C++ Array-Class Libraries , 1998, ECOOP Workshops.

[2]  Martin Odersky,et al.  Scalable component abstractions , 2005, OOPSLA '05.

[3]  Steve Karmesin,et al.  Array Design and Expression Evaluation in POOMA II , 1998, ISCOPE.

[4]  Tiark Rompf,et al.  Jet: An Embedded DSL for High Performance Big Data Processing , 2012 .

[5]  Philip Wadler,et al.  How to make ad-hoc polymorphism less ad hoc , 1989, POPL '89.

[6]  Martin Odersky,et al.  Matching Objects with Patterns , 2007, ECOOP.

[7]  Martin Odersky,et al.  JavaScript as an Embedded DSL , 2012, ECOOP.

[8]  Philip Wadler,et al.  Comprehending monads , 1990, LISP and Functional Programming.

[9]  Tiark Rompf Lightweight Modular Staging and Embedded Compilers , 2012 .

[10]  Sebastian Erdweg,et al.  SugarJ: library-based syntactic language extensibility , 2011, OOPSLA '11.

[11]  Kunle Olukotun,et al.  Language virtualization for heterogeneous parallel computing , 2010, OOPSLA.

[12]  Ralf Hinze,et al.  Haskell 98 — A Non−strict‚ Purely Functional Language , 1999 .

[13]  Martin Odersky,et al.  Lightweight modular staging: a pragmatic approach to runtime code generation and compiled DSLs , 2010, GPCE '10.

[14]  Mary Sheeran,et al.  The Design and Implementation of Feldspar - An Embedded Language for Digital Signal Processing , 2010, IFL.

[15]  Norman Ramsey,et al.  Stochastic lambda calculus and monads of probability distributions , 2002, POPL '02.

[16]  Kunle Olukotun,et al.  OptiML: An Implicitly Parallel Domain-Specific Language for Machine Learning , 2011, ICML.

[17]  Kunle Olukotun,et al.  Building-Blocks for Performance Oriented DSLs , 2011, DSL.

[18]  Sam Tobin-Hochstadt,et al.  The design and implementation of typed scheme , 2008, POPL '08.

[19]  Kunle Olukotun,et al.  A Heterogeneous Parallel Framework for Domain-Specific Languages , 2011, 2011 International Conference on Parallel Architectures and Compilation Techniques.

[20]  Kunle Olukotun,et al.  Optimizing data structures in high-level programs: new directions for extensible compilers based on staging , 2013, POPL.

[21]  Chung-chieh Shan,et al.  Embedded Probabilistic Programming , 2009, DSL.

[22]  Guy L. Steele,et al.  Growing a Language , 1999, High. Order Symb. Comput..

[23]  Joe aRmstRonG,et al.  Erlang , 2010, Commun. ACM.

[24]  Martin Odersky,et al.  Scala Macros, a Technical Report , 2012 .

[25]  Walid Taha,et al.  MetaML and multi-stage programming with explicit annotations , 2000, Theor. Comput. Sci..

[26]  Eugenio Moggi,et al.  Notions of Computation and Monads , 1991, Inf. Comput..

[27]  Matthias Felleisen,et al.  Linguistic reuse , 2001 .

[28]  Sebastian Erdweg,et al.  Language composition untangled , 2012, LDTA.

[29]  Simon L. Peyton Jones,et al.  Template meta-programming for Haskell , 2002, Haskell '02.

[30]  Martin Odersky,et al.  Fighting bit Rot with Types (Experience Report: Scala Collections) , 2009, FSTTCS.

[31]  Kunle Olukotun,et al.  Implementing Domain-Specific Languages for Heterogeneous Parallel Computing , 2011, IEEE Micro.

[32]  Sam Tobin-Hochstadt,et al.  Languages as libraries , 2011, PLDI '11.

[33]  Walid Taha,et al.  A Sound Reduction Semantics for Untyped CBN Multi-stage Computation. Or, the Theory of MetaML is Non-trivial (Extended Abstract) , 1999, PEPM.

[34]  Martin Odersky,et al.  Programming in Scala , 2008 .

[35]  Andrzej Filinski,et al.  Representing monads , 1994, POPL '94.

[36]  Walid Taha,et al.  A monadic approach for avoiding code duplication when staging memoized functions , 2006, PEPM '06.

[37]  Martin Odersky,et al.  Scala Actors: Unifying thread-based and event-based programming , 2009, Theor. Comput. Sci..

[38]  Martin Odersky,et al.  StagedSAC: a case study in performance-oriented DSL development , 2012, PEPM '12.

[39]  Robert P. Goldberg,et al.  Formal requirements for virtualizable third generation architectures , 1973, SOSP 1973.

[40]  Scott W. Haney,et al.  PETE: THE PORTABLE EXPRESSION TEMPLATE ENGINE , 1999 .

[41]  Kei Davis,et al.  C++ expression template performance issues in scientific computing , 1998, Proceedings of the First Merged International Parallel Processing Symposium and Symposium on Parallel and Distributed Processing.

[42]  Paul Hudak,et al.  Building domain-specific embedded languages , 1996, CSUR.

[43]  Jacques Carette,et al.  Finally tagless, partially evaluated: Tagless staged interpreters for simpler typed languages , 2007, Journal of Functional Programming.

[44]  Martin Odersky,et al.  Scala-virtualized , 2012, PEPM '12.

[45]  Martin Odersky,et al.  Implementing first-class polymorphic delimited continuations by a type-directed selective CPS-transform , 2009, ICFP.

[46]  Sergey Yekhanin,et al.  Private information retrieval , 2010, CACM.

[47]  Martin Odersky,et al.  Parser combinators in Scala , 2008 .

[48]  Jeremy G. Siek,et al.  The Matrix Template Library: A Generic Programming Approach to High Performance Numerical Linear Algebra , 1998, ISCOPE.

[49]  Martin Odersky,et al.  Type classes as objects and implicits , 2010, OOPSLA.

[50]  Christian Hofer,et al.  Polymorphic embedding of dsls , 2008, GPCE '08.

[51]  Andrzej Filinski Monads in action , 2010, POPL '10.

[52]  Gilles Dubochet Embedded Domain-Specific Languages using Libraries and Dynamic Metaprogramming , 2011 .

[53]  P. J. Landin,et al.  The next 700 programming languages , 1966, CACM.

[54]  Martin Odersky,et al.  Programming in Scala: A Comprehensive Step-by-Step Guide, 2nd Edition , 2010 .

[55]  Todd L. Veldhuizen,et al.  Expression templates , 1996 .

[56]  Eelco Visser,et al.  Concrete syntax for objects: domain-specific language embedding and assimilation without restrictions , 2004, OOPSLA '04.

[57]  Jeremy G. Siek General purpose languages should be metalanguages , 2010, PEPM '10.

[58]  John C. Reynolds User-defined types and procedural data structures as complementary approaches to data abstraction , 1994 .

[59]  Walid Taha,et al.  A sound reduction semantics for untyped CBN mutli-stage computation. Or, the theory of MetaML is non-trival (extended abstract) , 1999, PEPM '00.

[60]  Brian Cantwell Smith,et al.  Procedural reflection in programming languages , 1982 .

[61]  Paul Hudak,et al.  Modular domain specific languages and tools , 1998, Proceedings. Fifth International Conference on Software Reuse (Cat. No.98TB100203).

[62]  Sven-Bodo Scholz,et al.  Single Assignment C: efficient support for high-level array operations in a functional setting , 2003, Journal of Functional Programming.