First-class runtime generation of high-performance types using exotypes

We introduce exotypes, user-defined types that combine the flexibility of meta-object protocols in dynamically-typed languages with the performance control of low-level languages. Like objects in dynamic languages, exotypes are defined programmatically at run-time, allowing behavior based on external data such as a database schema. To achieve high performance, we use staged programming to define the behavior of an exotype during a runtime compilation step and implement exotypes in Terra, a low-level staged programming language. We show how exotype constructors compose, and use exotypes to implement high-performance libraries for serialization, dynamic assembly, automatic differentiation, and probabilistic programming. Each exotype achieves expressiveness similar to libraries written in dynamically-typed languages but implements optimizations that exceed the performance of existing libraries written in low-level statically-typed languages. Though each implementation is significantly shorter, our serialization library is 11 times faster than Kryo, and our dynamic assembler is 3--20 times faster than Google's Chrome assembler.

[1]  Peter Norvig,et al.  Design Patterns in Dynamic Programming , 2013 .

[2]  Noah D. Goodman,et al.  Lightweight Implementations of Probabilistic Programming Languages Via Transformational Compilation , 2011, AISTATS.

[3]  Walid Taha,et al.  A Gentle Introduction to Multi-stage Programming , 2003, Domain-Specific Program Generation.

[4]  Gregor Kiczales,et al.  An Architecture for An Open Compiler , 1998 .

[5]  Ralph Johnson,et al.  design patterns elements of reusable object oriented software , 2019 .

[6]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[7]  Jan Vitek,et al.  Terra: a multi-stage language for high-performance computing , 2013, PLDI.

[8]  Adam Chlipala,et al.  Ur: statically-typed metaprogramming with type-level record computation , 2010, PLDI '10.

[9]  Urs Hölzle,et al.  Optimizing dynamically-dispatched calls with run-time type feedback , 1994, PLDI '94.

[10]  Gregor Kiczales,et al.  Design pattern implementation in Java and aspectJ , 2002, OOPSLA '02.

[11]  Joshua B. Tenenbaum,et al.  Church: a language for generative models , 2008, UAI.

[12]  Paul Roe,et al.  Metaphor: A Multi-stage, Object-Oriented Programming Language , 2004, GPCE.

[13]  Andreas Griewank,et al.  Automatic Differentiation of Algorithms: From Simulation to Optimization , 2000, Springer New York.

[14]  Davide Ancona,et al.  RPython: a step towards reconciling dynamically and statically typed OO languages , 2007, DLS '07.

[15]  Eugene Burmako,et al.  Scala macros: let our powers combine!: on how rich syntax and static types work with metaprogramming , 2013, SCALA@ECOOP.

[16]  L. Peter Deutsch,et al.  Efficient implementation of the smalltalk-80 system , 1984, POPL.

[17]  Roberto Ierusalimschy,et al.  The evolution of Lua , 2007, HOPL.

[18]  Pat Hanrahan,et al.  Riposte: A trace-driven compiler and parallel VM for vector code in R , 2012, 2012 21st International Conference on Parallel Architectures and Compilation Techniques (PACT).

[19]  Kunle Olukotun,et al.  Optimizing data structures in high-level programs: new directions for extensible compilers based on staging , 2013, POPL.

[20]  Chung-chieh Shan,et al.  Embedded Probabilistic Programming , 2009, DSL.

[21]  Walid Taha,et al.  MetaML and multi-stage programming with explicit annotations , 2000, Theor. Comput. Sci..

[22]  Kenji Takeda,et al.  Strongly-Typed Language Support for Internet- Scale Information Sources , 2012 .

[23]  Daniel G. Bobrow,et al.  Book review: The Art of the MetaObject Protocol By Gregor Kiczales, Jim des Rivieres, Daniel G. and Bobrow(MIT Press, 1991) , 1991, SGAR.

[24]  Radford M. Neal MCMC Using Hamiltonian Dynamics , 2011, 1206.1901.

[25]  Shigeru Chiba,et al.  A metaobject protocol for C++ , 1995, OOPSLA.