SIMPL: A DSL for Automatic Specialization of Inference Algorithms

Inference algorithms in probabilistic programming languages (PPLs) can be thought of as interpreters, since an inference algorithm traverses a model given evidence to answer a query. As with interpreters, we can improve the efficiency of inference algorithms by compiling them once the model, evidence and query are known. We present SIMPL, a domain specific language for inference algorithms, which uses this idea in order to automatically specialize annotated inference algorithms. Due to the approach of specialization, unlike a traditional compiler, with SIMPL new inference algorithms can be added easily, and still be optimized using domain-specific information. We evaluate SIMPL and show that partial evaluation gives a 2-6x speedup, caching provides an additional 1-1.5x speedup, and generating C code yields an additional 13-20x speedup, for an overall speedup of 30-150x for several inference algorithms and models.

[1]  Yoshihiko Futamura,et al.  Partial Evaluation of Computation Process--An Approach to a Compiler-Compiler , 1999, High. Order Symb. Comput..

[2]  Avi Pfeffer,et al.  Practical Probabilistic Programming , 2016, ILP.

[3]  Frédo Durand,et al.  Halide: a language and compiler for optimizing parallelism, locality, and recomputation in image processing pipelines , 2013, PLDI 2013.

[4]  Ben Taskar,et al.  BLOG: Probabilistic Models with Unknown Objects , 2007 .

[5]  Kunle Olukotun,et al.  A domain-specific approach to heterogeneous parallelism , 2011, PPoPP '11.

[6]  Chung-Kil Hur,et al.  R2: An Efficient MCMC Sampler for Probabilistic Programs , 2014, AAAI.

[7]  Joshua B. Tenenbaum,et al.  Church: a language for generative models , 2008, UAI.

[8]  Avi Pfeffer Sampling with Memoization , 2007, AAAI.

[9]  Chung-Kil Hur,et al.  Slicing probabilistic programs , 2014, PLDI.

[10]  Erik Ruf,et al.  Data specialization , 1996, PLDI '96.

[11]  Peter Sestoft,et al.  Partial evaluation and automatic program generation , 1993, Prentice Hall international series in computer science.

[12]  Guy E. Blelloch,et al.  Selective memoization , 2003, POPL '03.

[13]  Stuart J. Russell,et al.  Extending Bayesian Networks to the Open-Universe Case , 2009 .

[14]  Stuart J. Russell,et al.  Global Seismic Monitoring: A Bayesian Approach , 2011, AAAI.

[15]  Jiqiang Guo,et al.  Stan: A Probabilistic Programming Language. , 2017, Journal of statistical software.

[16]  Noah D. Goodman,et al.  Lightweight Implementations of Probabilistic Programming Languages Via Transformational Compilation , 2011, AISTATS.

[17]  Calvin Lin,et al.  An annotation language for optimizing software libraries , 1999, DSL '99.

[18]  Walid Taha,et al.  MetaML and multi-stage programming with explicit annotations , 2000, Theor. Comput. Sci..

[19]  Kunle Olukotun,et al.  Surgical precision JIT compilers , 2014, PLDI.

[20]  Martin Odersky,et al.  Lightweight modular staging: a pragmatic approach to runtime code generation and compiled DSLs , 2010, GPCE '10.