DynamicPPL: Stan-like Speed for Dynamic Probabilistic Models

We present the preliminary high-level design and features of DynamicPPL.jl, a modular library providing a lightning-fast infrastructure for probabilistic programming. Besides a computational performance that is often close to or better than Stan, DynamicPPL provides an intuitive DSL that allows the rapid development of complex dynamic probabilistic programs. Being entirely written in Julia, a high-level dynamic programming language for numerical computing, DynamicPPL inherits a rich set of features available through the Julia ecosystem. Since DynamicPPL is a modular, stand-alone library, any probabilistic programming system written in Julia, such as Turing.jl, can use DynamicPPL to specify models and trace their model parameters. The main features of DynamicPPL are: 1) a meta-programming based DSL for specifying dynamic models using an intuitive tilde-based notation; 2) a tracing data-structure for tracking RVs in dynamic probabilistic models; 3) a rich contextual dispatch system allowing tailored behaviour during model execution; and 4) a user-friendly syntax for probabilistic queries. Finally, we show in a variety of experiments that DynamicPPL, in combination with Turing.jl, achieves computational performance that is often close to or better than Stan.

[1]  Alan Edelman,et al.  Julia: A Fresh Approach to Numerical Computing , 2014, SIAM Rev..

[2]  Frank D. Wood,et al.  A New Approach to Probabilistic Programming Inference , 2014, AISTATS.

[3]  Zoubin Ghahramani,et al.  Turing: A Language for Flexible Probabilistic Inference , 2018 .

[4]  Thomas B. Schön,et al.  Automated learning with a probabilistic programming language: Birch , 2018, Annu. Rev. Control..

[5]  Stuart J. Russell,et al.  BLOG: Probabilistic Models with Unknown Objects , 2005, IJCAI.

[6]  Thomas P. Minka,et al.  Gates: A Graphical Notation for Mixture Models , 2008 .

[7]  Avi Pfeffer,et al.  IBAL: A Probabilistic Rational Programming Language , 2001, IJCAI.

[8]  Yura N. Perov,et al.  Venture: a higher-order probabilistic programming platform with programmable inference , 2014, ArXiv.

[9]  David M. Blei,et al.  Variational Inference: A Review for Statisticians , 2016, ArXiv.

[10]  Joshua B. Tenenbaum,et al.  Church: a language for generative models , 2008, UAI.

[11]  Franz Pernkopf,et al.  Graph Tracking in Dynamic Probabilistic Programs via Source Transformations , 2019 .

[12]  Alan Edelman,et al.  A Differentiable Programming System to Bridge Machine Learning and Scientific Computing , 2019, ArXiv.

[13]  Martin C. Rinard,et al.  Probabilistic modeling and inference are becoming central computational tools across a broad range of fields , 2018 .

[14]  A. Doucet,et al.  Particle Markov chain Monte Carlo methods , 2010 .

[15]  Adam Ścibior,et al.  Strongly typed tracing of probabilistic programs , 2019 .

[16]  Andrew Thomas,et al.  WinBUGS - A Bayesian modelling framework: Concepts, structure, and extensibility , 2000, Stat. Comput..

[17]  Michael Carbin,et al.  Trace types and denotational semantics for sound programmable inference in probabilistic languages , 2020, Proc. ACM Program. Lang..

[18]  Radford M. Neal.  MCMC Using Hamiltonian Dynamics , 2011, 1206.1901.

[19]  Dexter Kozen,et al.  Semantics of probabilistic programs , 1979, 20th Annual Symposium on Foundations of Computer Science (sfcs 1979).

[20]  Jiqiang Guo,et al.  Stan: A Probabilistic Programming Language. , 2017, Journal of statistical software.

[21]  Noah D. Goodman,et al.  Pyro: Deep Universal Probabilistic Programming , 2018, J. Mach. Learn. Res..