Nesting Probabilistic Programs

We formalize the notion of nesting probabilistic programming queries and investigate the resulting statistical implications. We demonstrate that while query nesting allows the definition of models which could not otherwise be expressed, such as those involving agents reasoning about other agents, existing systems take approaches which lead to inconsistent estimates. We show how to correct this by delineating possible ways one might want to nest queries and asserting the respective conditions required for convergence. We further introduce a new online nested Monte Carlo estimator that makes it substantially easier to ensure these conditions are met, thereby providing a simple framework for designing statistically correct inference engines. We prove the correctness of this online estimator and show that, when using the recommended setup, its asymptotic variance is always better than that of the equivalent fixed estimator, while its bias is always within a factor of two.

[1]  David Tolpin,et al.  Design and Implementation of Probabilistic Programming Language Anglican , 2016, IFL 2016.

[2]  Rich Hickey,et al.  The Clojure programming language , 2008, DLS '08.

[3]  Martyn Plummer.  Cuts in Bayesian graphical models , 2015, Stat. Comput..

[4]  Noah D. Goodman,et al.  A Dynamic Programming Algorithm for Inference in Recursive Probabilistic Programs , 2012, StarAI@UAI.

[5]  Frank D. Wood,et al.  On the Pitfalls of Nested Monte Carlo , 2016, 1612.00951.

[6]  Fredrik Lindsten,et al.  Smoothing With Couplings of Conditional Particle Filters , 2017, Journal of the American Statistical Association.

[7]  Zoubin Ghahramani,et al.  MCMC for Doubly-intractable Distributions , 2006, UAI.

[8]  Noah D. Goodman,et al.  Practical optimal experiment design with probabilistic programs , 2016, ArXiv.

[9]  J. Møller,et al.  An efficient Markov chain Monte Carlo method for distributions with intractable normalising constants , 2006 .

[10]  Vikash K. Mansinghka,et al.  Using probabilistic programs as proposals , 2018, ArXiv.

[11]  Frank D. Wood,et al.  Bayesian Optimization for Probabilistic Programs , 2017, NIPS.

[12]  Noah D. Goodman,et al.  Reasoning about reasoning by nested conditioning: Modeling theory of mind with probabilistic programs , 2014, Cognitive Systems Research.

[13]  A. Doucet,et al.  Particle Markov chain Monte Carlo methods , 2010 .

[14]  Fredrik Lindsten,et al.  Nested Sequential Monte Carlo Methods , 2015, ICML.

[15]  Chung-chieh Shan,et al.  Composing Inference Algorithms as Program Transformations , 2016, UAI.

[16]  L. Jeff Hong,et al.  Estimating the mean of a non-linear function of conditional expectation , 2009, Proceedings of the 2009 Winter Simulation Conference (WSC).

[17]  David Tolpin,et al.  Probabilistic Programming in Anglican , 2015, ECML/PKDD.

[18]  Tom Rainforth,et al.  Automating inference, learning, and design using probabilistic programming , 2017 .

[19]  C. Andrieu,et al.  The pseudo-marginal approach for efficient Monte Carlo computations , 2009, 0903.5480.

[20]  K. Chaloner,et al.  Bayesian Experimental Design: A Review , 1995 .

[21]  Frank D. Wood,et al.  A New Approach to Probabilistic Programming Inference , 2014, AISTATS.

[22]  Joshua B. Tenenbaum,et al.  Church: a language for generative models , 2008, UAI.

[23]  Adam Ścibior.  Modular construction of Bayesian inference algorithms , 2016 .

[24]  G. Casella,et al.  Rao-Blackwellisation of sampling schemes , 1996 .

[25]  Gersende Fort,et al.  MCMC design-based non-parametric regression for rare event. Application to nested risk computations , 2017, Monte Carlo Methods Appl..

[26]  Frank D. Wood,et al.  On the Opportunities and Pitfalls of Nesting Monte Carlo Estimators , 2017 .

[27]  Hongseok Yang,et al.  On Nesting Monte Carlo Estimators , 2017, ICML.

[28]  Thomas A. Henzinger,et al.  Probabilistic programming , 2014, FOSE.

[29]  Zoubin Ghahramani,et al.  Bayesian Learning in Undirected Graphical Models: Approximate MCMC Algorithms , 2004, UAI.

[30]  Gerda Janssens,et al.  Nesting Probabilistic Inference , 2011, ArXiv.

[31]  O. François,et al.  Approximate Bayesian Computation (ABC) in practice. , 2010, Trends in ecology & evolution.

[32]  Yura N. Perov,et al.  Venture: a higher-order probabilistic programming platform with programmable inference , 2014, ArXiv.

[33]  Peter W. Glynn,et al.  Exact estimation for Markov chain equilibrium expectations , 2014, Journal of Applied Probability.

[34]  David Bruce Wilson,et al.  Exact sampling with coupled Markov chains and applications to statistical mechanics , 1996, Random Struct. Algorithms.

[35]  Frank Wood,et al.  Nested Compiled Inference for Hierarchical Reinforcement Learning , 2016 .