Maximum a Posteriori Estimation by Search in Probabilistic Programs

We introduce an approximate search algorithm for fast maximum a posteriori probability estimation in probabilistic programs, which we call Bayesian ascent Monte Carlo (BaMC). Probabilistic programs represent probabilistic models with varying number of mutually dependent finite, countable, and continuous random variables. BaMC is an anytime MAP search algorithm applicable to any combination of random variables and dependencies. We compare BaMC to other MAP estimation algorithms and show that BaMC is faster and more robust on a range of probabilistic models.

[1]  C. D. Gelatt,et al.  Optimization by Simulated Annealing , 1983, Science.

[2]  Frank D. Wood,et al.  A New Approach to Probabilistic Programming Inference , 2014, AISTATS.

[3]  Rémi Munos,et al.  Pure exploration in finitely-armed and continuous-armed bandits , 2011, Theor. Comput. Sci..

[4]  Michael I. Jordan.  Graphical Models , 2003 .

[5]  Simon J. Godsill,et al.  Marginal maximum a posteriori estimation using Markov chain Monte Carlo , 2002, Stat. Comput..

[7]  Shipra Agrawal,et al.  Analysis of Thompson Sampling for the Multi-armed Bandit Problem , 2011, COLT.

[8]  Guy Shani,et al.  Evaluating Recommender Systems , 2015, Recommender Systems Handbook.

[9]  Rémi Munos,et al.  Optimistic Planning in Markov Decision Processes Using a Generative Model , 2014, NIPS.

[10]  Feng Wu,et al.  Bayesian Mixture Modelling and Inference based Thompson Sampling in Monte-Carlo Tree Search , 2013, NIPS.

[11]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[12]  Csaba Szepesvári,et al.  Bandit Based Monte-Carlo Planning , 2006, ECML.

[13]  G. G. Stokes.  "J." , 1890, The New Yale Book of Quotations.

[14]  Thomas A. Henzinger,et al.  Probabilistic programming , 2014, FOSE.

[15]  David Pfau,et al.  Probabilistic Deterministic Infinite Automata , 2010, NIPS.

[16]  Kevin P. Murphy,et al.  Machine learning - a probabilistic perspective , 2012, Adaptive computation and machine learning series.

[17]  Changhe Yuan,et al.  Dynamic Weighting A* Search-based MAP Algorithm for Bayesian Networks , 2006, Probabilistic Graphical Models.

[18]  W. R. Thompson.  ON THE LIKELIHOOD THAT ONE UNKNOWN PROBABILITY EXCEEDS ANOTHER IN VIEW OF THE EVIDENCE OF TWO SAMPLES , 1933 .

[19]  Liang He,et al.  Evaluating recommender systems , 2012, Seventh International Conference on Digital Information Management (ICDIM 2012).

[20]  Sindhu V. Raghavan,et al.  Bayesian Abductive Logic Programs: A Probabilistic Logic for Abductive Reasoning , 2011, IJCAI.

[21]  Alistair I. Mees,et al.  Convergence of an annealing algorithm , 1986, Math. Program..

[22]  Adnan Darwiche,et al.  Solving MAP Exactly using Systematic Search , 2002, UAI.

[23]  Denis Deratani Mauá,et al.  Anytime Marginal MAP Inference , 2012, ICML.

[24]  G. C. Wei,et al.  A Monte Carlo Implementation of the EM Algorithm and the Poor Man's Data Augmentation Algorithms , 1990 .

[25]  David Tolpin,et al.  Selecting Computations: Theory and Applications , 2012, UAI.

[26]  Nataliya Sokolovska,et al.  Continuous Upper Confidence Trees , 2011, LION.

[27]  Steven L. Scott,et al.  A modern Bayesian look at the multi-armed bandit , 2010 .

[28]  Noah D. Goodman,et al.  Lightweight Implementations of Probabilistic Programming Languages Via Transformational Compilation , 2011, AISTATS.

[29]  Joshua B. Tenenbaum,et al.  Church: a language for generative models , 2008, UAI.

[30]  Yee Whye Teh,et al.  Asynchronous Anytime Sequential Monte Carlo , 2014, NIPS.

[31]  John K Kruschke,et al.  Bayesian data analysis. , 2010, Wiley interdisciplinary reviews. Cognitive science.

[32]  Christophe Andrieu,et al.  Simulated annealing for maximum a Posteriori parameter estimation of hidden Markov models , 2000, IEEE Trans. Inf. Theory.