Adaptive MCMC with Bayesian Optimization

This paper proposes a new randomized strategy for adaptive MCMC using Bayesian optimization. This approach applies to nondifferentiable objective functions and trades off exploration and exploitation to reduce the number of potentially costly objective function evaluations. We demonstrate the strategy in the complex setting of sampling from constrained, discrete and densely connected probabilistic graphical models where, for each variation of the problem, one needs to adjust the parameters of the proposal mechanism automatically to ensure efficient mixing of the Markov chains.

[1]  K. Kawasaki.  Diffusion Constants near the Critical Point for Time-Dependent Ising Models. I , 1966 .

[2]  D. Bertsekas.  Projected Newton methods for optimization problems with simple constraints , 1981, 1981 20th IEEE Conference on Decision and Control including the Symposium on Adaptive Processes.

[3]  Geoffrey E. Hinton,et al.  A Learning Algorithm for Boltzmann Machines , 1985, Cogn. Sci..

[4]  S. Duane,et al.  Hybrid Monte Carlo , 1987 .

[5]  Kenny Q. Ye.  Orthogonal Column Latin Hypercubes and Their Application in Computer Experiments , 1998 .

[6]  Donald R. Jones,et al.  Global versus local search in constrained optimization of computer models , 1998 .

[7]  Hoon Kim,et al.  Monte Carlo Statistical Methods , 2000, Technometrics.

[8]  C. Robert,et al.  Controlled MCMC for Optimal Sampling , 2001 .

[9]  H. Haario,et al.  An adaptive Metropolis algorithm , 2001 .

[10]  P. Diggle,et al.  Childhood malaria in the Gambia: a case-study in model-based geostatistics. , 2002 .

[11]  A. P. Dawid,et al.  Gaussian Processes to Speed up Hybrid Monte Carlo for Expensive Bayesian Integrals , 2003 .

[12]  Petros Dellaportas,et al.  An Introduction to MCMC , 2003 .

[13]  D. Finkel,et al.  Direct optimization algorithm user guide , 2003 .

[14]  Robert B. Gramacy,et al.  Parameter space exploration with Gaussian process trees , 2004, ICML.

[15]  Nando de Freitas,et al.  An Introduction to MCMC for Machine Learning , 2004, Machine Learning.

[16]  C. Andrieu,et al.  On the ergodicity properties of some adaptive MCMC algorithms , 2006, math/0610317.

[17]  Julien Bect,et al.  On the convergence of the expected improvement algorithm , 2007 .

[18]  J. Rosenthal,et al.  Coupling and Ergodicity of Adaptive Markov Chain Monte Carlo Algorithms , 2007, Journal of Applied Probability.

[19]  P. Diggle,et al.  Model‐based geostatistics , 2007 .

[20]  Christophe Andrieu,et al.  A tutorial on adaptive MCMC , 2008, Stat. Comput..

[21]  D. Lizotte.  Practical bayesian optimization , 2008 .

[22]  Carl E. Rasmussen,et al.  Gaussian processes for machine learning , 2005, Adaptive computation and machine learning.

[23]  Gareth O. Roberts,et al.  Examples of Adaptive MCMC , 2009 .

[24]  Andreas Krause,et al.  Information-Theoretic Regret Bounds for Gaussian Process Optimization in the Bandit Setting , 2009, IEEE Transactions on Information Theory.

[25]  Nando de Freitas,et al.  Intracluster Moves for Constrained Discrete-Space MCMC , 2010, UAI.

[26]  E. Saksman,et al.  On the ergodicity of the adaptive Metropolis algorithm on unbounded domains , 2008, 0806.2933.

[27]  Matti Vihola,et al.  Grapham: Graphical models with adaptive random walk Metropolis algorithms , 2008, Comput. Stat. Data Anal..

[28]  Michael A. Osborne.  Bayesian Gaussian processes for sequential prediction, optimisation and quadrature , 2010 .

[29]  Sonja Kuhnt,et al.  Design and analysis of computer experiments , 2010 .

[30]  Nando de Freitas,et al.  A Tutorial on Bayesian Optimization of Expensive Cost Functions, with Application to Active User Modeling and Hierarchical Reinforcement Learning , 2010, ArXiv.

[31]  Radford M. Neal.  MCMC Using Hamiltonian Dynamics , 2011, 1206.1901.

[32]  Adam D. Bull,et al.  Convergence Rates of Efficient Global Optimization Algorithms , 2011, J. Mach. Learn. Res..

[33]  P. Priouret,et al.  Bayesian Time Series Models: Adaptive Markov chain Monte Carlo: theory and methods , 2011 .

[34]  Nando de Freitas,et al.  Portfolio Allocation for Bayesian Optimization , 2010, UAI.

[35]  Ziyun Wang,et al.  Predictive Adaptation of Hybrid Monte Carlo with Bayesian Parametric Bandits , 2011 .

[36]  D. Lizotte,et al.  An experimental methodology for response surface optimization methods , 2012, J. Glob. Optim..

[37]  David S. Leslie,et al.  Optimistic Bayesian Sampling in Contextual-Bandit Problems , 2012, J. Mach. Learn. Res..

[38]  Nando de Freitas,et al.  Self-Avoiding Random Dynamics on Integer Complex Systems , 2011, TOMC.