An Overview of Stochastic Approximation Monte Carlo

During the past few decades, Markov chain Monte Carlo (MCMC) has been widely used in Bayesian statistical inference and scientific computing. Its successes have proven it to be a very powerful and typically unique computational tool for analyzing data of complex structures. However, conventional MCMC algorithms often suffer from the local trap problem which renders the simulation ineffective. This paper provides an overview of the theory, variants, and applications for stochastic approximation Monte Carlo (SAMC), an advanced MCMC algorithm that is essentially immune to the local trap problem. WIREs Comput Stat 2014, 6:240–254. doi: 10.1002/wics.1305 For further resources related to this article, please visit the WIREs website. Conflict of interest: The author has declared no conflicts of interest for this article.

[1]  Faming Liang,et al.  Improving SAMC using smoothing methods: Theory and applications to Bayesian model selection problems , 2009, 0908.3553.

[2]  B. Berg,et al.  Multicanonical algorithms for first order phase transitions , 1991 .

[3]  S. Kou,et al.  Equi-energy sampler with applications in statistical inference and statistical mechanics , 2005, math/0507080.

[4]  D. Balding,et al.  Approximate Bayesian computation in population genetics. , 2002, Genetics.

[5]  C. D. Gelatt,et al.  Optimization by Simulated Annealing , 1983, Science.

[6]  W. Wong,et al.  Real-Parameter Evolutionary Monte Carlo With Applications to Bayesian Mixture Models , 2001 .

[7]  W. K. Hastings,et al.  Monte Carlo Sampling Methods Using Markov Chains and Their Applications , 1970 .

[8]  Faming Liang,et al.  Author's Personal Copy Computational Statistics and Data Analysis Learning Bayesian Networks for Discrete Data , 2022 .

[9]  Jun S. Liu,et al.  The Multiple-Try Method and Local Optimization in Metropolis Sampling , 2000 .

[10]  Hesselbo,et al.  Monte Carlo simulation and global optimization without parameters. , 1995, Physical review letters.

[11]  Boris Polyak,et al.  Acceleration of stochastic approximation by averaging , 1992 .

[12]  Wang,et al.  Nonuniversal critical dynamics in Monte Carlo simulations. , 1987, Physical review letters.

[13]  Faming Liang,et al.  Bayesian Analysis for Exponential Random Graph Models Using the Adaptive Exchange Sampler. , 2013, Statistics and its interface.

[14]  Faming Liang,et al.  A double Metropolis–Hastings sampler for spatial models with intractable normalizing constants , 2010 .

[15]  Walter R. Gilks,et al.  Adaptive Direction Sampling , 1994 .

[16]  J. Møller,et al.  An efficient Markov chain Monte Carlo method for distributions with intractable normalising constants , 2006 .

[17]  F. Liang,et al.  Bayesian Subset Modeling for High-Dimensional Generalized Linear Models , 2013 .

[18]  Faming Liang,et al.  Annealing stochastic approximation Monte Carlo algorithm for neural network training , 2007, Machine Learning.

[19]  Berg,et al.  Multicanonical ensemble: A new approach to simulate first-order phase transitions. , 1992, Physical review letters.

[20]  C. Geyer,et al.  Annealing Markov chain Monte Carlo with applications to ancestral inference , 1995 .

[21]  S. Walker.  Invited comment on the paper "Slice Sampling" by Radford Neal , 2003 .

[22]  Faming Liang,et al.  A Monte Carlo Metropolis-Hastings Algorithm for Sampling from Distributions with Intractable Normalizing Constants , 2013, Neural Computation.

[23]  Faming Liang,et al.  Bayesian Peak Picking for NMR Spectra , 2013, Genom. Proteom. Bioinform..

[24]  H. Robbins.  A Stochastic Approximation Method , 1951 .

[25]  Jiwen Lu,et al.  Abrupt Motion Tracking Via Intensively Adaptive Markov-Chain Monte Carlo Sampling , 2012, IEEE Transactions on Image Processing.

[26]  Faming Liang,et al.  Simulated Stochastic Approximation Annealing for Global Optimization With a Square-Root Cooling Schedule , 2014 .

[27]  D. Higdon.  Auxiliary Variable Methods for Markov Chain Monte Carlo with Applications , 1998 .

[28]  Faming Liang,et al.  Annealing evolutionary stochastic approximation Monte Carlo for global optimization , 2011, Stat. Comput..

[29]  Faming Liang,et al.  Folding small proteins via annealing stochastic approximation Monte Carlo , 2011, Biosyst..

[30]  J. Geweke,et al.  Bayesian Inference in Econometric Models Using Monte Carlo Integration , 1989 .

[31]  R. Carroll,et al.  Stochastic Approximation in Monte Carlo Computation , 2007 .

[32]  G. Torrie,et al.  Nonphysical sampling distributions in Monte Carlo free-energy estimation: Umbrella sampling , 1977 .

[33]  Peter Schmidt,et al.  Testing the restrictions implied by the rational expectations hypothesis , 1981 .

[34]  F. Liang,et al.  Author ' s personal copy Bayesian phylogeny analysis via stochastic approximation Monte Carlo , 2009 .

[35]  Faming Liang,et al.  Stochastic approximation Monte Carlo importance sampling for approximating exact conditional probabilities , 2014, Stat. Comput..

[36]  Faming Liang.  Generalized 1/k-ensemble algorithm. , 2004, Physical review. E, Statistical, nonlinear, and soft matter physics.

[37]  F. Liang.  Dynamically Weighted Importance Sampling in Monte Carlo Computation , 2002 .

[38]  F. Liang,et al.  Longitudinal functional principal component modelling via Stochastic Approximation Monte Carlo , 2010, The Canadian journal of statistics = Revue canadienne de statistique.

[39]  K. Hukushima,et al.  Exchange Monte Carlo Method and Application to Spin Glass Simulations , 1995, cond-mat/9512035.

[40]  J. Rosenthal.  Minorization Conditions and Convergence Rates for Markov Chain Monte Carlo , 1995 .

[41]  M. Xiong,et al.  Bayesian Detection of Causal Rare Variants under Posterior Consistency , 2013, PloS one.

[42]  Zelda B. Zabinsky,et al.  A Numerical Evaluation of Several Stochastic Algorithms on Selected Continuous Global Optimization Test Problems , 2005, J. Glob. Optim..

[43]  Qing Zhou.  Multi-Domain Sampling With Applications to Structural Inference of Bayesian Networks , 2011 .

[44]  H. Haario,et al.  An adaptive Metropolis algorithm , 2001 .

[45]  W H Wong,et al.  Dynamic weighting in Monte Carlo and optimization. , 1997, Proceedings of the National Academy of Sciences of the United States of America.

[46]  Han-Fu Chen,et al.  Asymptotic Efficiency of Perturbation-Analysis-Based Stochastic Approximation with Averaging , 1999 .

[47]  F. Liang.  Trajectory averaging for stochastic approximation MCMC algorithms , 2010, 1011.2587.

[48]  Jun S. Liu,et al.  The Wang-Landau algorithm in general state spaces: Applications and convergence analysis , 2010 .

[49]  Donald Geman,et al.  Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images , 1984, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[50]  Faming Liang,et al.  Efficient p-value evaluation for resampling-based tests. , 2011, Biostatistics.

[51]  J. Propp,et al.  Exact sampling with coupled Markov chains and applications to statistical mechanics , 1996 .

[52]  R. Tweedie,et al.  Geometric convergence and central limit theorems for multidimensional Hastings and Metropolis algorithms , 1996 .

[53]  A theory for the multiset sampler , 2012 .

[54]  Faming Liang,et al.  A Theory for Dynamic Weighting in Monte Carlo Computation , 2001 .

[55]  F. Liang.  Continuous Contour Monte Carlo for Marginal Density Estimation With an Application to a Spatial Statistical Model , 2007 .

[56]  F. Liang.  A Generalized Wang–Landau Algorithm for Monte Carlo Computation , 2005 .

[57]  D. Landau,et al.  Efficient, multiple-range random walk algorithm to calculate the density of states. , 2000, Physical review letters.

[58]  Eric Moulines,et al.  Stability of Stochastic Approximation under Verifiable Conditions , 2005, Proceedings of the 44th IEEE Conference on Decision and Control.

[59]  F. Liang.  On the use of stochastic approximation Monte Carlo for Monte Carlo integration , 2009 .

[60]  Faming Liang,et al.  Phylogenetic tree construction using sequential stochastic approximation Monte Carlo , 2008, Biosyst..

[61]  Michael Lavine,et al.  The Multiset Sampler , 2009 .

[62]  N. Metropolis,et al.  Equation of State Calculations by Fast Computing Machines , 1953, Resonance.

[63]  D. Stroock,et al.  Asymptotics of the spectral gap with applications to the theory of simulated annealing , 1989 .

[64]  G. Parisi,et al.  Simulated tempering: a new Monte Carlo scheme , 1992, hep-lat/9205018.