A Short History of Markov Chain Monte Carlo: Subjective Recollections from Incomplete Data

In this note we attempt to trace the history and development of Markov chain Monte Carlo (MCMC) from its early inception in the late 1940's through its use today. We see how the earlier stages of the Monte Carlo (MC, not MCMC) research have led to the algorithms currently in use. More importantly, we see how the development of this methodology has not only changed our solutions to problems, but has changed the way we think about problems.

[1]  N. Metropolis,et al.  Equation of State Calculations by Fast Computing Machines , 1953, Resonance.

[2]  J. Hammersley,et al.  Poor Man's Monte Carlo , 1954 .

[3]  A. W. Rosenbluth,et al.  MONTE CARLO CALCULATION OF THE AVERAGE EXTENSION OF MOLECULAR CHAINS , 1955 .

[4]  D. Handscomb,et al.  Computation of Order Parameters in an Ising Lattice by the Monte Carlo Method , 1960 .

[5]  John G. Kemeny,et al.  Finite Markov Chains. , 1960 .

[6]  J. Halton.  Sequential Monte Carlo , 1962, Mathematical Proceedings of the Cambridge Philosophical Society.

[7]  T. Kuhn,et al.  The Structure of Scientific Revolutions. , 1964 .

[8]  J. Hammersley,et al.  Monte Carlo Methods , 1965 .

[9]  A. Barker.  Monte Carlo calculations of the radial distribution functions for a proton-electron plasma , 1965 .

[10]  D. Mayne,et al.  Monte Carlo techniques to estimate the conditional expectation in multi-stage non-linear filtering† , 1969 .

[11]  W. K. Hastings,et al.  Monte Carlo Sampling Methods Using Markov Chains and Their Applications , 1970 .

[12]  T. Kuhn.  The Structure of Scientific Revolutions 2nd edition , 1970 .

[13]  P. Peskun,et al.  Optimum Monte-Carlo sampling using Markov chains , 1973 .

[14]  John P. Moussouris.  Gibbs and Markov random systems with constraints , 1974 .

[15]  J. Besag.  Spatial Interaction and the Statistical Analysis of Lattice Systems , 1974 .

[16]  J. Besag.  Statistical Analysis of Non-Lattice Data , 1975 .

[17]  John G. Kemeny,et al.  Finite Markov chains , 1960 .

[18]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[19]  K. Athreya,et al.  A New Approach to the Limit Theory of Recurrent Markov Chains , 1978 .

[20]  P. Billingsley,et al.  Probability and Measure , 1980 .

[21]  G. Lawler.  A self-avoiding random walk , 1980 .

[22]  P. Peskun.  Guidelines for choosing the transition matrix in Monte Carlo methods using Markov chains , 1981 .

[23]  C. D. Gelatt,et al.  Optimization by Simulated Annealing , 1983, Science.

[24]  Donald Geman,et al.  Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images , 1984, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[25]  S. Varadhan,et al.  Central limit theorem for additive functionals of reversible Markov processes and applications to simple exclusions , 1986 .

[26]  J. Besag.  On the Statistical Analysis of Dirty Pictures , 1986 .

[27]  Judea Pearl,et al.  Evidential Reasoning Using Stochastic Simulation of Causal Models , 1987, Artif. Intell..

[28]  Brian D. Ripley,et al.  Stochastic Simulation , 2005 .

[29]  W. Wong,et al.  The calculation of posterior distributions by data augmentation , 1987 .

[30]  G. Celeux,et al.  Une version de type recuit simule de l'algorithme EM , 1989 .

[31]  J. Besag,et al.  Generalized Monte Carlo significance tests , 1989 .

[32]  A. Gelman,et al.  Estimating the Electoral Consequences of Legislative Redistricting , 1990 .

[33]  G. C. Wei,et al.  A Monte Carlo Implementation of the EM Algorithm and the Poor Man's Data Augmentation Algorithms , 1990 .

[34]  W. Qian,et al.  Parameter estimation for hidden Gibbs chains , 1990 .

[35]  Adrian F. M. Smith,et al.  Sampling-Based Approaches to Calculating Marginal Densities , 1990 .

[36]  S. E. Hills,et al.  Illustration of Bayesian Inference in Normal Data Models Using Gibbs Sampling , 1990 .

[37]  Scott L. Zeger,et al.  Generalized linear models with random e ects: a Gibbs sampling approach , 1991 .

[38]  J. Besag,et al.  Bayesian image restoration, with two applications in spatial statistics , 1991 .

[39]  N. Madras,et al.  THE SELF-AVOIDING WALK , 2006 .

[40]  Adrian F. M. Smith,et al.  Bayesian Analysis of Constrained Parameter and Truncated Data Problems , 1991 .

[41]  A. Raftery,et al.  Stopping the Gibbs Sampler,the Use of Morphology,and Other Issues in Spatial Statistics (Bayesian image restoration,with two applications in spatial statistics) -- (Discussion) , 1991 .

[42]  G. Celeux,et al.  A stochastic approximation type EM algorithm for the mixture problem , 1992 .

[43]  G. Casella,et al.  Explaining the Gibbs Sampler , 1992 .

[44]  A. Gelfand,et al.  Hierarchical Bayes Models for the Progression of HIV Infection Using Longitudinal CD4 T-Cell Numbers , 1992 .

[45]  D. Rubin,et al.  Inference from Iterative Simulation Using Multiple Sequences , 1992 .

[46]  Alan E. Gelfand,et al.  Bayesian statistics without tears: A sampling-resampling perspective , 1992 .

[47]  Adrian F. M. Smith,et al.  Hierarchical Bayesian Analysis of Changepoint Problems , 1992 .

[48]  L. Tierney.  Exploring Posterior Distributions Using Markov Chains , 1992 .

[49]  E. George.  Capture—recapture estimation via Gibbs sampling , 1992 .

[50]  N. Gordon,et al.  Novel approach to nonlinear/non-Gaussian Bayesian state estimation , 1993 .

[51]  D. Gianola,et al.  Marginal inferences about variance components in a mixed linear model using Gibbs sampling , 1993, Genetics Selection Evolution.

[52]  Xiao-Li Meng,et al.  Maximum likelihood estimation via the ECM algorithm: A general framework , 1993 .

[53]  Jun S. Liu,et al.  Detecting subtle sequence signals: a Gibbs sampling strategy for multiple alignment. , 1993, Science.

[54]  Julian Besag,et al.  Towards Bayesian image analysis , 1993 .

[55]  Mike Rees,et al.  5. Statistics for Spatial Data , 1993 .

[56]  S. Chib,et al.  Bayesian analysis of binary and polychotomous response data , 1993 .

[57]  A F Smith,et al.  Bayesian inference in multipoint gene mapping , 1993, Annals of human genetics.

[58]  E. George,et al.  Journal of the American Statistical Association is currently published by American Statistical Association. , 2007 .

[59]  Jun S. Liu,et al.  Sequential Imputations and Bayesian Missing Data Problems , 1994 .

[60]  D. Gianola,et al.  Bayesian analysis of mixed linear models via Gibbs sampling with an application to litter size in Iberian pigs , 1994, Genetics Selection Evolution.

[61]  D. Rubin,et al.  The ECME algorithm: A simple extension of EM and ECM with faster monotone convergence , 1994 .

[62]  J. Rosenthal.  Minorization Conditions and Convergence Rates for Markov Chain Monte Carlo , 1995 .

[63]  Jun S. Liu,et al.  Covariance structure of the Gibbs sampler with applications to the comparisons of estimators and augmentation schemes , 1994 .

[64]  C. Geyer,et al.  Simulation Procedures and Likelihood Inference for Spatial Point Processes , 1994 .

[65]  C. Geyer,et al.  Discussion: Markov Chains for Exploring Posterior Distributions , 1994 .

[66]  C. Robert,et al.  Estimation of Finite Mixture Distributions Through Bayesian Sampling , 1994 .

[67]  D. Stephens.  Bayesian Retrospective Multiple‐Changepoint Identification , 1994 .

[68]  A. Gelfand,et al.  Bayesian Model Choice: Asymptotics and Exact Calculations , 1994 .

[69]  L. Tierney.  Markov Chains for Exploring Posterior Distributions , 1994 .

[70]  Hani Doss.  Discussion: Markov Chains for Exploring Posterior Distributions , 1994 .

[71]  Adrian F. M. Smith,et al.  Bayesian Analysis of Linear and Non‐Linear Population Models by Using the Gibbs Sampler , 1994 .

[72]  J. Dupuis.  Bayesian estimation of movement and survival probabilities from capture-recapture data , 1995 .

[73]  Jun S. Liu,et al.  Covariance Structure and Convergence Rate of the Gibbs Sampler with Various Scans , 1995 .

[74]  C. Geyer,et al.  Annealing Markov chain Monte Carlo with applications to ancestral inference , 1995 .

[75]  B. Carlin,et al.  Bayesian Model Choice Via Markov Chain Monte Carlo Methods , 1995 .

[76]  Bin Yu,et al.  Regeneration in Markov chain samplers , 1995 .

[77]  P. Green.  Reversible jump Markov chain Monte Carlo computation and Bayesian model determination , 1995 .

[78]  C. Robert.  Convergence Control Methods for Markov Chain Monte Carlo Algorithms , 1995 .

[79]  W. Gilks,et al.  Adaptive Rejection Metropolis Sampling Within Gibbs Sampling , 1995 .

[80]  Jun S. Liu,et al.  Blind Deconvolution via Sequential Imputations , 1995 .

[81]  R. Tweedie,et al.  Rates of convergence of the Hastings and Metropolis algorithms , 1996 .

[82]  G. Kitagawa.  Monte Carlo Filter and Smoother for Non-Gaussian Nonlinear State Space Models , 1996 .

[83]  Radford M. Neal.  Sampling from multimodal distributions using tempered transitions , 1996, Stat. Comput..

[84]  David Bruce Wilson,et al.  Exact sampling with coupled Markov chains and applications to statistical mechanics , 1996, Random Struct. Algorithms.

[85]  K. Athreya,et al.  ON THE CONVERGENCE OF THE MARKOV CHAIN SIMULATION METHOD , 1996 .

[86]  T. Kuhn.  The structure of scientific revolutions, 3rd ed. , 1996 .

[87]  G. Casella,et al.  The Effect of Improper Priors on Gibbs Sampling in Hierarchical Linear Mixed Models , 1996 .

[88]  James Allen Fill,et al.  An interruptible algorithm for perfect sampling via Markov chains , 1997, STOC '97.

[89]  P. Green,et al.  On Bayesian Analysis of Mixtures with an Unknown Number of Components (with discussion) , 1997 .

[90]  A. Gelman,et al.  Weak convergence and optimal scaling of random walk Metropolis algorithms , 1997 .

[91]  J. A. Fill.  An interruptible algorithm for perfect sampling via Markov chains , 1998 .

[92]  Morten Fismen.  Exact Simulation Using Markov Chains , 1998 .

[93]  R. Tweedie,et al.  Perfect simulation and backward coupling , 1998 .

[94]  C. Robert,et al.  Discretization of Continuous Markov Chains and Markov Chain Monte Carlo Convergence Assessment , 1998 .

[95]  J. A. Fill.  The Move-to-Front Rule: A Case Study for two Perfect Sampling Algorithms , 1998, Probability in the Engineering and Informational Sciences.

[96]  Michael I. Jordan.  Graphical Models , 1998 .

[97]  G. Roberts,et al.  Adaptive Markov Chain Monte Carlo through Regeneration , 1998 .

[98]  Xiao-Li Meng,et al.  Seeking efficient data augmentation schemes via conditional and marginal augmentation , 1999 .

[99]  P. Damlen,et al.  Gibbs sampling for Bayesian non‐conjugate and hierarchical models by using auxiliary variables , 1999 .

[100]  J. Rosenthal,et al.  Convergence of Slice Sampler Markov Chains , 1999 .

[101]  Jun S. Liu,et al.  Parameter Expansion for Data Augmentation , 1999 .

[102]  R. Angel.  Equations of State , 2000 .

[103]  Simon J. Godsill,et al.  On sequential Monte Carlo sampling methods for Bayesian filtering , 2000, Stat. Comput..

[104]  M. Stephens.  Bayesian analysis of mixture models with an unknown number of components- an alternative to reversible jump methods , 2000 .

[105]  W. Kendall,et al.  Perfect simulation using dominating processes on ordered spaces, with application to locally stable point processes , 2000, Advances in Applied Probability.

[106]  Christian P. Robert,et al.  Reversible Jump MCMC Converging to Birth-and-Death MCMC and More General Continuous Time Samplers , 2001 .

[107]  Galin L. Jones,et al.  Honest Exploration of Intractable Probability Distributions via Markov Chain Monte Carlo , 2001 .

[108]  W. Gilks,et al.  Following a moving target—Monte Carlo inference for dynamic Bayesian models , 2001 .

[109]  Biometrika: One Hundred Years , 2002 .

[110]  G. Casella,et al.  Explaining the Perfect Sampler , 2001 .

[111]  Xeni K. Dimakos,et al.  A Guide to Exact Simulation , 2001 .

[112]  W. Michael Conklin,et al.  Monte Carlo Methods in Bayesian Computation , 2001, Technometrics.

[113]  P. Ferrari,et al.  Perfect simulation for interacting point processes, loss networks and Ising models , 1999, math/9911162.

[114]  P. Moral,et al.  Sequential Monte Carlo samplers , 2002, cond-mat/0212648.

[115]  Galin L. Jones,et al.  On the applicability of regenerative simulation in Markov chain Monte Carlo , 2002 .

[116]  D. Rubin,et al.  MULTIPLE IMPUTATIONS IN SAMPLE SURVEYS-A PHENOMENOLOGICAL BAYESIAN APPROACH TO NONRESPONSE , 2002 .

[117]  J. Møller,et al.  Statistical Inference and Simulation for Spatial Point Processes , 2003 .

[118]  Djc MacKay,et al.  Slice sampling - Discussion , 2003 .

[119]  Petros Dellaportas,et al.  An Introduction to MCMC , 2003 .

[120]  David B. Hitchcock,et al.  A History of the Metropolis–Hastings Algorithm , 2003 .

[121]  Timothy J. Robinson,et al.  Sequential Monte Carlo Methods in Practice , 2003 .

[122]  J. Forster.  Efficient construction of reversible jump Markov chain Monte Carlo proposal distributions - Discussion , 2003 .

[123]  K. K. Berthelsen,et al.  Likelihood and Non‐parametric Bayesian MCMC Inference for Spatial Point Processes Based on Perfect Simulation and Path Sampling , 2003 .

[124]  G. Roberts,et al.  Efficient construction of reversible jump Markov chain Monte Carlo proposal distributions , 2003 .

[125]  O. Cappé,et al.  Reversible jump, birth‐and‐death and more general continuous time Markov chain Monte Carlo samplers , 2003 .

[126]  Nando de Freitas,et al.  An Introduction to MCMC for Machine Learning , 2004, Machine Learning.

[127]  Christian P. Robert,et al.  Monte Carlo Statistical Methods , 2005, Springer Texts in Statistics.

[128]  W. Gilks.  Markov Chain Monte Carlo , 2005 .

[129]  P. Atzberger.  The Monte-Carlo Method , 2006 .

[130]  Galin L. Jones,et al.  Fixed-Width Output Analysis for Markov Chain Monte Carlo , 2006, math/0601446.

[131]  J. Rosenthal,et al.  Markov Chain Monte Carlo , 2018 .

[132]  Joti Jain,et al.  Statistical Inference and Simulation , 2006 .

[133]  Bruce A. Finlayson,et al.  Equations of State , 2006 .

[134]  J. Rosenthal,et al.  Coupling and Ergodicity of Adaptive Markov Chain Monte Carlo Algorithms , 2007, Journal of Applied Probability.

[135]  Jeffrey S. Rosenthal,et al.  Coupling and Ergodicity of Adaptive MCMC , 2007 .

[136]  Jean-Michel Marin,et al.  Bayesian Core: A Practical Approach to Computational Bayesian Statistics , 2010 .

[137]  P. Fearnhead,et al.  Building Robust Simulation-based Filters for Evolving Data Sets , 2007 .

[138]  J. Hobert,et al.  A theoretical comparison of the data augmentation, marginal augmentation and PX-DA algorithms , 2008, 0804.0671.

[139]  Zhe Jiang,et al.  Spatial Statistics , 2013 .

[140]  By W. R. GILKSt,et al.  Adaptive Rejection Sampling for Gibbs Sampling , 2010 .

[141]  S. Sisson,et al.  Reversible jump Markov chain Monte Carlo , 2010, 1001.2055.

[142]  A. Pettitt,et al.  Introduction to MCMC , 2012 .

[143]  N. Metropolis.  THE BEGINNING of the MONTE CARLO METHOD , 2022 .