MARKOV CHAIN MONTE CARLO METHODS: COMPUTATION AND INFERENCE

This chapter reviews the recent developments in Markov chain Monte Carlo simulation methods. These methods, which are concerned with the simulation of high dimensional probability distributions, have gained enormous prominence and revolutionized Bayesian statistics. The chapter provides background on the relevant Markov chain theory and provides detailed information on the theory and practice of Markov chain sampling based on the Metropolis-Hastings and Gibbs sampling algorithms. Convergence diagnostics and strategies for implementation are also discussed. A number of examples drawn from Bayesian statistics are used to illustrate the ideas. The chapter also covers in detail the application of MCMC methods to the problems of prediction and model choice.

[1]  B. Carlin,et al.  Bayesian Model Choice Via Markov Chain Monte Carlo Methods , 1995 .

[2]  L. Wasserman,et al.  Computing Bayes Factors Using a Generalization of the Savage-Dickey Density Ratio , 1995 .

[3]  C. Geyer,et al.  Discussion: Markov Chains for Exploring Posterior Distributions , 1994 .

[4]  Alison L. Gibbs,et al.  Convergence of Markov chain Monte Carlo algorithms with applications to image restoration , 2000 .

[5]  R. Kohn,et al.  Markov chain Monte Carlo in conditionally Gaussian state space models , 1996 .

[6]  David Bruce Wilson,et al.  Exact sampling with coupled Markov chains and applications to statistical mechanics , 1996, Random Struct. Algorithms.

[7]  E. Tsionas.  Monte Carlo inference in econometric models with symmetric stable disturbances , 1999 .

[8]  S. Chib,et al.  Bayesian analysis of binary and polychotomous response data , 1993 .

[9]  Bani K. Mallick,et al.  Generalized linear models with unknown link functions , 1994 .

[10]  Peter Green,et al.  Markov chain Monte Carlo in Practice , 1996 .

[11]  S. Chib.  Bayes inference in the Tobit censored regression model , 1992 .

[12]  Charles J. Geyer.  Conditioning in Markov Chain Monte Carlo , 1995 .

[13]  R. Tweedie,et al.  Rates of convergence of the Hastings and Metropolis algorithms , 1996 .

[14]  Stephen L Taylor,et al.  MODELING STOCHASTIC VOLATILITY: A REVIEW AND COMPARATIVE STUDY , 1994 .

[15]  S. Chib,et al.  Bayesian analysis of cross-section and clustered data treatment models , 2000 .

[16]  J. Rosenthal.  Minorization Conditions and Convergence Rates for Markov Chain Monte Carlo , 1995 .

[17]  Andrew D. Martin,et al.  Voter Choice in Multi-Party Democracies: A Test of Competing Theories and Models , 1999 .

[18]  Sunil Gupta,et al.  The Shopping Basket: A Model for Multicategory Purchase Incidence Decisions , 1999 .

[19]  Alan E. Gelfand,et al.  Bayesian statistics without tears: A sampling-resampling perspective , 1992 .

[20]  R. Kohn,et al.  Nonparametric regression using Bayesian variable selection , 1996 .

[21]  Peter Müller,et al.  Issues in Bayesian Analysis of Neural Network Models , 1998, Neural Computation.

[22]  E. George,et al.  APPROACHES FOR BAYESIAN VARIABLE SELECTION , 1997 .

[23]  N. Shephard,et al.  The simulation smoother for time series models , 1995 .

[24]  Adrian F. M. Smith,et al.  Simple conditions for the convergence of the Gibbs sampler and Metropolis-Hastings algorithms , 1994 .

[25]  P. Müller,et al.  Bayesian curve fitting using multivariate normal mixtures , 1996 .

[26]  Alastair Smith,et al.  Testing theories of strategic choice: The example of crisis escalation , 1999 .

[27]  W. K. Hastings,et al.  Monte Carlo Sampling Methods Using Markov Chains and Their Applications , 1970 .

[28]  S. Chib,et al.  Marginal Likelihood From the Metropolis–Hastings Output , 2001 .

[29]  Gary Koop,et al.  Bayes factors and nonlinearity: Evidence from economic time series , 1999 .

[30]  Christopher A. Sims,et al.  Advances in Econometrics , 1996 .

[31]  J. Rosenthal,et al.  Convergence of Slice Sampler Markov Chains , 1999 .

[32]  D. Rubin.  Using the SIR algorithm to simulate posterior distributions , 1988 .

[33]  Jun S. Liu,et al.  Sequential Monte Carlo methods for dynamic systems , 1997 .

[34]  N. Metropolis,et al.  Equation of State Calculations by Fast Computing Machines , 1953, Resonance.

[35]  N. Shephard,et al.  Stochastic Volatility: Likelihood Inference And Comparison With Arch Models , 1996 .

[36]  G. Roberts,et al.  Updating Schemes, Correlation Structure, Blocking and Parameterization for the Gibbs Sampler , 1997 .

[37]  Jun S. Liu,et al.  The Collapsed Gibbs Sampler in Bayesian Computations with Applications to a Gene Regulation Problem , 1994 .

[38]  D. Madigan,et al.  Bayesian Model Averaging for Linear Regression Models , 1997 .

[39]  L. M. M.-T..  Theory of Probability , 1929, Nature.

[40]  K. Chan,et al.  Monte Carlo EM Estimation for Time Series Models Involving Counts , 1995 .

[41]  S. Chib,et al.  Posterior Simulation and Bayes Factors in Panel Count Data Models , 1998 .

[42]  Hani Doss.  Discussion: Markov Chains for Exploring Posterior Distributions , 1994 .

[43]  Kung-Sik Chan.  Asymptotic behavior of the Gibbs sampler , 1993 .

[44]  Nicholas G. Polson,et al.  Inference for nonconjugate Bayesian Models using the Gibbs sampler , 1991 .

[45]  S. Chib,et al.  Analysis of multivariate probit models , 1998 .

[46]  Nicholas G. Polson,et al.  A Monte Carlo Approach to Nonnormal and Nonlinear State-Space Modeling , 1992 .

[47]  Ľuboš Pástor,et al.  Costs of Equity Capital and Model Mispricing , 1998 .

[48]  S. Meyn,et al.  Computable Bounds for Geometric Convergence Rates of Markov Chains , 1994 .

[49]  G. Parisi,et al.  Simulated tempering: a new Monte Carlo scheme , 1992, hep-lat/9205018.

[50]  James D. Hamilton.  A New Approach to the Economic Analysis of Nonstationary Time Series and the Business Cycle , 1989 .

[51]  H. Chipman,et al.  Bayesian CART Model Search , 1998 .

[52]  P. Diggle.  Analysis of Longitudinal Data , 1995 .

[53]  Craig B. Borkowf,et al.  Random Number Generation and Monte Carlo Methods , 2000, Technometrics.

[54]  Tony Lancaster.  Exact Structural Inference in Optimal Job Search Models , 1997 .

[55]  W. Wong,et al.  The calculation of posterior distributions by data augmentation , 1987 .

[56]  Bradley P. Carlin,et al.  Hierarchical Spatio-Temporal Mapping of Disease Rates , 1997 .

[57]  S. Frühwirth-Schnatter.  Data Augmentation and Dynamic Linear Models , 1994 .

[58]  John M. Olin.  On MCMC sampling in hierarchical longitudinal models , 1999 .

[59]  D. Rubin,et al.  Inference from Iterative Simulation Using Multiple Sequences , 1992 .

[60]  Gael M. Martin,et al.  US deficit sustainability: a new approach based on multiple endogenous breaks , 2000 .

[61]  Dani Gamerman,et al.  Markov Chain Monte Carlo: Stochastic Simulation for Bayesian Inference , 1997 .

[62]  L. Tierney,et al.  Accurate Approximations for Posterior Moments and Marginal Densities , 1986 .

[63]  L. Tierney.  Markov Chains for Exploring Posterior Distributions , 1994 .

[64]  R. Tweedie,et al.  Geometric convergence and central limit theorems for multidimensional Hastings and Metropolis algorithms , 1996 .

[65]  Dale J. Poirier,et al.  Intermediate Statistics and Econometrics: A Comparative Approach , 1995 .

[66]  S. Chib,et al.  Bayes inference via Gibbs sampling of autoregressive time series subject to Markov mean and variance shifts , 1993 .

[67]  R. McCulloch,et al.  STATISTICAL ANALYSIS OF ECONOMIC TIME SERIES VIA MARKOV SWITCHING MODELS , 1994 .

[68]  P. Green,et al.  Corrigendum: On Bayesian analysis of mixtures with an unknown number of components , 1997 .

[69]  Calyampudi Radhakrishna Rao,et al.  Statistical methods in finance , 1996 .

[70]  Chang‐Jin Kim,et al.  Has the U.S. Economy Become More Stable? A Bayesian Approach Based on a Markov-Switching Model of the Business Cycle , 1999, Review of Economics and Statistics.

[71]  Ming-Hui Chen.  Importance-Weighted Marginal Bayesian Posterior Density Estimation , 1994 .

[72]  Bradley P. Carlin,et al.  Markov Chain Monte Carlo conver-gence diagnostics: a comparative review , 1996 .

[73]  S. Chib.  Estimation and comparison of multiple change-point models , 1998 .

[74]  Petros Dellaportas,et al.  On Bayesian model and variable selection using MCMC , 2002, Stat. Comput..

[75]  M. Kendall,et al.  Kendall's advanced theory of statistics , 1995 .

[76]  Peter E. Rossi,et al.  Bayesian Analysis of Stochastic Volatility Models , 1994 .

[77]  Siddhartha Chib,et al.  Bayes regression with autoregressive errors : A Gibbs sampling approach , 1993 .

[78]  A. Raftery,et al.  How Many Iterations in the Gibbs Sampler , 1991 .

[79]  T. Louis.  Finding the Observed Information Matrix When Using the EM Algorithm , 1982 .

[80]  Eric T. Bradlow,et al.  A hierarchical latent variable model for ordinal data from a customer satisfaction survey with no answer responses , 1999 .

[81]  John M. Olin.  Calculating posterior distributions and modal estimates in Markov mixture models , 1996 .

[82]  Adrian F. M. Smith,et al.  Bayesian Inference for Generalized Linear and Proportional Hazards Models Via Gibbs Sampling , 1993 .

[83]  N. Shephard,et al.  Markov chain Monte Carlo methods for stochastic volatility models , 2002 .

[84]  Bradley P. Carlin,et al.  BAYES AND EMPIRICAL BAYES METHODS FOR DATA ANALYSIS , 1996, Stat. Comput..

[85]  Adrian F. M. Smith,et al.  Bayesian computation via the gibbs sampler and related markov chain monte carlo methods (with discus , 1993 .

[86]  J. Besag.  Spatial Interaction and the Statistical Analysis of Lattice Systems , 1974 .

[87]  M. Pitt,et al.  Filtering via Simulation: Auxiliary Particle Filters , 1999 .

[88]  C. Geyer,et al.  Annealing Markov chain Monte Carlo with applications to ancestral inference , 1995 .

[89]  G. Roberts,et al.  An Approach to Diagnosing Total Variation Convergence of MCMC Algorithms , 1997 .

[90]  Adrian F. M. Smith,et al.  Hierarchical Bayesian Analysis of Changepoint Problems , 1992 .

[91]  Peter E. Rossi,et al.  Bayesian Analysis of Stochastic Volatility Models: Comments: Reply , 1994 .

[92]  J. Geweke,et al.  Variable selection and model comparison in regression , 1994 .

[93]  S. Chib,et al.  Bayesian residual analysis for binary response regression models , 1995 .

[94]  Siddhartha Chib,et al.  Markov Chain Monte Carlo Simulation Methods in Econometrics , 1996, Econometric Theory.

[95]  David F. Percy,et al.  Prediction for Seemingly Unrelated Regressions , 1992 .

[96]  Greg M. Allenby,et al.  A Dynamic Model of Purchase Timing with Application to Direct Marketing , 1999 .

[97]  J. Ware,et al.  Random-effects models for longitudinal data. , 1982, Biometrics.

[98]  Gary Chamberlain,et al.  Predictive Distributions based on Longitudinal Earnings Data , 1999 .

[99]  Mark F. J. Steel,et al.  Bayesian Analysis of the Prototypal Search Model , 1998 .

[100]  M. Pitt,et al.  Analytic Convergence Rates and Parameterization Issues for the Gibbs Sampler Applied to State Space Models , 1999 .

[101]  D. Stephens.  Bayesian Retrospective Multiple‐Changepoint Identification , 1994 .

[102]  Adrian F. M. Smith,et al.  Bayesian Analysis of Linear and Non‐Linear Population Models by Using the Gibbs Sampler , 1994 .

[103]  Donald Geman,et al.  Boundary Detection by Constrained Optimization , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[104]  Mary Kathryn Cowles,et al.  Accelerating Monte Carlo Markov chain convergence for cumulative-link generalized linear models , 1996, Stat. Comput..

[105]  S. Chib,et al.  Understanding the Metropolis-Hastings Algorithm , 1995 .

[106]  Charles H. Bennett,et al.  Efficient estimation of free energy differences from Monte Carlo data , 1976 .

[107]  Stephen Gordon,et al.  Business cycle durations , 1998 .

[108]  Thomas S. Shively,et al.  Variable Selection and Function Estimation in Additive Nonparametric Regression Using a Data-Based Prior , 1999 .

[109]  Donald Geman,et al.  Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images , 1984, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[110]  C. Robert,et al.  Estimation of Finite Mixture Distributions Through Bayesian Sampling , 1994 .

[111]  Ming-Hui Chen,et al.  Reparameterizing the generalized linear model to accelerate gibbs sampler convergence , 1996 .

[112]  A. Gelfand,et al.  Efficient parametrisations for normal linear mixed models , 1995 .

[113]  P. Green.  Reversible jump Markov chain Monte Carlo computation and Bayesian model determination , 1995 .

[114]  Siddhartha Chib,et al.  Hierarchical analysis of SUR models with extensions to correlated serial errors and time-varying parameter models☆ , 1995 .

[115]  G. C. Wei,et al.  A Monte Carlo Implementation of the EM Algorithm and the Poor Man's Data Augmentation Algorithms , 1990 .

[116]  Mike K. P. So,et al.  A Stochastic Volatility Model With Markov Switching , 1998 .

[117]  D. Gamerman,et al.  Dynamic Hierarchical Models , 1993 .

[118]  M. Tanner,et al.  Facilitating the Gibbs Sampler: The Gibbs Stopper and the Griddy-Gibbs Sampler , 1992 .

[119]  E. Nummelin.  General irreducible Markov chains and non-negative operators: List of symbols and notation , 1984 .

[120]  P. Lenk.  Bayesian inference for semiparametric regression using a Fourier representation , 1999 .

[121]  Li Kai,et al.  Bayesian inference in a simultaneous equation model with limited dependent variables , 1998 .

[122]  Kishore Gawande,et al.  Comparing Theories of Endogenous Protection: Bayesian Comparison of Tobit Models Using Gibbs Sampling Output , 1998, Review of Economics and Statistics.

[123]  Richard Paap,et al.  Bayes Estimates of Markov Trends in Possibly Cointegrated Series , 2002 .

[124]  Kerrie Mengersen,et al.  [Bayesian Computation and Stochastic Systems]: Rejoinder , 1995 .

[125]  Richard L. Tweedie,et al.  Markov Chains and Stochastic Stability , 1993, Communications and Control Engineering Series.

[126]  N. Shephard.  Partial non-Gaussian state space , 1994 .

[127]  T. Kloek,et al.  Bayesian estimates of equation system parameters, An application of integration by Monte Carlo , 1976 .

[128]  D. Rubin,et al.  Parameter expansion to accelerate EM : The PX-EM algorithm , 1997 .

[129]  P. Damlen,et al.  Gibbs sampling for Bayesian non‐conjugate and hierarchical models by using auxiliary variables , 1999 .

[130]  S. Chib,et al.  Bayesian Tests and Model Diagnostics in Conditionally Independent Hierarchical Models , 1997 .

[131]  Jim Albert.  Teaching Bayesian Statistics Using Sampling Methods and MINITAB , 1993 .

[132]  G. Casella,et al.  Explaining the Gibbs Sampler , 1992 .

[133]  Wayne S. DeSarbo,et al.  The Stochastic Modeling of Purchase Intentions and Behavior , 1998 .

[134]  C. Robert,et al.  Bayesian estimation of switching ARMA models , 1999, Journal of Econometrics.

[135]  L. Devroye.  Non-Uniform Random Variate Generation , 1986 .

[136]  S. E. Hills,et al.  Illustration of Bayesian Inference in Normal Data Models Using Gibbs Sampling , 1990 .

[137]  S. Chib.  Marginal Likelihood from the Gibbs Output , 1995 .

[138]  L. Wasserman,et al.  Computing Bayes Factors by Combining Simulation and Asymptotic Approximations , 1997 .

[139]  Richard J. Patz,et al.  A Straightforward Approach to Markov Chain Monte Carlo Methods for Item Response Models , 1999 .

[140]  J. Berger.  Statistical Decision Theory and Bayesian Analysis , 1988 .

[141]  Edward E. Leamer,et al.  Specification Searches: Ad Hoc Inference with Nonexperimental Data , 1980 .

[142]  John Geweke,et al.  Efficient Simulation from the Multivariate Normal and Student-t Distributions Subject to Linear Constraints and the Evaluation of Constraint Probabilities , 1991 .

[143]  R. Kohn,et al.  On Gibbs sampling for state space models , 1994 .

[144]  J H Albert,et al.  Sequential Ordinal Modeling with Applications to Survival Data , 2001, Biometrics.

[145]  John Geweke,et al.  Evaluating the accuracy of sampling-based approaches to the calculation of posterior moments , 1991 .

[146]  Ming-Hui Chen,et al.  Monte Carlo Estimation of Bayesian Credible and HPD Intervals , 1999 .

[147]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[148]  John Geweke,et al.  Posterior simulators in econometrics , 1995 .

[149]  Gary King,et al.  Binomial-Beta Hierarchical Models for Ecological Inference , 1999 .

[150]  S. Chib,et al.  Bayes inference in regression models with ARMA (p, q) errors , 1994 .

[151]  P. Green,et al.  On Bayesian Analysis of Mixtures with an Unknown Number of Components (with discussion) , 1997 .

[152]  J. Besag,et al.  Bayesian Computation and Stochastic Systems , 1995 .

[153]  N. Shephard,et al.  Likelihood INference for Discretely Observed Non-linear Diffusions , 2001 .

[154]  Ming-Hui Chen,et al.  On Monte Carlo methods for estimating ratios of normalizing constants , 1997 .

[155]  A. Zellner.  An Introduction to Bayesian Inference in Econometrics , 1971 .

[156]  C. Robert.  Convergence Control Methods for Markov Chain Monte Carlo Algorithms , 1995 .

[157]  Jun S. Liu,et al.  Covariance structure of the Gibbs sampler with applications to the comparisons of estimators and augmentation schemes , 1994 .

[158]  Adrian F. M. Smith,et al.  A Bayesian CART algorithm , 1998 .

[159]  New York Dover,et al.  ON THE CONVERGENCE PROPERTIES OF THE EM ALGORITHM , 1983 .

[160]  Jun S. Liu,et al.  Covariance Structure and Convergence Rate of the Gibbs Sampler with Various Scans , 1995 .

[161]  Brian D. Ripley,et al.  Stochastic Simulation , 2005 .

[162]  J. Geweke,et al.  Bayesian Inference in Econometric Models Using Monte Carlo Integration , 1989 .

[163]  Robert C. Blattberg,et al.  Shrinkage Estimation of Price and Promotional Elasticities: Seemingly Unrelated Equations , 1991 .

[164]  Peter E. Rossi,et al.  Estimating Price Elasticities with Theory-Based Priors , 1999 .

[165]  Michael A. West,et al.  Evaluation and Comparison of EEG Traces: Latent Structure in Nonstationary Time Series , 1999 .

[166]  A. Harvey,et al.  5 Stochastic volatility , 1996 .

[167]  Andrew L. Rukhin,et al.  Tools for statistical inference , 1991 .

[168]  J. Richard,et al.  Specification Searches: Ad Hoc Inference with Nonexperimental Data , 1980 .

[169]  Xiao-Li Meng,et al.  SIMULATING RATIOS OF NORMALIZING CONSTANTS VIA A SIMPLE IDENTITY: A THEORETICAL EXPLORATION , 1996 .

[170]  Stephen P. Brooks,et al.  Markov chain Monte Carlo method and its application , 1998 .

[171]  E. George,et al.  Journal of the American Statistical Association is currently published by American Statistical Association. , 2007 .

[172]  M. Escobar,et al.  Bayesian Density Estimation and Inference Using Mixtures , 1995 .

[173]  G. Casella,et al.  Rao-Blackwellisation of sampling schemes , 1996 .

[174]  Bani K. Mallick,et al.  Semiparametric errors-in-variables models A Bayesian approach , 1996 .

[175]  T. Louis,et al.  Bayes and Empirical Bayes Methods for Data Analysis. , 1997 .

[176]  C. Robert,et al.  Bayesian estimation of hidden Markov chains: a stochastic implementation , 1993 .

[177]  A. Zellner,et al.  Gibbs Sampler Convergence Criteria , 1995 .

[178]  Hoon Kim,et al.  Monte Carlo Statistical Methods , 2000, Technometrics.

[179]  J. Booth,et al.  Maximizing generalized linear mixed model likelihoods with an automated Monte Carlo EM algorithm , 1999 .

[180]  A. Gelfand,et al.  Bayesian Model Choice: Asymptotics and Exact Calculations , 1994 .

[181]  Adrian F. M. Smith,et al.  Sampling-Based Approaches to Calculating Marginal Densities , 1990 .

[182]  N. Shephard.  Statistical aspects of ARCH and stochastic volatility , 1996 .

[183]  Dale J. Poirier,et al.  Intermediate Statistics and Econometrics: A Comparative Approach , 1995 .

[184]  Martin A. Tanner,et al.  Posterior Computations for Censored Regression Data , 1990 .

[185]  C. D. Litton,et al.  Theory of Probability (3rd Edition) , 1984 .

[186]  David J. Hand,et al.  A Handbook of Small Data Sets , 1993 .

[187]  Scott L. Zeger,et al.  Generalized linear models with random e ects: a Gibbs sampling approach , 1991 .