On the Convergence Rate of Random Permutation Sampler and ECR Algorithm in Missing Data Models

Label switching is a well-known phenomenon that occurs in MCMC outputs targeting the parameters’ posterior distribution of many latent variable models. Although its appearence is necessary for the convergence of the simulated Markov chain, it turns out to be a problem in the estimation procedure. In a recent paper, Papastamoulis and Iliopoulos (J Comput Graph Stat 19:313–331, 2010) introduced the Equivalence Classes Representatives (ECR) algorithm as a solution of this problem in the context of finite mixtures of distributions. In this paper, label switching is considered under a general missing data model framework that includes as special cases finite mixtures, hidden Markov models, and Markov random fields. The use of ECR algorithm is extended to this general framework and is shown that the relabelled sequence which it produces converges to its target distribution at the same rate as the Random Permutation Sampler of Frühwirth-Schnatter (2001) and that both converge at least as fast as the Markov chain generated by the original MCMC output.

[1]  R. Redner,et al.  Mixture densities, maximum likelihood, and the EM algorithm , 1984 .

[2]  Ajay Jasra,et al.  Markov Chain Monte Carlo Methods and the Label Switching Problem in Bayesian Mixture Modeling , 2005 .

[3]  M. Stephens.  Dealing with label switching in mixture models , 2000 .

[4]  Partial Correlations in Regression Computations , 1961 .

[5]  P. Green,et al.  On Bayesian Analysis of Mixtures with an Unknown Number of Components (with discussion) , 1997 .

[6]  Zhihui Liu,et al.  BAYESIAN MIXTURE· MODELS , 2010 .

[7]  S. Eddy.  Hidden Markov models. , 1996, Current opinion in structural biology.

[8]  John Odentrantz,et al.  Markov Chains: Gibbs Fields, Monte Carlo Simulation, and Queues , 2000, Technometrics.

[9]  S. Frühwirth-Schnatter.  Markov chain Monte Carlo Estimation of Classical and Dynamic Switching and Mixture Models , 2001 .

[10]  George Iliopoulos,et al.  An Artificial Allocations Based Solution to the Label Switching Problem in Bayesian Analysis of Mixtures of Distributions , 2010 .

[11]  C. Robert,et al.  Computational and Inferential Difficulties with Mixture Posterior Distributions , 2000 .

[12]  C. Holmes,et al.  MCMC and the Label Switching Problem in Bayesian Mixture Modelling 1 Markov Chain Monte Carlo Methods and the Label Switching Problem in Bayesian Mixture Modelling , 2004 .

[13]  C. Robert,et al.  Estimation of Finite Mixture Distributions Through Bayesian Sampling , 1994 .

[14]  Jean-Michel Marin,et al.  Bayesian Core: A Practical Approach to Computational Bayesian Statistics , 2010 .

[15]  W. Gilks.  Markov Chain Monte Carlo , 2005 .

[16]  C. Robert,et al.  Bayesian inference in hidden Markov models through the reversible jump Markov chain Monte Carlo method , 2000 .

[17]  Sylvia Frühwirth-Schnatter,et al.  Finite Mixture and Markov Switching Models , 2006 .

[18]  P. Saama.  MAXIMUM LIKELIHOOD AND BAYESIAN METHODS FOR MIXTURES OF NORMAL DISTRIBUTIONS , 1997 .

[19]  Adrian F. M. Smith,et al.  Sampling-Based Approaches to Calculating Marginal Densities , 1990 .

[20]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[21]  Tobias Rydén,et al.  Hidden Markov Models , 2006 .

[22]  Jean-Michel Marin,et al.  Bayesian Modelling and Inference on Mixtures of Distributions , 2005 .

[23]  Ernst Wit,et al.  Probabilistic relabelling strategies for the label switching problem in Bayesian mixture models , 2010, Stat. Comput..