Conditions for Torpid Mixing of Parallel and Simulated Tempering on Multimodal Distributions

We obtain upper bounds on the convergence rates of Markov chains constructed by parallel and simulated tempering. These bounds are used to provide a set of sufficient conditions for torpid mixing of both techniques. We apply these conditions to show torpid mixing of parallel and simulated tempering for three examples: a normal mixture model with unequal covariances in RM and the mean-field Potts model with q ≥ 3, regardless of the number and choice of temperatures, and the meanfield Ising model when an insufficient set of temperatures is chosen. The latter result contrasts with the rapid mixing of parallel and simulated tempering on the meanfield Ising model with a linearly increasing set of temperatures as shown previously.

[1]  Donald Geman,et al.  Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images , 1984, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[2]  A. Sokal,et al.  Bounds on the ² spectrum for Markov chains and Markov processes: a generalization of Cheeger’s inequality , 1988 .

[3]  C. Geyer.  Markov Chain Monte Carlo Maximum Likelihood , 1991 .

[4]  N. Madras,et al.  THE SELF-AVOIDING WALK , 2006 .

[5]  G. Parisi,et al.  Simulated tempering: a new Monte Carlo scheme , 1992, hep-lat/9205018.

[6]  L. Tierney.  Markov Chains for Exploring Posterior Distributions , 1994 .

[7]  C. Geyer,et al.  Annealing Markov chain Monte Carlo with applications to ancestral inference , 1995 .

[8]  Ravi Kannan,et al.  Sampling according to the multivariate normal density , 1996, Proceedings of 37th Conference on Foundations of Computer Science.

[9]  Mark Jerrum,et al.  The Swendsen-Wang process does not always mix rapidly , 1997, STOC '97.

[10]  R. Tweedie,et al.  Geometric L 2 and L 1 convergence are equivalent for reversible Markov chains , 2001, Journal of Applied Probability.

[11]  D. Randall,et al.  Markov chain decomposition for convergence rate analysis , 2002 .

[12]  P. Green,et al.  Hidden Markov Models and Disease Mapping , 2002 .

[13]  Sw. Banerjee,et al.  Hierarchical Modeling and Analysis for Spatial Data , 2003 .

[14]  Neal Madras,et al.  On the swapping algorithm , 2003, Random Struct. Algorithms.

[15]  J. Rosenthal,et al.  General state space Markov chains and MCMC algorithms , 2004, math/0404033.

[16]  C. Predescu,et al.  The incomplete beta function law for parallel tempering sampling of classical canonical systems. , 2003, The Journal of chemical physics.

[17]  Dana Randall,et al.  Torpid mixing of simulated tempering on the Potts model , 2004, SODA '04.

[18]  D. Woodard,et al.  Conditions for Rapid and Torpid Mixing of Parallel and Simulated Tempering on Multimodal Distributions , 2009, 0906.2341.