Geometric ergodicity and the spectral gap of non-reversible Markov chains

We argue that the spectral theory of non-reversible Markov chains may often be more effectively cast within the framework of the naturally associated weighted-L∞ space $${L_\infty^V}$$ , instead of the usual Hilbert space L2 = L2(π), where π is the invariant measure of the chain. This observation is, in part, based on the following results. A discrete-time Markov chain with values in a general state space is geometrically ergodic if and only if its transition kernel admits a spectral gap in $${L_\infty^V}$$ . If the chain is reversible, the same equivalence holds with L2 in place of $${L_\infty^V}$$ . In the absence of reversibility it fails: There are (necessarily non-reversible, geometrically ergodic) chains that admit a spectral gap in $${L_\infty^V}$$ but not in L2. Moreover, if a chain admits a spectral gap in L2, then for any $${h\in L_2}$$ there exists a Lyapunov function $${V_h\in L_1}$$ such that Vh dominates h and the chain admits a spectral gap in $${L_\infty^{V_h}}$$ . The relationship between the size of the spectral gap in $${L_\infty^V}$$ or L2, and the rate at which the chain converges to equilibrium is also briefly discussed.

[1]  D. Vere-Jones Markov Chains , 1972, Nature.

[2]  R. C. Bradley Information regularity and the central limit question , 1983 .

[3]  E. Nummelin General irreducible Markov chains and non-negative operators: List of symbols and notation , 1984 .

[4]  T. Stephenson Image analysis , 1992, Nature.

[5]  D. Whittaker,et al.  A Course in Functional Analysis , 1991, The Mathematical Gazette.

[6]  A. Sokal,et al.  Bounds on the ² spectrum for Markov chains and Markov processes: a generalization of Cheeger’s inequality , 1988 .

[7]  P. Diaconis,et al.  Geometric Bounds for Eigenvalues of Markov Chains , 1991 .

[8]  J. A. Fill Eigenvalue bounds on convergence to stationarity for nonreversible markov chains , 1991 .

[9]  P. Diaconis,et al.  COMPARISON THEOREMS FOR REVERSIBLE MARKOV CHAINS , 1993 .

[10]  Richard L. Tweedie,et al.  Markov Chains and Stochastic Stability , 1993, Communications and Control Engineering Series.

[11]  S. Meyn,et al.  Computable Bounds for Geometric Convergence Rates of Markov Chains , 1994 .

[12]  J. Rosenthal Minorization Conditions and Convergence Rates for Markov Chain Monte Carlo , 1995 .

[13]  James Allen Fill,et al.  THE MOVE-TO-FRONT RULE FOR SELF-ORGANIZING LISTS WITH MARKOV DEPENDENT REQUESTS· , 1995 .

[14]  J. Rosenthal Correction: Minorization Condition and Convergence Rates for Markov Chain Monte Carlo , 1995 .

[15]  Persi Diaconis,et al.  What do we know about the Metropolis algorithm? , 1995, STOC '95.

[16]  Gerhard Winkler,et al.  Image analysis, random fields and dynamic Monte Carlo methods: a mathematical introduction , 1995, Applications of mathematics.

[17]  David Dyte,et al.  On the move-to-front scheme with Markov dependent requests , 1997 .

[18]  J. Rosenthal,et al.  Geometric Ergodicity and Hybrid Markov Chains , 1997 .

[19]  Gareth O. Roberts,et al.  Convergence assessment techniques for Markov chain Monte Carlo , 1998, Stat. Comput..

[20]  Leszek Wojnar,et al.  Image Analysis , 1998 .

[21]  O. Junge,et al.  On the Approximation of Complicated Dynamical Behavior , 1999 .

[22]  Y. Peres,et al.  Broadcasting on trees and the Ising model , 2000 .

[23]  John Odentrantz,et al.  Markov Chains: Gibbs Fields, Monte Carlo Simulation, and Queues , 2000, Technometrics.

[24]  Radford M. Neal,et al.  ANALYSIS OF A NONREVERSIBLE MARKOV CHAIN SAMPLER , 2000 .

[25]  R. Tweedie,et al.  Geometric L 2 and L 1 convergence are equivalent for reversible Markov chains , 2001, Journal of Applied Probability.

[26]  S. Meyn,et al.  Spectral theory and limit theorems for geometrically ergodic Markov processes , 2002, math/0209200.

[27]  Persi Diaconis,et al.  MATHEMATICAL DEVELOPMENTS FROM THE ANALYSIS OP RIFFLE SHUFFLING , 2003 .

[28]  S. Meyn,et al.  Phase transitions and metastability in Markovian and molecular systems , 2004 .

[29]  Olle Häggström,et al.  Acknowledgement of priority concerning ``On the central limit theorem for geometrically ergodic Markov chains'' , 2006 .

[30]  Ravi Montenegro,et al.  Mathematical Aspects of Mixing Times in Markov Chains , 2006, Found. Trends Theor. Comput. Sci..

[31]  Olle Häggström,et al.  On the central limit theorem for geometrically ergodic Markov chains , 2005 .

[32]  S. Meyn,et al.  Large Deviations Asymptotics and the Spectral Theory of Multiplicatively Regular Markov Processes , 2005, math/0509310.

[33]  Metropolis : Le jour où l’étoile probabilité entra dans le champ gravitationnel de la galaxie microlocale , 2007 .

[34]  Persi Diaconis,et al.  The Markov chain Monte Carlo revolution , 2008 .

[35]  P. Diaconis,et al.  Gibbs sampling, exponential families and orthogonal polynomials , 2008, 0808.3852.

[36]  $L^{2}$-spectral gaps for time discrete reversible Markov chains , 2009, 0908.0897.

[37]  $L^{2}$-spectral gaps, weak-reversible and very weak-reversible Markov chains , 2009, 0908.0888.

[38]  Rutger van Haasteren,et al.  Gibbs Sampling , 2010, Encyclopedia of Machine Learning.