ANALYSIS OF A NONREVERSIBLE MARKOV CHAIN SAMPLER

We analyze the convergence to stationarity of a simple nonreversible Markov chain that serves as a model for several nonreversible Markov chain sampling methods that are used in practice. Our theoretical and numerical results show that nonreversibility can indeed lead to improvements over the diffusive behavior of simple Markov chain sampling schemes. The analysis uses both probabilistic techniques and an explicit diagonalization. 1. Introduction. Markov chain sampling methods are commonly used in statistics [33, 32], computer science [31], statistical mechanics [3] and quantum field theory [34, 23]. In all these fields, distributions are encountered that are difficult to sample from directly, but for which a Markov chain that converges to the distribution can easily be constructed. For many such methods (e.g., the Metropolis algorithm [25, 13], and the Gibbs sampler [17, 16] with a random scan) the Markov chain constructed is reversible. Some of these methods explore the distribution by means of a diffusive random walk. We use the term “diffusive” for processes like the ordinary random walk on a d-dimensional lattice which require time of order T 2 to travel distance T. Some other common methods, such as the Gibbs sampler with a systematic scan, use a Markov chain that is not reversible, but have diffusive behavior resembling that of a related reversible chain [30]. Some Markov chain methods attempt to avoid the inefficiencies of such diffusive exploration. The Hybrid Monte Carlo method [15] uses an elaborate Metropolis proposal that can make large changes to the state. In a variant of this method due to Horowitz [21], a similar effect is produced using a Markov chain that is carefully designed to be nonreversible. (See [34, 23, 27] for reviews of these methods.) The overrelaxation method [1] also employs a nonreversible Markov chain as a way of suppressing diffusive behavior, as discussed in [29]. In this paper, we analyze a nonreversible Markov chain that does a onedimensional walk, as an abstraction of these practical sampling methods, particularly that of Horowitz [21]. Gustafson [19] has also recently tried using adaptations of Horowitz’s method. We find that the nonreversible walk does indeed converge more rapidly than the usual simple random walk. We ana