Bayesian analysis for reversible Markov chains

We introduce a natural conjugate prior for the transition matrix of a reversible Markov chain. This allows estimation and testing. The prior arises from random walk with reinforcement in the same way the Dirichlet prior arises from Polya’s urn. We give closed form normalizing constants, a simple method of simulation from the posterior and a characterization along the lines of W. E. Johnson’s characterization of the Dirichlet prior.

[1]  J. Doob.  Stochastic processes , 1953 .

[2]  D. Freedman.  Mixtures of Markov Processes , 1962 .

[3]  Irving John Good,et al.  The Estimation of Probabilities: An Essay on Modern Bayesian Methods , 1965 .

[4]  I. Good,et al.  The Estimation of Probabilities: An Essay on Modern Bayesian Methods. , 1967 .

[5]  Thomas Höglund,et al.  Central limit theorems and statistical inference for finite Markov chains , 1974 .

[6]  Stephen B. Maurer.  Matrix Generalizations of Some Theorems on Trees, Cycles and Cocycles in Graphs , 1976 .

[7]  P. Giblin.  Graphs, surfaces, and homology , 1977 .

[8]  P. Giblin,et al.  Graphs, surfaces, and homology : an introduction to algebraic topology , 1977 .

[9]  D. Freedman,et al.  De Finetti's Theorem for Markov Chains , 1980 .

[10]  S. Zabell.  W. E. Johnson's "Sufficientness" Postulate , 1982 .

[11]  W. J. Hall,et al.  Approximating Priors by Mixtures of Natural Conjugate Priors , 1983 .

[12]  J. F. Crook,et al.  The Robustness and Sensitivity of the Mixed-Dirichlet Bayesian Test for "Independence" in Contingency Tables , 1987 .

[13]  Robin Pemantle,et al.  Phase transition in reinforced random walk and RWRE on trees , 1988 .

[14]  W. Wagenaar,et al.  The perception of randomness , 1991 .

[15]  E. Capelas de Oliveira.  On generating functions , 1992 .

[16]  S. Zabell.  Characterizing Markov exchangeable sequences , 1995 .

[17]  David P. Robbins,et al.  On the Volume of a Certain Polytope , 1998, Exp. Math..

[18]  Keane,et al.  Edge-reinforced random walk on finite graphs , 2000 .

[19]  L. Ladelli,et al.  On mixtures of distributions of Markov chains , 2002 .

[20]  Silke W. W. Rolles,et al.  How edge-reinforced random walk arises naturally , 2003 .