Norm Comparisons for Data Augmentation

This short paper considers comparisons of different data augmentation algorithms in terms of their convergence and efficiency. It examines connections between the partial order 1 on Markov kernels, and inequalities of operator norms. It applies notions from Roberts and Rosenthal (2006) related to variance bounding Markov chains, together with L2 theory, to data augmentation algorithms (Tanner and Wong, 1987; Liu and Wu, 1999; Meng and van Dyk, 1999; Hobert and Marchev, 2006). In particular, our main result, Theorem 10, is a direct generalisation of one of the theorems in Hobert and Marchev (2006).

[1]  Gareth O. Roberts,et al.  Variance bounding Markov chains. , 2008, 0806.2747.

[2]  Jun S. Liu,et al.  Parameter Expansion for Data Augmentation , 1999 .

[3]  J. Rosenthal,et al.  Markov Chains and De-initialising Processes by , 2001 .

[4]  Barnes.  Discussion of the Paper , 1961, Public health papers and reports.

[5]  L. A. Goldberg,et al.  Markov chain comparison , 2004, math/0410331.

[6]  Gareth O. Roberts,et al.  Markov Chains and De‐initializing Processes , 2001 .

[7]  W. Wong,et al.  The calculation of posterior distributions by data augmentation , 1987 .

[8]  J. Rosenthal,et al.  Geometric Ergodicity and Hybrid Markov Chains , 1997 .

[9]  Y. Amit.  On rates of convergence of stochastic relaxation for Gaussian and non-Gaussian distributions , 1991 .

[10]  S. Varadhan,et al.  Central limit theorem for additive functionals of reversible Markov processes and applications to simple exclusions , 1986 .

[11]  S. Rosenthal,et al.  Asymptotic Variance and Convergence Rates of Nearly-Periodic MCMC Algorithms , 2002 .

[12]  Xiao-Li Meng,et al.  Seeking efficient data augmentation schemes via conditional and marginal augmentation , 1999 .

[13]  J. Hobert,et al.  A theoretical comparison of the data augmentation, marginal augmentation and PX-DA algorithms , 2008, 0804.0671.

[14]  P. Peskun,et al.  Optimum Monte-Carlo sampling using Markov chains , 1973 .

[15]  Antonietta Mira,et al.  Ordering Monte Carlo Markov Chains , 1999 .

[16]  L. Tierney.  A note on Metropolis-Hastings kernels for general state spaces , 1998 .

[17]  J. Rosenthal,et al.  General state space Markov chains and MCMC algorithms , 2004, math/0404033.

[18]  Antonietta Mira,et al.  Ordering and Improving the Performance of Monte Carlo Markov Chains , 2001 .

[19]  Jun S. Liu,et al.  Covariance structure of the Gibbs sampler with applications to the comparisons of estimators and augmentation schemes , 1994 .

[20]  J. A. Fill.  Eigenvalue bounds on convergence to stationarity for nonreversible markov chains , 1991 .