MCMC Algorithms for Posteriors on Matrix Spaces

We study Markov chain Monte Carlo (MCMC) algorithms for target distributions defined on matrix spaces. Such an important sampling problem has yet to be analytically explored. We carry out a major step in covering this gap by developing the proper theoretical framework that allows for the identification of ergodicity properties of typical MCMC algorithms, relevant in such a context. Beyond the standard Random-Walk Metropolis (RWM) and preconditioned Crank--Nicolson (pCN), a contribution of this paper in the development of a novel algorithm, termed the `Mixed' pCN (MpCN). RWM and pCN are shown not to be geometrically ergodic for an important class of matrix distributions with heavy tails. In contrast, MpCN has very good empirical performance within this class. Geometric ergodicity for MpCN is not fully proven in this work, as some remaining drift conditions are quite challenging to obtain owing to the complexity of the state space. We do, however, make a lot of progress towards a proof, and show in detail the last steps left for future work. We illustrate the computational performance of the various algorithms through simulation studies, first for the trivial case of an Inverse-Wishart target, and then for a challenging model arising in financial statistics.

[1]  G. Roberts,et al.  MCMC methods for diffusion bridges , 2008 .

[2]  R. Tweedie,et al.  Geometric convergence and central limit theorems for multidimensional Hastings and Metropolis algorithms , 1996 .

[3]  Abel Rodriguez,et al.  Bayesian Inference for General Gaussian Graphical Models With Application to Multivariate Lattice Data , 2010, Journal of the American Statistical Association.

[4]  R. Tweedie,et al.  Geometric L 2 and L 1 convergence are equivalent for reversible Markov chains , 2001, Journal of Applied Probability.

[5]  Mario Ullrich,et al.  Positivity of hit-and-run and related algorithms , 2012, 1212.4512.

[6]  A. Doucet,et al.  Particle Markov chain Monte Carlo methods , 2010 .

[7]  D. Harville.  Matrix Algebra From a Statistician's Perspective , 1998 .

[8]  Michalis K. Titsias,et al.  Scalable inference for a full multivariate stochastic volatility model , 2015, Journal of Econometrics.

[9]  Alex Lenkoski,et al.  A direct sampler for G‐Wishart variates , 2013, 1304.1350.

[10]  S. Geisser,et al.  Posterior Distributions for Multivariate Normal Parameters , 1963 .

[11]  M. Wand,et al.  Simple Marginally Noninformative Prior Distributions for Covariance Matrices , 2013 .

[12]  Hyunjoong Kim,et al.  Functional Analysis I , 2017 .

[13]  G. Roberts,et al.  MCMC Methods for Functions: ModifyingOld Algorithms to Make Them Faster , 2012, 1202.0709.

[14]  J. K. Hunter,et al.  Measure Theory , 2007 .

[15]  A. Roverato.  Hyper Inverse Wishart Distribution for Non-decomposable Graphs and its Application to Bayesian Inference for Gaussian Graphical Models , 2002 .

[16]  A. Zaslavsky,et al.  Domain-Level Covariance Analysis for Multilevel Survey Data With Structured Nonresponse , 2008 .

[17]  Xiao-Li Meng,et al.  Modeling covariance matrices in terms of standard deviations and correlations, with application to shrinkage , 2000 .

[18]  K. Kamatani,et al.  Ergodicity of Markov chain Monte Carlo with reversible proposal , 2016, Journal of Applied Probability.

[19]  C. Andrieu,et al.  The pseudo-marginal approach for efficient Monte Carlo computations , 2009, 0903.5480.

[20]  Y. Chikuse.  Statistics on special manifolds , 2003 .

[21]  Brigitte Maier,et al.  Fundamentals Of Differential Geometry , 2016 .

[22]  O. Barndorff-Nielsen,et al.  Positive-definite matrix processes of finite variation , 2007 .

[23]  James M. Dickey,et al.  Matricvariate Generalizations of the Multivariate $t$ Distribution and the Inverted Multivariate $t$ Distribution , 1967 .

[24]  J. Rosenthal,et al.  Geometric Ergodicity and Hybrid Markov Chains , 1997 .

[25]  Andrew M. Stuart,et al.  Inverse problems: A Bayesian perspective , 2010, Acta Numerica.

[26]  Laurence A. Wolsey,et al.  Production Planning by Mixed Integer Programming (Springer Series in Operations Research and Financial Engineering) , 2006 .

[27]  S. F. Jarner,et al.  Geometric ergodicity of Metropolis algorithms , 2000 .

[28]  Richard L. Tweedie,et al.  Markov Chains and Stochastic Stability , 1993, Communications and Control Engineering Series.

[29]  S. Resnick.  Heavy-Tail Phenomena: Probabilistic and Statistical Modeling , 2006 .

[30]  A. P. Dawid,et al.  Regression and Classification Using Gaussian Process Priors , 2009 .

[31]  N. Shephard,et al.  Non‐Gaussian Ornstein–Uhlenbeck‐based models and some of their uses in financial economics , 2001 .

[32]  Hao Wang,et al.  Efficient Gaussian graphical model determination under G-Wishart prior distributions , 2012 .

[33]  M. Meerschaert.  Regular Variation in R k , 1988 .