Estimation of Riemannian distances between covariance operators and Gaussian processes

In thisworkwe study several recently formulated Riemannian distances between infinite-dimensional positive definite Hilbert-Schmidt operators in the context of covariance operators associated with functional random processes. Specifically, we focus on the affine-invariant Riemannian and Log-Hilbert-Schmidt distances and the family of Alpha Procrustes distances, which include both the Bures-Wasserstein and Log-Hilbert-Schmidt distances as special cases. In particular, we present finitedimensional approximations of the infinite-dimensional distances and show their convergence to the exact distances. The theoretical formulation is illustrated with numerical experiments on covariance operators of Gaussian processes.

[1]  S. Cambanis,et al.  Gaussian Processes and Gaussian Measures , 1972 .

[2]  M. Gelbrich.  On a Formula for the L2 Wasserstein Metric between Measures on Euclidean and Hilbert Spaces , 1990 .

[3]  Ha Quang Minh,et al.  Affine-Invariant Riemannian Distance Between Infinite-Dimensional Covariance Operators , 2015, GSI.

[4]  Mehrtash Tafazzoli Harandi,et al.  Bregman Divergences for Infinite Dimensional Covariance Matrices , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[5]  H. Minh,et al.  Infinite-Dimensional Log-Determinant Divergences III: Log-Euclidean and Log-Hilbert–Schmidt Divergences , 2016 .

[6]  Luigi Malagò,et al.  Wasserstein Riemannian geometry of Gaussian densities , 2018, Information Geometry.

[7]  H. Minh,et al.  Infinite-dimensional Log-Determinant divergences between positive definite Hilbert–Schmidt operators , 2017, Positivity.

[8]  Minh Ha Quang.  Convergence and finite sample approximations of entropic regularized Wasserstein distances in Gaussian and RKHS settings , 2021 .

[9]  Minh Ha Quang.  Entropic regularization of Wasserstein distance between infinite-dimensional Gaussian measures and Gaussian processes. , 2020 .

[10]  Nicholas Ayache,et al.  Geometric Means in a Novel Vector Space Structure on Symmetric Positive-Definite Matrices , 2007, SIAM J. Matrix Anal. Appl..

[11]  S. Smale,et al.  Learning Theory Estimates via Integral Operators and Their Approximations , 2007 .

[12]  Asuka Takatsu.  Wasserstein geometry of Gaussian measures , 2011 .

[13]  Ha Quang Minh,et al.  Infinite-dimensional Log-Determinant divergences between positive definite trace class operators , 2017 .

[14]  D. Dowson,et al.  The Fréchet distance between multivariate normal distributions , 1982 .

[15]  Hongwei Sun,et al.  Mercer theorem for RKHS on noncompact sets , 2005, J. Complex..

[16]  R. Bhatia,et al.  On the Bures–Wasserstein distance between positive definite matrices , 2017, Expositiones Mathematicae.

[17]  Xavier Pennec,et al.  A Riemannian Framework for Tensor Computing , 2005, International Journal of Computer Vision.

[18]  Andreas Christmann,et al.  Support vector machines , 2008, Data Mining and Knowledge Discovery Handbook.

[19]  Hans-Georg Müller,et al.  Functional Data Analysis , 2016 .

[20]  R. Bhatia.  Positive Definite Matrices , 2007 .

[21]  Piotr Kokoszka,et al.  Inference for Functional Data with Applications , 2012 .

[22]  Christopher Lance,et al.  FUNDAMENTALS OF THE THEORY OF OPERATOR ALGEBRAS Volume I Elementary Theory, Volume II Advanced Theory , 1988 .

[23]  Z. Q. John Lu,et al.  Nonparametric Functional Data Analysis: Theory And Practice , 2007, Technometrics.

[24]  C. Givens,et al.  A class of Wasserstein metrics for probability distributions. , 1984 .

[25]  Mikhail Belkin,et al.  On Learning with Integral Operators , 2010, J. Mach. Learn. Res..

[26]  Vittorio Murino,et al.  Log-Hilbert-Schmidt metric between positive definite operators on Hilbert spaces , 2014, NIPS.

[27]  M. H. Quang.  Alpha Procrustes metrics between positive definite operators: a unifying formulation for the Bures-Wasserstein and Log-Euclidean/Log-Hilbert-Schmidt metrics. , 2019, 1908.09275.

[28]  Hà Quang Minh.  A Unified Formulation for the Bures-Wasserstein and Log-Euclidean/Log-Hilbert-Schmidt Distances between Positive Definite Operators , 2019, GSI.

[29]  Guodong Zhang,et al.  Functional Variational Bayesian Neural Networks , 2019, ICLR.

[30]  John H. Maddocks,et al.  Second-Order Comparison of Gaussian Random Functions and the Geometry of DNA Minicircles , 2010 .

[31]  M. Reed.  Methods of Modern Mathematical Physics. I: Functional Analysis , 1972 .

[32]  W. Petryshyn,et al.  Direct and iterative methods for the solution of linear operator equations in Hilbert space , 1962 .

[33]  I. Dryden,et al.  Non-Euclidean statistics for covariance matrices, with applications to diffusion tensor imaging , 2009, 0910.1656.

[34]  Felipe Cucker,et al.  On the mathematical foundations of learning , 2001 .

[35]  James Hensman,et al.  On Sparse Variational Methods and the Kullback-Leibler Divergence between Stochastic Processes , 2015, AISTATS.

[36]  Yoav Zemel,et al.  Procrustes Metrics on Covariance Operators and Optimal Transportation of Gaussian Processes , 2018, Sankhya A.

[37]  Hideki Kosaki,et al.  Inequalities for the Schatten p -norm V , 1987 .

[38]  Piotr Kokoszka,et al.  Testing the Equality of Covariance Operators in Functional Samples , 2011, 1104.4049.

[39]  G. Larotonda.  Nonpositive curvature: A geometrical approach to Hilbert–Schmidt operators , 2007 .

[40]  Piercesare Secchi,et al.  Distances and inference for covariance operators , 2014 .

[41]  H. Minh,et al.  Alpha-Beta Log-Determinant Divergences Between Positive Definite Trace Class Operators , 2019, Information Geometry.