Covariances in Computer Vision and Machine Learning

Abstract Covariance matrices play important roles in many areas of mathematics, statistics, and machine learning, as well as their applications. In computer vision and image processing, they give rise to a powerful data representation, namely the covariance descriptor, with numerous practical applications. In this book, we begin by presenting an overview of the {\it finite-dimensional covariance matrix} representation approach of images, along with its statistical interpretation. In particular, we discuss the various distances and divergences that arise from the intrinsic geometrical structures of the set of Symmetric Positive Definite (SPD) matrices, namely Riemannian manifold and convex cone structures. Computationally, we focus on kernel methods on covariance matrices, especially using the Log-Euclidean distance. We then show some of the latest developments in the generalization of the finite-dimensional covariance matrix representation to the {\it infinite-dimensional covariance operator} representati...

[1]  Lei Zhang,et al.  Log-Euclidean Kernels for Sparse Representation and Dictionary Learning , 2013, 2013 IEEE International Conference on Computer Vision.

[2]  L. Bregman.  The relaxation method of finding the common point of convex sets and its application to the solution of problems in convex programming , 1967 .

[3]  Hongdong Li,et al.  Kernel Methods on the Riemannian Manifold of Symmetric Positive Definite Matrices , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[4]  Hamid Laga,et al.  Covariance Descriptors for 3D Shape Matching and Retrieval , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[5]  Jun Zhang,et al.  Divergence Function, Duality, and Convex Analysis , 2004, Neural Computation.

[6]  Janusz Konrad,et al.  Action Recognition Using Sparse Representation on Covariance Manifolds of Optical Flow , 2010, 2010 7th IEEE International Conference on Advanced Video and Signal Based Surveillance.

[7]  Frances Y. Kuo,et al.  High-dimensional integration: The quasi-Monte Carlo way*† , 2013, Acta Numerica.

[8]  Søren Hauberg,et al.  Geodesic exponential kernels: When curvature and linearity conflict , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[9]  Ha Quang Minh,et al.  Infinite-dimensional Log-Determinant divergences between positive definite Hilbert–Schmidt operators , 2017, Positivity.

[10]  Robert B. Fisher,et al.  A research tool for long-term and continuous analysis of fish assemblage in coral-reefs using underwater camera footage , 2014, Ecol. Informatics.

[11]  Larry S. Davis,et al.  Covariance discriminative learning: A natural and efficient approach to image set classification , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[12]  Vassilios Morellas,et al.  Tensor Sparse Coding for Positive Definite Matrices , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[13]  Maher Moakher,et al.  The Riemannian Geometry of the Space of Positive-Definite Matrices and Its Application to the Regularization of Positive-Definite Matrix-Valued Data , 2011, Journal of Mathematical Imaging and Vision.

[14]  Lei Zhang,et al.  RAID-G: Robust Estimation of Approximate Infinite Dimensional Gaussian with Application to Material Recognition , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[15]  Gérard Letac,et al.  All Invariant Moments of the Wishart Distribution , 2004 .

[16]  Moo K. Chung,et al.  Manifold learning on brain functional networks in aging , 2015, Medical Image Anal..

[17]  Andrew Zisserman,et al.  Efficient Additive Kernels via Explicit Feature Maps , 2012, IEEE Trans. Pattern Anal. Mach. Intell..

[18]  Saeed Shiry Ghidary,et al.  Kernel learning over the manifold of symmetric positive definite matrices for dimensionality reduction in a BCI application , 2016, Neurocomputing.

[19]  Brian C. Lovell,et al.  Sparse Coding on Symmetric Positive Definite Manifolds Using Bregman Divergences , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[20]  Shiguang Shan,et al.  Hybrid Euclidean-and-Riemannian Metric Learning for Image Set Classification , 2014, ACCV.

[21]  G. Larotonda.  Nonpositive curvature: A geometrical approach to Hilbert–Schmidt operators , 2007 .

[22]  F. Barbaresco.  Information Geometry of Covariance Matrix: Cartan-Siegel Homogeneous Bounded Domains, Mostow/Berger Fibration and Fréchet Median , 2013 .

[23]  Anoop Cherian,et al.  Jensen-Bregman LogDet Divergence with Application to Efficient Similarity Search for Covariance Matrices , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[24]  Nicholas Ayache,et al.  Fast and Simple Calculus on Tensors in the Log-Euclidean Framework , 2005, MICCAI.

[25]  Mehrtash Harandi,et al.  Dimensionality Reduction on SPD Manifolds: The Emergence of Geometry-Aware Methods , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[26]  Alan Julian Izenman,et al.  Modern Multivariate Statistical Techniques , 2008 .

[27]  D. Le Bihan,et al.  Diffusion tensor imaging: Concepts and applications , 2001, Journal of magnetic resonance imaging : JMRI.

[28]  Xuelong Li,et al.  Gabor-Based Region Covariance Matrices for Face Recognition , 2008, IEEE Transactions on Circuits and Systems for Video Technology.

[29]  Dario Bini,et al.  Computing the Karcher mean of symmetric positive definite matrices , 2013 .

[30]  Pierre Formont,et al.  On the Use of Matrix Information Geometry for Polarimetric SAR Image Classification , 2013 .

[31]  Cristian Sminchisescu,et al.  Matrix Backpropagation for Deep Networks with Structured Layers , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[32]  Bingpeng Ma,et al.  Covariance descriptor based on bio-inspired features for person re-identification and face verification , 2014, Image Vis. Comput..

[33]  Hongdong Li,et al.  Kernel Methods on Riemannian Manifolds with Gaussian RBF Kernels , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[34]  Mehrtash Tafazzoli Harandi,et al.  Bregman Divergences for Infinite Dimensional Covariance Matrices , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[35]  Vassilios Morellas,et al.  Compact covariance descriptors in 3D point clouds for object recognition , 2012, 2012 IEEE International Conference on Robotics and Automation.

[36]  Jitendra Malik,et al.  Hypercolumns for object segmentation and fine-grained localization , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[37]  Barbara Caputo,et al.  Class-Specific Material Categorisation , 2005, ICCV.

[38]  Yunde Jia,et al.  Manifold Kernel Sparse Representation of Symmetric Positive-Definite Matrices and Its Applications , 2015, IEEE Transactions on Image Processing.

[39]  Yin Wang,et al.  Efficient Temporal Sequence Comparison and Classification Using Gram Matrix Embeddings on a Riemannian Manifold , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[40]  Vittorio Murino,et al.  Kernelized covariance for action recognition , 2016, 2016 23rd International Conference on Pattern Recognition (ICPR).

[41]  Manfredo P. do Carmo,et al.  Differential Geometry of Surfaces , 1994 .

[42]  Christian Jutten,et al.  Multiclass Brain–Computer Interface Classification by Riemannian Geometry , 2012, IEEE Transactions on Biomedical Engineering.

[43]  Nikolaos Papanikolopoulos,et al.  RGB-D object classification using covariance descriptors , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[44]  Rama Chellappa,et al.  From sample similarity to ensemble similarity: probabilistic distance measures in reproducing kernel Hilbert space , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[45]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[46]  Yin Wang,et al.  Jensen Bregman LogDet Divergence Optimal Filtering in the Manifold of Positive Definite Matrices , 2016, ECCV.

[47]  Ha Quang Minh,et al.  Affine-Invariant Riemannian Distance Between Infinite-Dimensional Covariance Operators , 2015, GSI.

[48]  Xiaoqin Zhang,et al.  Single and Multiple Object Tracking Using Log-Euclidean Riemannian Subspace and Block-Division Appearance Model , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[49]  Fatih Murat Porikli,et al.  Covariance Tracking using Model Update Based on Lie Algebra , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[50]  Fatih Murat Porikli,et al.  Region Covariance: A Fast Descriptor for Detection and Classification , 2006, ECCV.

[51]  Mehrtash Tafazzoli Harandi,et al.  Image set classification by symmetric positive semi-definite matrices , 2016, 2016 IEEE Winter Conference on Applications of Computer Vision (WACV).

[52]  Ha Quang Minh,et al.  Infinite-dimensional Log-Determinant divergences between positive definite trace class operators , 2017 .

[53]  Nicholas J. Higham,et al.  Functions of matrices - theory and computation , 2008 .

[54]  Mehrtash Tafazzoli Harandi,et al.  Approximate infinite-dimensional Region Covariance Descriptors for image classification , 2015, 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[55]  Rama Chellappa,et al.  Kernel Learning for Extrinsic Classification of Manifold Features , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[56]  David Declercq,et al.  3D face recognition using covariance based descriptors , 2016, Pattern Recognit. Lett..

[57]  Xiaoqian Sun,et al.  Improved Stein-type shrinkage estimators for the high-dimensional multivariate normal covariance matrix , 2011, Comput. Stat. Data Anal..

[58]  Xavier Pennec,et al.  A Riemannian Framework for Tensor Computing , 2005, International Journal of Computer Vision.

[59]  Rachid Deriche,et al.  Statistics on the Manifold of Multivariate Normal Distributions: Theory and Application to Diffusion Tensor MRI Processing , 2006, Journal of Mathematical Imaging and Vision.

[60]  Maher Moakher,et al.  Means of Hermitian positive-definite matrices based on the log-determinant α-divergence function , 2012 .

[61]  Vittorio Murino,et al.  From Covariance Matrices to Covariance Operators: Data Representation from Finite to Infinite-Dimensional Settings , 2016 .

[62]  Nicholas Ayache,et al.  Geometric Means in a Novel Vector Space Structure on Symmetric Positive-Definite Matrices , 2007, SIAM J. Matrix Anal. Appl..

[63]  Sergio Cruces,et al.  Log-Determinant Divergences Revisited: Alpha-Beta and Gamma Log-Det Divergences , 2014, Entropy.

[64]  L. Pardo.  Statistical Inference Based on Divergence Measures , 2005 .

[65]  Vittorio Murino,et al.  Multi-class Classification on Riemannian Manifolds for Video Surveillance , 2010, ECCV.

[66]  Tsuyoshi Kato,et al.  Stochastic Dykstra Algorithms for Metric Learning with Positive Definite Covariance Descriptors , 2016, ECCV.

[67]  Vittorio Murino,et al.  Characterizing Humans on Riemannian Manifolds , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[68]  H. Minh,et al.  Some Properties of Gaussian Reproducing Kernel Hilbert Spaces and Their Implications for Function Approximation and Learning Theory , 2010 .

[69]  Alexandre Barachant,et al.  Riemannian geometry for EEG-based brain-computer interfaces; a primer and a review , 2017 .

[70]  Anestis Touloumis,et al.  Nonparametric Stein-type shrinkage covariance matrix estimators in high-dimensional settings , 2014, Comput. Stat. Data Anal..

[71]  Olivier Ledoit,et al.  A well-conditioned estimator for large-dimensional covariance matrices , 2004 .

[72]  Peter Meer,et al.  Semi-Supervised Kernel Mean Shift Clustering , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[73]  Bernhard Schölkopf,et al.  Nonlinear Component Analysis as a Kernel Eigenvalue Problem , 1998, Neural Computation.

[74]  Anoop Cherian,et al.  Riemannian Dictionary Learning and Sparse Coding for Positive Definite Matrices , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[75]  Vittorio Murino,et al.  Approximate Log-Hilbert-Schmidt Distances between Covariance Operators for Image Classification , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[76]  Inderjit S. Dhillon,et al.  Low-Rank Kernel Learning with Bregman Matrix Divergences , 2009, J. Mach. Learn. Res..

[77]  Pol Cirujeda,et al.  MCOV: A Covariance Descriptor for Fusion of Texture and Shape Features in 3D Point Clouds , 2014, 2014 2nd International Conference on 3D Vision.

[78]  Yuan Yao,et al.  Mercer's Theorem, Feature Maps, and Smoothing , 2006, COLT.

[79]  Bernt Schiele,et al.  Analyzing appearance and contour based methods for object categorization , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[80]  Fatih Murat Porikli,et al.  Pedestrian Detection via Classification on Riemannian Manifolds , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[81]  Stefano Mancini,et al.  The volume of Gaussian states by information geometry , 2015, 1509.01049.

[82]  Thomas S. Huang,et al.  Emotion Recognition from Arbitrary View Facial Images , 2010, ECCV.

[83]  Christian Jutten,et al.  Classification of covariance matrices using a Riemannian-based kernel for BCI applications , 2013, Neurocomputing.

[84]  Marc Arnaudon,et al.  Riemannian Medians and Means With Applications to Radar Signal Processing , 2013, IEEE Journal of Selected Topics in Signal Processing.

[85]  Victor Y. Pan,et al.  The complexity of the matrix eigenproblem , 1999, STOC '99.

[86]  Brian C. Lovell,et al.  Spatio-temporal covariance descriptors for action and gesture recognition , 2013, 2013 IEEE Workshop on Applications of Computer Vision (WACV).

[87]  Hamid Laga,et al.  Covariance-Based Descriptors for Efficient 3D Shape Matching, Retrieval, and Classification , 2015, IEEE Transactions on Multimedia.

[88]  I. Dryden,et al.  Non-Euclidean statistics for covariance matrices, with applications to diffusion tensor imaging , 2009, 0910.1656.

[89]  Felipe Cucker,et al.  On the mathematical foundations of learning , 2001 .