Image classification based on log-Euclidean Fisher Vectors for covariance matrix descriptors

This paper introduces an image classification method based on the encoding of a set of covariance matrices. This encoding relies on Fisher vectors adapted to the log-Euclidean metric: the log-Euclidean Fisher vectors (LE FV). This approach is next extended to full local Gaussian descriptors composed by a set of local mean vectors and local covariance matrices. For that, the local Gaussian model is transformed to a zero-mean Gaussian model with an augmented covariance matrix. All these approaches are used to encode handcrafted or deep learning features. Finally, they are applied in a remote sensing application on the UC Merced dataset which consists in classifying land cover images. A sensitivity analysis is carried out to evaluate the potential of the proposed LE FV.

[1]  Yannick Berthoumieu,et al.  Covariance Matrices Encoding Based on the Log-Euclidean and Affine Invariant Riemannian Metrics , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[2]  Thomas Mensink,et al.  Improving the Fisher Kernel for Large-Scale Image Classification , 2010, ECCV.

[3]  Cordelia Schmid,et al.  Aggregating local descriptors into a compact image representation , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[4]  Laurent Ferro-Famil,et al.  Statistical Classification for Heterogeneous Polarimetric SAR Images , 2011, IEEE Journal of Selected Topics in Signal Processing.

[5]  Florent Perronnin,et al.  Fisher Kernels on Visual Vocabularies for Image Categorization , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[6]  Alexei A. Efros,et al.  Discovering objects and their location in images , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[7]  Marc Donias,et al.  Structure Tensor Riemannian Statistical Models for CBIR and Classification of Remote Sensing Images , 2017, IEEE Transactions on Geoscience and Remote Sensing.

[8]  Florent Perronnin,et al.  Fisher vectors meet Neural Networks: A hybrid classification architecture , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[9]  Shawn D. Newsam,et al.  Bag-of-visual-words and spatial extensions for land-use classification , 2010, GIS '10.

[10]  Brian C. Lovell,et al.  Fisher tensors for classifying human epithelial cells , 2014, Pattern Recognit..

[11]  Miroslav Lovric,et al.  Multivariate Normal Distributions Parametrized as a Riemannian Symmetric Space , 2000 .

[12]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[13]  Andrew Zisserman,et al.  All About VLAD , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[14]  Thomas Mensink,et al.  Image Classification with the Fisher Vector: Theory and Practice , 2013, International Journal of Computer Vision.

[15]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[16]  Mehrtash Harandi,et al.  Dimensionality Reduction on SPD Manifolds: The Emergence of Geometry-Aware Methods , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[17]  Florent Perronnin,et al.  Large-scale image retrieval with compressed Fisher vectors , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[18]  N. Ayache,et al.  Log‐Euclidean metrics for fast and simple calculus on diffusion tensors , 2006, Magnetic resonance in medicine.

[19]  Matthijs C. Dorst.  Distinctive Image Features from Scale-Invariant Keypoints , 2011 .

[20]  Mehrtash Tafazzoli Harandi,et al.  More about VLAD: A leap from Euclidean to Riemannian manifolds , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[21]  Yannick Berthoumieu,et al.  Texture Classification Using Rao's Distance on the Space of Covariance Matrices , 2015, GSI.

[22]  Cordelia Schmid,et al.  Combining attributes and Fisher vectors for efficient image retrieval , 2011, CVPR 2011.

[23]  Gerard Salton,et al.  Term-Weighting Approaches in Automatic Text Retrieval , 1988, Inf. Process. Manag..

[24]  Josef Sivic,et al.  NetVLAD: CNN Architecture for Weakly Supervised Place Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[25]  Yannick Berthoumieu,et al.  Fisher Vector Coding for Covariance Matrix Descriptors Based on the Log-Euclidean and Affine Invariant Riemannian Metrics , 2018, J. Imaging.

[26]  Iasonas Kokkinos,et al.  Deep Filter Banks for Texture Recognition, Description, and Segmentation , 2015, International Journal of Computer Vision.

[27]  Cong Lin,et al.  Integrating Multilayer Features of Convolutional Neural Networks for Remote Sensing Scene Classification , 2017, IEEE Transactions on Geoscience and Remote Sensing.

[28]  Larry S. Davis,et al.  Exploiting local features from deep networks for image retrieval , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[29]  Christian Germain,et al.  Texture image classification with Riemannian fisher vectors , 2016, 2016 IEEE International Conference on Image Processing (ICIP).

[30]  Shiguang Shan,et al.  Log-Euclidean Metric Learning on Symmetric Positive Definite Manifold with Application to Image Set Classification , 2015, ICML.

[31]  Conrad Sanderson,et al.  Log-Euclidean bag of words for human action recognition , 2014, IET Comput. Vis..

[32]  Andrew Zisserman,et al.  Deep Fisher Networks for Large-Scale Image Classification , 2013, NIPS.

[33]  Luc Van Gool,et al.  Deep visual words: Improved fisher vector for image classification , 2017, 2017 Fifteenth IAPR International Conference on Machine Vision Applications (MVA).

[34]  Christian Jutten,et al.  Classification of covariance matrices using a Riemannian-based kernel for BCI applications , 2013, Neurocomputing.

[35]  Stephen J. Maybank,et al.  Human Action Recognition under Log-Euclidean Riemannian Metric , 2009, ACCV.