GMMs similarity measure based on LPP-like projection of the parameter space

We propose the novel more efficient similarity measure between GMMs.It is done by projecting GMMs from high dimensional to a lower dimensional space.GMMs distance is reduced to the distance between lower dimensional euclidian vectors.Greater discriminativity and lower computational cost is obtained.We confirm our results on artificial and real experimental data. The need for a comparison between two Gaussian Mixture Models (GMMs) plays a crucial role in various pattern recognition tasks and is involved as a key components in many expert and artificial intelligence (AI) systems dealing with real-life problems. As those system often operate on large data-sets and use high dimensional features, it is crucial for their recognition component to be computationally efficient in addition to its good recognition accuracy. In this work we deliver the novel similarity measure between GMMs, by LPP-like projecting the components of a particular GMM, from the high dimensional original parameter space, to a much lower dimensional space. Thus, finding the distance between two GMMs in the original space is reduced to finding the distance between sets of lower dimensional Euclidian vectors, pondered by corresponding weights. By doing so, we manage to obtain much better trade-off between the recognition accuracy and the computational complexity, in comparison to the measures between GMMs utilizing distances between Gaussian components evaluated in the original parameter space. Thus, the GMM measure that we propose is suitable for applications in AI systems that use GMMs in their recognition tasks and operate on large data sets, as the required number of overall Gaussian components involved in such systems is always large. We evaluate the proposed GMM measure on artificial, as well as real-world experimental data obtaining a much better trade-off between recognition accuracy and the computational complexity, in comparison to all baseline GMM similarity measures tested.

[1]  Leonidas J. Guibas,et al.  The Earth Mover's Distance as a Metric for Image Retrieval , 2000, International Journal of Computer Vision.

[2]  Shree K. Nayar,et al.  Reflectance and texture of real-world surfaces , 1999, TOGS.

[3]  Miroslav Lovric,et al.  Multivariate Normal Distributions Parametrized as a Riemannian Symmetric Space , 2000 .

[4]  John R. Hershey,et al.  Approximating the Kullback Leibler Divergence Between Gaussian Mixture Models , 2007, 2007 IEEE International Conference on Acoustics, Speech and Signal Processing - ICASSP '07.

[5]  Xiaoyang Tan,et al.  Pattern Recognition , 2016, Communications in Computer and Information Science.

[6]  Solomon Kullback,et al.  Information Theory and Statistics , 1970, The Mathematical Gazette.

[7]  Yong Xu,et al.  A Projective Invariant for Textures , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[8]  M. Turk,et al.  Eigenfaces for Recognition , 1991, Journal of Cognitive Neuroscience.

[9]  Mario Fritz,et al.  On the Significance of Real-World Conditions for Material Classification , 2004, ECCV.

[10]  Lei Zhang,et al.  A Novel Earth Mover's Distance Methodology for Image Matching with Gaussian Mixture Models , 2013, 2013 IEEE International Conference on Computer Vision.

[11]  Hongbin Zha,et al.  Sorted Random Projections for robust texture classification , 2011, 2011 International Conference on Computer Vision.

[12]  Daoqiang Zhang,et al.  Efficient and robust feature extraction by maximum margin criterion , 2003, IEEE Transactions on Neural Networks.

[13]  Larry S. Davis,et al.  Discriminative Tensor Sparse Coding for Image Classification , 2013, BMVC.

[14]  Hagai Aronowitz,et al.  A distance measure between GMMs based on the unscented transform and its application to speaker recognition , 2005, INTERSPEECH.

[15]  J. Douglas Faires,et al.  Numerical Analysis , 1981 .

[16]  Cordelia Schmid,et al.  Local Features and Kernels for Classification of Texture and Object Categories: A Comprehensive Study , 2006, 2006 Conference on Computer Vision and Pattern Recognition Workshop (CVPRW'06).

[17]  Shiri Gordon,et al.  An efficient image similarity measure based on approximations of KL-divergence between two gaussian mixtures , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[18]  George J. Klir,et al.  Fuzzy sets and fuzzy logic - theory and applications , 1995 .

[19]  Haibin Ling,et al.  An Efficient Earth Mover's Distance Algorithm for Robust Histogram Comparison , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[20]  Yong Xu,et al.  Wavelet Domain Multifractal Analysis for Static and Dynamic Texture Classification , 2013, IEEE Transactions on Image Processing.

[21]  Andrew R. Webb,et al.  Statistical Pattern Recognition , 1999 .

[22]  David J. Kriegman,et al.  Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection , 1996, ECCV.

[23]  Vassilios Morellas,et al.  Tensor Sparse Coding for Region Covariances , 2010, ECCV.

[24]  Joshua B. Tenenbaum,et al.  Mapping a Manifold of Perceptual Observations , 1997, NIPS.

[25]  Jean-Philippe Thiran,et al.  Lower and upper bounds for approximation of the Kullback-Leibler divergence between Gaussian Mixture Models , 2012, 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[26]  Zhenhua Guo,et al.  A Completed Modeling of Local Binary Pattern Operator for Texture Classification , 2010, IEEE Transactions on Image Processing.

[27]  Andrew Zisserman,et al.  Texture classification: are filter banks necessary? , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[28]  René Vidal,et al.  Clustering and dimensionality reduction on Riemannian manifolds , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[29]  Xiaofei He,et al.  Locality Preserving Projections , 2003, NIPS.

[30]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[31]  Mikhail Belkin,et al.  Laplacian Eigenmaps and Spectral Techniques for Embedding and Clustering , 2001, NIPS.

[32]  Hongdong Li,et al.  Kernel Methods on the Riemannian Manifold of Symmetric Positive Definite Matrices , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[33]  S T Roweis,et al.  Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.