On the Role of ML Estimation and Bregman Divergences in Sparse Representation of Covariance and Precision Matrices

Sparse representation of structured signals requires modelling strategies that maintain specific signal properties, in addition to preserving original information content and achieving simpler signal representation. Therefore, the major design challenge is to introduce adequate problem formulations and offer solutions that will efficiently lead to desired representations. In this context, sparse representation of covariance and precision matrices, which appear as feature descriptors or mixture model parameters, respectively, will be in the main focus of this paper.

[1]  Ramesh A. Gopinath,et al.  Maximum likelihood modeling with Gaussian distributions for classification , 1998, Proceedings of the 1998 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP '98 (Cat. No.98CH36181).

[2]  Vassilios Morellas,et al.  Tensor Sparse Coding for Region Covariances , 2010, ECCV.

[3]  Yongqiang Wang,et al.  Modeling inverse covariance matrices by expansion of tied basis matrices for online handwritten Chinese character recognition , 2009, Pattern Recognit..

[4]  Michael Elad,et al.  Sparse and Redundant Representations - From Theory to Applications in Signal and Image Processing , 2010 .

[5]  Vassilios Morellas,et al.  Tensor Sparse Coding for Positive Definite Matrices , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[6]  Robert N. McDonough,et al.  Detection of signals in noise , 1971 .

[7]  Fatih Murat Porikli,et al.  Region Covariance: A Fast Descriptor for Detection and Classification , 2006, ECCV.

[8]  Sang Joon Kim,et al.  A Mathematical Theory of Communication , 2006 .

[9]  Larry S. Davis,et al.  Discriminative Tensor Sparse Coding for Image Classification , 2013, BMVC.

[10]  C. Helstrom,et al.  Statistical theory of signal detection , 1968 .

[11]  David J. Field,et al.  Emergence of simple-cell receptive field properties by learning a sparse code for natural images , 1996, Nature.

[12]  S. Kullback,et al.  Information Theory and Statistics , 1959 .

[13]  H. Akaike.  A new look at the statistical model identification , 1974 .

[14]  Huaiyu Zhu.  On Information and Sufficiency , 1997 .

[15]  Miodrag Lovric,et al.  International Encyclopedia of Statistical Science , 2011 .

[16]  A. Whalen.  Statistical theory of signal detection and parameter estimation , 1984, IEEE Communications Magazine.

[17]  Vladimir S. Crnojevic,et al.  Sparse representation of precision matrices used in GMMs , 2014, Applied Intelligence.

[18]  David J. Field,et al.  Sparse coding with an overcomplete basis set: A strategy employed by V1? , 1997, Vision Research.

[19]  Sailes K. Sengijpta.  Fundamentals of Statistical Signal Processing: Estimation Theory , 1995 .

[20]  Solomon Kullback,et al.  Information Theory and Statistics , 1970, The Mathematical Gazette.

[21]  Peder A. Olsen,et al.  Modeling inverse covariance matrices by basis expansion , 2002, IEEE Transactions on Speech and Audio Processing.

[22]  Michael Elad,et al.  On the Role of Sparse and Redundant Representations in Image Processing , 2010, Proceedings of the IEEE.

[23]  Thomas M. Cover,et al.  Elements of Information Theory (Wiley Series in Telecommunications and Signal Processing) , 2006 .

[24]  Scott Axelrod,et al.  Modeling with a subspace constraint on inverse covariance matrices , 2002, INTERSPEECH.

[25]  S. Sra.  Positive definite matrices and the S-divergence , 2011, 1110.1773.

[26]  Ananth Sankar,et al.  Mixtures of inverse covariances , 2003, IEEE Transactions on Speech and Audio Processing.