Multivariate Generalized Gaussian Distribution: Convexity and Graphical Models

We consider covariance estimation in the multivariate generalized Gaussian distribution (MGGD) and elliptically symmetric (ES) distribution. The maximum likelihood optimization associated with this problem is non-convex, yet it has been proved that its global solution can be often computed via simple fixed point iterations. Our first contribution is a new analysis of this likelihood based on geodesic convexity that requires weaker assumptions. Our second contribution is a generalized framework for structured covariance estimation under sparsity constraints. We show that the optimizations can be formulated as convex minimization as long the MGGD shape parameter is larger than half and the sparsity pattern is chordal. These include, for example, maximum likelihood estimation of banded inverse covariances in multivariate Laplace distributions, which are associated with time varying autoregressive processes.

[1]  Olcay Arslan,et al.  Convergence behavior of an iterative reweighting algorithm to compute multivariate M-estimates for location and scatter , 2004 .

[2]  Mathias Drton,et al.  Robust graphical modeling of gene networks using classical and alternative t-distributions , 2010, 1009.3669.

[3]  Teng Zhang.  Robust subspace recovery by geodesically convex optimization , 2012, 1206.1386.

[4]  Peter Buhlmann,et al.  High dimensional sparse covariance estimation via directed acyclic graphs , 2009, 0911.2375.

[5]  M. A. Chmielewski,et al.  Elliptically Symmetric Distributions: A Review and Bibliography , 1981 .

[6]  Fang Han,et al.  Transelliptical Graphical Models , 2012, NIPS.

[7]  Alfred O. Hero,et al.  Decomposable Principal Component Analysis , 2009, IEEE Transactions on Signal Processing.

[8]  T. Rapcsák.  Geodesic convexity in nonlinear optimization , 1991 .

[9]  P. Bickel,et al.  Regularized estimation of large covariance matrices , 2008, 0803.1909.

[10]  R. Bhatia.  Positive Definite Matrices , 2007 .

[11]  BessonOlivier,et al.  Regularized Covariance Matrix Estimation in Complex Elliptically Symmetric Distributions Using the Expected Likelihood Approach—Part 2 , 2013 .

[12]  Malene Højbjerre,et al.  Lecture Notes in Statistics 101: Linear and Graphical Models for the Multivariate Complex Normal Distribution , 1995 .

[13]  Te-Won Lee,et al.  On the multivariate Laplace distribution , 2006, IEEE Signal Processing Letters.

[14]  Jean-Yves Tourneret,et al.  Parameter Estimation For Multivariate Generalized Gaussian Distributions , 2013, IEEE Transactions on Signal Processing.

[15]  Ami Wiesel,et al.  Geodesic Convexity and Covariance Estimation , 2012, IEEE Transactions on Signal Processing.

[16]  E. Ruh,et al.  Angular Gaussian and Cauchy estimation , 2005 .

[17]  Peter J. Huber,et al.  Robust Statistics , 2005, Wiley Series in Probability and Statistics.

[18]  Alfred O. Hero,et al.  Covariance Estimation in Decomposable Gaussian Graphical Models , 2009, IEEE Transactions on Signal Processing.

[19]  Bala Rajaratnam,et al.  Natural order recovery for banded covariance models , 2012, 2012 IEEE 7th Sensor Array and Multichannel Signal Processing Workshop (SAM).

[20]  Ami Wiesel,et al.  Unified Framework to Regularized Covariance Estimation in Scaled Gaussian Models , 2012, IEEE Transactions on Signal Processing.

[21]  Yuri I. Abramovich,et al.  Time-Varying Autoregressive (TVAR) Models for Multiple Radar Observations , 2007, IEEE Transactions on Signal Processing.

[22]  Vwani P. Roychowdhury,et al.  Covariance selection for nonchordal graphs via chordal embedding , 2008, Optim. Methods Softw..

[23]  H. Vincent Poor,et al.  Complex Elliptically Symmetric Distributions: Survey, New Results and Applications , 2012, IEEE Transactions on Signal Processing.

[24]  B. Ripley,et al.  Robust Statistics , 2018, Encyclopedia of Mathematical Geosciences.

[25]  Alfred O. Hero,et al.  Robust Shrinkage Estimation of High-Dimensional Covariance Matrices , 2010, IEEE Transactions on Signal Processing.

[26]  Philippe Forster,et al.  Covariance Structure Maximum-Likelihood Estimates in Compound Gaussian Noise: Existence and Algorithm Analysis , 2008, IEEE Transactions on Signal Processing.

[27]  Michael I. Jordan.  Graphical Models , 2003 .

[28]  Jean-Yves Tourneret,et al.  Performance of the maximum likelihood estimators for the parameters of multivariate generalized Gaussian distributions , 2012, 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[29]  Tülay Adali,et al.  A Complex Generalized Gaussian Distribution— Characterization, Generation, and Estimation , 2010, IEEE Transactions on Signal Processing.

[30]  Kazuo Murota,et al.  Exploiting Sparsity in Semidefinite Programming via Matrix Completion I: General Framework , 2000, SIAM J. Optim..

[31]  Yuri I. Abramovich,et al.  Expected likelihood approach for covariance matrix estimation: Complex angular central Gaussian case , 2012, 2012 IEEE 7th Sensor Array and Multichannel Signal Processing Workshop (SAM).

[32]  Adam J. Rothman,et al.  A new approach to Cholesky-based covariance regularization in high dimensions , 2009, 0903.0645.

[33]  David E. Tyler,et al.  Redescending $M$-Estimates of Multivariate Location and Scatter , 1991 .

[34]  Constantin P. Niculescu,et al.  Convex Functions and Their Applications: A Contemporary Approach , 2005 .

[35]  David E. Tyler.  A Distribution-Free $M$-Estimator of Multivariate Scatter , 1987 .