Covalsa: Covariance estimation from compressive measurements using alternating minimization

The estimation of covariance matrices from compressive measurements has recently attracted considerable research efforts in various fields of science and engineering. Owing to the small number of observations, the estimation of the covariance matrices is a severely ill-posed problem. This can be overcome by exploiting prior information about the structure of the covariance matrix. This paper presents a class of convex formulations and respective solutions to the high-dimensional covariance matrix estimation problem under compressive measurements, imposing either Toeplitz, sparseness, null-pattern, low rank, or low permuted rank structure on the solution, in addition to positive semi-definiteness. To solve the optimization problems, we introduce the Co-Variance by Augmented Lagrangian Shrinkage Algorithm (CoVALSA), which is an instance of the Split Augmented Lagrangian Shrinkage Algorithm (SALSA). We illustrate the effectiveness of our approach in comparison with state-of-the-art algorithms.

[1]  A. U.S.,et al.  Sparse Estimation of a Covariance Matrix , 2010 .

[2]  José M. Bioucas-Dias,et al.  An Augmented Lagrangian Approach to the Constrained Optimization Formulation of Imaging Inverse Problems , 2009, IEEE Transactions on Image Processing.

[3]  Qing Ling,et al.  High-dimensional sparse covariance estimation for random signals , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[4]  Petre Stoica,et al.  On Estimation of Covariance Matrices With Kronecker Product Structure , 2008, IEEE Transactions on Signal Processing.

[5]  Parikshit Shah,et al.  Sketching Sparse Matrices , 2013, ArXiv.

[6]  John S. Thompson,et al.  Compressive power spectral density estimation , 2011, 2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[7]  Adam J. Rothman,et al.  Sparse permutation invariant covariance estimation , 2008, 0801.4837.

[8]  Geert Leus,et al.  Compressive Wideband Power Spectrum Estimation , 2012, IEEE Transactions on Signal Processing.

[9]  Yonina C. Eldar,et al.  Recovery of Sparse Matrices via Matrix Sketching , 2013, ArXiv.

[10]  Ali Jalali,et al.  High-dimensional Sparse Inverse Covariance Estimation using Greedy Methods , 2011, AISTATS.

[11]  P. Bickel,et al.  Banded regularization of autocovariance matrices in application to parameter estimation and forecasting of time series , 2011 .

[12]  Leon Hirsch,et al.  Fundamentals Of Convex Analysis , 2016 .

[13]  B. Mercier,et al.  A dual algorithm for the solution of nonlinear variational problems via finite element approximation , 1976 .

[14]  I. Johnstone,et al.  On Consistency and Sparsity for Principal Components Analysis in High Dimensions , 2009, Journal of the American Statistical Association.

[15]  Alfred O. Hero,et al.  On Convergence of Kronecker Graphical Lasso Algorithms , 2012, IEEE Transactions on Signal Processing.

[16]  P. Bickel,et al.  Covariance regularization by thresholding , 2009, 0901.3079.

[17]  Alfred O. Hero,et al.  Covariance Estimation in High Dimensions Via Kronecker Product Expansions , 2013, IEEE Transactions on Signal Processing.

[18]  Bin Yu,et al.  High-dimensional covariance estimation by minimizing ℓ1-penalized log-determinant divergence , 2008, 0811.3628.

[19]  Karim Lounici High-dimensional covariance matrix estimation with missing observations , 2012, 1201.2577.

[20]  Yonina C. Eldar,et al.  Sub-Nyquist Sampling for Power Spectrum Sensing in Cognitive Radios: A Unified Approach , 2013, IEEE Transactions on Signal Processing.

[21]  S. Frick,et al.  Compressed Sensing , 2014, Computer Vision, A Reference Guide.

[22]  Danijela Cabric,et al.  High-resolution cyclic spectrum reconstruction from sub-Nyquist samples , 2013, 2013 IEEE 14th Workshop on Signal Processing Advances in Wireless Communications (SPAWC).

[23]  Patrick L. Combettes,et al.  Proximal Splitting Methods in Signal Processing , 2009, Fixed-Point Algorithms for Inverse Problems in Science and Engineering.

[24]  R. Tibshirani,et al.  Sparse estimation of a covariance matrix. , 2011, Biometrika.

[25]  Dimitri P. Bertsekas,et al.  On the Douglas—Rachford splitting method and the proximal point algorithm for maximal monotone operators , 1992, Math. Program..

[26]  J. Hiriart-Urruty,et al.  Fundamentals of Convex Analysis , 2004 .

[27]  T. Blumensath,et al.  Theory and Applications , 2011 .