Weighted SPICE: A unifying approach for hyperparameter-free sparse estimation

Abstract In this paper we present the SPICE approach for sparse parameter estimation in a framework that unifies it with other hyperparameter-free methods, namely LIKES, SLIM and IAA. 1 Specifically, we show how the latter methods can be interpreted as variants of an adaptively reweighted SPICE method. Furthermore, we establish a connection between SPICE and the l 1 -penalized LAD estimator as well as the square-root LASSO method. We evaluate the four methods mentioned above in a generic sparse regression problem and in an array processing application.

[1]  Michael E. Tipping Sparse Bayesian Learning and the Relevance Vector Machine , 2001, J. Mach. Learn. Res..

[2]  E. M. L. Beale,et al.  Nonlinear Programming: A Unified Approach. , 1970 .

[3]  R. Tibshirani Regression Shrinkage and Selection via the Lasso , 1996 .

[4]  Raffaele Grasso,et al.  Single-snapshot DOA estimation by using Compressed Sensing , 2014, EURASIP Journal on Advances in Signal Processing.

[5]  C. Kelley Iterative Methods for Linear and Nonlinear Equations , 1987 .

[6]  H. Carfantan,et al.  A Sparsity-Based Method for the Estimation of Spectral Lines From Irregularly Sampled Data , 2007, IEEE Journal of Selected Topics in Signal Processing.

[7]  A. Belloni,et al.  Square-Root Lasso: Pivotal Recovery of Sparse Signals via Conic Programming , 2011 .

[8]  D. Donoho,et al.  Atomic Decomposition by Basis Pursuit , 2001 .

[9]  Jian Li,et al.  New Method of Sparse Parameter Estimation in Separable Models and Its Use for Spectral Analysis of Irregularly Sampled Data , 2011, IEEE Transactions on Signal Processing.

[10]  Jian Li,et al.  Source Localization and Sensing: A Nonparametric Iterative Adaptive Approach Based on Weighted Least Squares , 2010, IEEE Transactions on Aerospace and Electronic Systems.

[11]  Lie Wang The L1L1 penalized LAD estimator for high dimensional linear regression , 2013, J. Multivar. Anal..

[12]  Joel A. Tropp,et al.  Just relax: convex programming methods for identifying sparse signals in noise , 2006, IEEE Transactions on Information Theory.

[13]  Jian Li,et al.  Iterative Adaptive Approaches to MIMO Radar Imaging , 2010, IEEE Journal of Selected Topics in Signal Processing.

[14]  M. Lustig,et al.  Compressed Sensing MRI , 2008, IEEE Signal Processing Magazine.

[15]  Jian Li,et al.  Sparse Learning via Iterative Minimization With Application to MIMO Radar Imaging , 2011, IEEE Transactions on Signal Processing.

[16]  Jean-Jacques Fuchs,et al.  On sparse representations in arbitrary redundant bases , 2004, IEEE Transactions on Information Theory.

[17]  Michael B. Wakin,et al.  An Introduction To Compressive Sampling [A sensing/sampling paradigm that goes against the common knowledge in data acquisition] , 2008 .

[18]  Petre Stoica,et al.  Spectral Analysis of Signals , 2009 .

[19]  Nikos D. Sidiropoulos,et al.  Parallel factor analysis in sensor array processing , 2000, IEEE Trans. Signal Process..

[20]  E.J. Candes,et al.  An Introduction To Compressive Sampling , 2008, IEEE Signal Processing Magazine.

[21]  Volker Roth,et al.  The generalized LASSO , 2004, IEEE Transactions on Neural Networks.

[22]  Petre Stoica,et al.  Connection between SPICE and Square-Root LASSO for sparse parameter estimation , 2014, Signal Process..

[23]  Petre Stoica,et al.  SPICE and LIKES: Two hyperparameter-free methods for sparse-parameter estimation , 2012, Signal Process..

[24]  D. Ruppert The Elements of Statistical Learning: Data Mining, Inference, and Prediction , 2004 .

[25]  Nikos D. Sidiropoulos,et al.  Almost-sure identifiability of multidimensional harmonic retrieval , 2001, IEEE Trans. Signal Process..

[26]  Bhaskar D. Rao,et al.  Sparse signal reconstruction from limited data using FOCUSS: a re-weighted minimum norm algorithm , 1997, IEEE Trans. Signal Process..

[27]  P. Stoica,et al.  Cyclic minimizers, majorization techniques, and the expectation-maximization algorithm: a refresher , 2004, IEEE Signal Process. Mag..

[28]  Emmanuel J. Candès,et al.  Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information , 2004, IEEE Transactions on Information Theory.

[29]  Jian Li,et al.  SPICE: A Sparse Covariance-Based Estimation Method for Array Processing , 2011, IEEE Transactions on Signal Processing.

[30]  Michael Elad,et al.  Sparse and Redundant Representations - From Theory to Applications in Signal and Image Processing , 2010 .

[31]  Håkan Hjalmarsson,et al.  A Note on the SPICE Method , 2012, IEEE Transactions on Signal Processing.

[32]  Stephen P. Boyd,et al.  Enhancing Sparsity by Reweighted ℓ1 Minimization , 2007, 0711.1612.

[33]  Dmitry M. Malioutov,et al.  A sparse signal reconstruction perspective for source localization with sensor arrays , 2005, IEEE Transactions on Signal Processing.

[34]  R. Tibshirani,et al.  Least angle regression , 2004, math/0406456.

[35]  Björn E. Ottersten,et al.  Covariance Matching Estimation Techniques for Array Signal Processing Applications , 1998, Digit. Signal Process..

[36]  Robert Tibshirani,et al.  The Elements of Statistical Learning: Data Mining, Inference, and Prediction, 2nd Edition , 2001, Springer Series in Statistics.

[37]  Arian Maleki,et al.  Optimally Tuned Iterative Reconstruction Algorithms for Compressed Sensing , 2009, IEEE Journal of Selected Topics in Signal Processing.

[38]  Steven Kay,et al.  Fundamentals Of Statistical Signal Processing , 2001 .

[39]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.