Sparsity estimation from compressive projections via sparse random matrices

The aim of this paper is to develop strategies to estimate the sparsity degree of a signal from compressive projections, without the burden of recovery. We consider both the noise-free and the noisy settings, and we show how to extend the proposed framework to the case of non-exactly sparse signals. The proposed method employs γ-sparsified random matrices and is based on a maximum likelihood (ML) approach, exploiting the property that the acquired measurements are distributed according to a mixture model whose parameters depend on the signal sparsity. In the presence of noise, given the complexity of ML estimation, the probability model is approximated with a two-component Gaussian mixture (2-GMM), which can be easily learned via expectation-maximization.Besides the design of the method, this paper makes two novel contributions. First, in the absence of noise, sufficient conditions on the number of measurements are provided for almost sure exact estimation in different regimes of behavior, defined by the scaling of the measurements sparsity γ and the signal sparsity. In the presence of noise, our second contribution is to prove that the 2-GMM approximation is accurate in the large system limit for a proper choice of γ parameter. Simulations validate our predictions and show that the proposed algorithms outperform the state-of-the-art methods for sparsity estimation. Finally, the estimation strategy is applied to non-exactly sparse signals. The results are very encouraging, suggesting further extension to more general frameworks.

[1]  Dmitry M. Malioutov,et al.  Compressed sensing with sequential observations , 2008, 2008 IEEE International Conference on Acoustics, Speech and Signal Processing.

[2]  Hyundong Shin,et al.  Joint Channel Identification and Estimation in Wireless Network: Sparsity and Optimization , 2018, IEEE Transactions on Wireless Communications.

[3]  Symeon Chatzinotas,et al.  Compressive Sparsity Order Estimation for Wideband Cognitive Radio Receiver , 2014, IEEE Transactions on Signal Processing.

[4]  Chen Li,et al.  Distributed Compressive Spectrum Sensing in Cooperative Multihop Cognitive Networks , 2011, IEEE Journal of Selected Topics in Signal Processing.

[5]  Joel A. Tropp,et al.  Signal Recovery From Random Measurements Via Orthogonal Matching Pursuit , 2007, IEEE Transactions on Information Theory.

[6]  S. Frick,et al.  Compressed Sensing , 2014, Computer Vision, A Reference Guide.

[7]  Laurent El Ghaoui,et al.  An Homotopy Algorithm for the Lasso with Online Observations , 2008, NIPS.

[8]  W. Feller,et al.  An Introduction to Probability Theory and Its Applications, Vol. 1 , 1967 .

[9]  W. Hoeffding Probability Inequalities for sums of Bounded Random Variables , 1963 .

[10]  Enrico Magli,et al.  On the fly estimation of the sparsity degree in Compressed Sensing using sparse sensing matrices , 2015, 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[11]  T. Apostol Multi-variable calculus and linear algebra, with applications to differential equations and probability , 1969 .

[12]  Rajeev Rastogi,et al.  Data Stream Management: Processing High-Speed Data Streams (Data-Centric Systems and Applications) , 2019 .

[13]  Deanna Needell,et al.  CoSaMP: Iterative signal recovery from incomplete and inaccurate samples , 2008, ArXiv.

[14]  E. Candès,et al.  Stable signal recovery from incomplete and inaccurate measurements , 2005, math/0503066.

[15]  E. Candès The restricted isometry property and its implications for compressed sensing , 2008 .

[16]  Yonina C. Eldar Generalized SURE for Exponential Families: Applications to Regularization , 2008, IEEE Transactions on Signal Processing.

[17]  Anil K. Jain,et al.  Unsupervised Learning of Finite Mixture Models , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[18]  M. Salman Asif,et al.  Streaming measurements in compressive sensing: ℓ1 filtering , 2008, 2008 42nd Asilomar Conference on Signals, Systems and Computers.

[19]  Chunyan Feng,et al.  Sparsity Order Estimation and its Application in Compressive Spectrum Sensing for Cognitive Radios , 2012, IEEE Transactions on Wireless Communications.

[20]  Rachel Ward,et al.  Compressed Sensing With Cross Validation , 2008, IEEE Transactions on Information Theory.

[21]  Rajeev Rastogi,et al.  Data Stream Management , 2016, Data-Centric Systems and Applications.

[22]  Mário A. T. Figueiredo,et al.  Gradient Projection for Sparse Reconstruction: Application to Compressed Sensing and Other Inverse Problems , 2007, IEEE Journal of Selected Topics in Signal Processing.

[23]  Luca Trevisan,et al.  Counting Distinct Elements in a Data Stream , 2002, RANDOM.

[24]  Miles E. Lopes Estimating Unknown Sparsity in Compressed Sensing , 2013 .

[25]  Emmanuel J. Candès,et al.  Near-Optimal Signal Recovery From Random Projections: Universal Encoding Strategies? , 2004, IEEE Transactions on Information Theory.

[26]  Lu Gan Block Compressed Sensing of Natural Images , 2007, 2007 15th International Conference on Digital Signal Processing.

[27]  Enrico Magli,et al.  SparseHash: Embedding Jaccard coefficient between supports of signals , 2016, 2016 IEEE International Conference on Multimedia & Expo Workshops (ICMEW).

[28]  Piotr Indyk,et al.  Sparse Recovery Using Sparse Matrices , 2010, Proceedings of the IEEE.

[29]  William Feller,et al.  An Introduction to Probability Theory and Its Applications , 1967 .

[30]  Martin J. Wainwright,et al.  Information-Theoretic Limits on Sparse Signal Recovery: Dense versus Sparse Measurement Matrices , 2008, IEEE Transactions on Information Theory.

[31]  Feller William,et al.  An Introduction To Probability Theory And Its Applications , 1950 .

[32]  Enrico Magli,et al.  Signal sparsity estimation from compressive noisy projections via γ-sparsified random matrices , 2016, 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[33]  Miles E. Lopes Unknown Sparsity in Compressed Sensing: Denoising and Inference , 2015, IEEE Transactions on Information Theory.

[34]  Emmanuel J. Candès,et al.  On the Fundamental Limits of Adaptive Sensing , 2011, IEEE Transactions on Information Theory.

[35]  Martin J. Wainwright,et al.  High-dimensional Variable Selection with Sparse Random Projections: Measurement Sparsity and Statistical Efficiency , 2010, J. Mach. Learn. Res..

[36]  P. Indyk,et al.  Near-Optimal Sparse Recovery in the L1 Norm , 2008, 2008 49th Annual IEEE Symposium on Foundations of Computer Science.

[37]  Jian Yang,et al.  Zero-point attracting projection algorithm for sequential compressive sensing , 2013, IEICE Electron. Express.

[38]  Ping Li,et al.  Compressed Sensing with Very Sparse Gaussian Random Projections , 2014, AISTATS.

[39]  Qi Zhang,et al.  Sparsity estimation in image compressive sensing , 2012, 2012 IEEE International Symposium on Circuits and Systems.

[40]  Richard G. Baraniuk,et al.  Asymptotic Analysis of LASSOs Solution Path with Implications for Approximate Message Passing , 2013, ArXiv.

[41]  New York Dover,et al.  ON THE CONVERGENCE PROPERTIES OF THE EM ALGORITHM , 1983 .