Subspace Recovery From Structured Union of Subspaces

Lower dimensional signal representation schemes frequently assume that the signal of interest lies in a single vector space. In the context of the recently developed theory of compressive sensing, it is often assumed that the signal of interest is sparse in an orthonormal basis. However, in many practical applications, this requirement may be too restrictive. A generalization of the standard sparsity assumption is that the signal lies in a union of subspaces. Recovery of such signals from a small number of samples has been studied recently in several works. Here, we consider the problem of only subspace recovery in which our goal is to identify the subspace (from the union) in which the signal lies using a small number of samples, in the presence of noise. More specifically, we derive performance bounds and conditions under which reliable subspace recovery is guaranteed using maximum likelihood (ML) estimation. We begin by treating general unions and then obtain the results for the special case in which the subspaces have structure leading to block sparsity. In our analysis, we treat both general sampling operators and random sampling matrices. With general unions, we show that under certain conditions, the number of measurements required for reliable subspace recovery in the presence of noise via ML is less than that implied using the restricted isometry property, which guarantees complete signal recovery. In the special case of block sparse signals, we quantify the gain achievable over standard sparsity in subspace recovery. Our results also strengthen existing results on sparse support recovery in the presence of noise under the standard sparsity model.

[1]  Minh N. Do,et al.  A Theory for Sampling Signals from a Union of Subspaces , 2022 .

[2]  Galen Reeves,et al.  Sampling bounds for sparse support recovery in the presence of noise , 2008, 2008 IEEE International Symposium on Information Theory.

[3]  Alfred M. Bruckstein,et al.  The resolution of overlapping echos , 1985, IEEE Trans. Acoust. Speech Signal Process..

[4]  V.K. Goyal,et al.  Compressive Sampling and Lossy Compression , 2008, IEEE Signal Processing Magazine.

[5]  Martin J. Wainwright,et al.  Sharp Thresholds for High-Dimensional and Noisy Sparsity Recovery Using $\ell _{1}$ -Constrained Quadratic Programming (Lasso) , 2009, IEEE Transactions on Information Theory.

[6]  Jun Fang,et al.  Block-sparsity pattern recovery from noisy observations , 2012, 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[7]  Vahid Tarokh,et al.  Shannon-Theoretic Limits on Noisy Compressive Sampling , 2007, IEEE Transactions on Information Theory.

[8]  R. Tibshirani,et al.  A note on the group lasso and a sparse group lasso , 2010, 1001.0736.

[9]  Martin J. Wainwright,et al.  Information-Theoretic Limits on Sparse Signal Recovery: Dense versus Sparse Measurement Matrices , 2008, IEEE Transactions on Information Theory.

[10]  Yonina C. Eldar,et al.  Time-Delay Estimation From Low-Rate Samples: A Union of Subspaces Approach , 2009, IEEE Transactions on Signal Processing.

[11]  Alan J. Miller,et al.  Subset Selection in Regression , 1991 .

[12]  Pramod K. Varshney,et al.  Performance Bounds for Sparsity Pattern Recovery With Quantized Noisy Random Projections , 2012, IEEE Journal of Selected Topics in Signal Processing.

[13]  David L Donoho,et al.  Compressed sensing , 2006, IEEE Transactions on Information Theory.

[14]  Francis R. Bach,et al.  Consistency of the group Lasso and multiple kernel learning , 2007, J. Mach. Learn. Res..

[15]  Erik G. Larsson,et al.  Linear Regression With a Sparse Parameter Vector , 2007, IEEE Trans. Signal Process..

[16]  Georgios B. Giannakis,et al.  Compressed Sensing for Wideband Cognitive Radios , 2007, 2007 IEEE International Conference on Acoustics, Speech and Signal Processing - ICASSP '07.

[17]  Daniel Percival Theoretical Properties of the Overlapping Groups Lasso , 2011, 1103.4614.

[18]  John G. Proakis,et al.  Probability, random variables and stochastic processes , 1985, IEEE Trans. Acoust. Speech Signal Process..

[19]  Gongguo Tang,et al.  Performance Analysis for Sparse Support Recovery , 2009, IEEE Transactions on Information Theory.

[20]  Yonina C. Eldar,et al.  Blind Multiband Signal Reconstruction: Compressed Sensing for Analog Signals , 2007, IEEE Transactions on Signal Processing.

[21]  Yonina C. Eldar,et al.  Xampling: Analog to digital at sub-Nyquist rates , 2009, IET Circuits Devices Syst..

[22]  I. S. Gradshteyn,et al.  Table of Integrals, Series, and Products , 1976 .

[23]  Emmanuel J. Candès,et al.  Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information , 2004, IEEE Transactions on Information Theory.

[24]  Gitta Kutyniok,et al.  1 . 2 Sparsity : A Reasonable Assumption ? , 2012 .

[25]  Sundeep Rangan,et al.  Necessary and Sufficient Conditions for Sparsity Pattern Recovery , 2008, IEEE Transactions on Information Theory.

[26]  Robert D. Nowak,et al.  Universal Measurement Bounds for Structured Sparse Signal Recovery , 2012, AISTATS.

[27]  Emmanuel J. Candès,et al.  Near-Optimal Signal Recovery From Random Projections: Universal Encoding Strategies? , 2004, IEEE Transactions on Information Theory.

[28]  Yonina C. Eldar,et al.  Performance Bounds and Design Criteria for Estimating Finite Rate of Innovation Signals , 2010, IEEE Transactions on Information Theory.

[29]  Yonina C. Eldar,et al.  Near-Oracle Performance of Greedy Block-Sparse Estimation Techniques From Noisy Measurements , 2010, IEEE Journal of Selected Topics in Signal Processing.

[30]  Volkan Cevher,et al.  Model-Based Compressive Sensing , 2008, IEEE Transactions on Information Theory.

[31]  Balas K. Natarajan,et al.  Sparse Approximate Solutions to Linear Systems , 1995, SIAM J. Comput..

[32]  Martin J. Wainwright,et al.  Information-theoretic limits on sparsity recovery in the high-dimensional and noisy setting , 2009, IEEE Trans. Inf. Theory.

[33]  Richard Baraniuk,et al.  Recovery of Clustered Sparse Signals from Compressive Measurements , 2009 .

[34]  Yonina C. Eldar,et al.  Block-Sparse Signals: Uncertainty Relations and Efficient Recovery , 2009, IEEE Transactions on Signal Processing.

[35]  Alan J. Miller Subset Selection in Regression , 1992 .

[36]  Richard G. Baraniuk,et al.  Distributed Compressive Sensing , 2009, ArXiv.

[37]  Yonina C. Eldar,et al.  Robust Recovery of Signals From a Structured Union of Subspaces , 2008, IEEE Transactions on Information Theory.

[38]  Thierry Blu,et al.  Extrapolation and Interpolation) , 2022 .

[39]  Yonina C. Eldar,et al.  Structured Compressed Sensing: From Theory to Applications , 2011, IEEE Transactions on Signal Processing.

[40]  Dmitry M. Malioutov,et al.  A sparse signal reconstruction perspective for source localization with sensor arrays , 2005, IEEE Transactions on Signal Processing.

[41]  I. Miller Probability, Random Variables, and Stochastic Processes , 1966 .

[42]  Mike E. Davies,et al.  Sampling Theorems for Signals From the Union of Finite-Dimensional Linear Subspaces , 2009, IEEE Transactions on Information Theory.

[43]  Guoan Bi,et al.  The Group Lasso for Stable Recovery of Block-Sparse Signal Representations , 2011, IEEE Transactions on Signal Processing.

[44]  Yonina C. Eldar,et al.  Wideband Spectrum Sensing at Sub-Nyquist Rates , 2010, ArXiv.

[45]  Babak Hassibi,et al.  Recovering Sparse Signals Using Sparse Measurement Matrices in Compressed DNA Microarrays , 2008, IEEE Journal of Selected Topics in Signal Processing.

[46]  Yonina C. Eldar,et al.  Wideband Spectrum Sensing at Sub-Nyquist Rates [Applications Corner] , 2010, IEEE Signal Processing Magazine.

[47]  Michael A. Saunders,et al.  Atomic Decomposition by Basis Pursuit , 1998, SIAM J. Sci. Comput..