Sparse representations & compressed sensing with application to the problem of direction-of-arrival estimation

The significance of sparse representations has been highlighted in numerous signal processing applications ranging from denoising to source separation and the emerging field of compressed sensing has provided new theoretical insights into the problem of inverse systems with sparsity constraints. In this thesis, these advances are exploited in order to tackle the problem of directionof-arrival (DOA) estimation in sensor arrays. Assuming spatial sparsity e.g. few sources impinging on the array, the problem of DOA estimation is formulated as a sparse representation problem in an overcomplete basis. The resulting inverse problem can be solved using typical sparse recovery methods based on convex optimization i.e. `1 minimization. However, in this work a suite of novel sparse recovery algorithms is initially developed, which reduce the computational cost and yield approximate solutions. Moreover, the proposed algorithms of Polytope Faces Pursuits (PFP) allow for the induction of structured sparsity models on the signal of interest, which can be quite beneficial when dealing with multi-channel data acquired by sensor arrays, as it further reduces the complexity and provides performance gain under certain conditions. Regarding the DOA estimation problem, experimental results demonstrate that the proposed methods outperform popular subspace based methods such as the multiple signal classification (MUSIC) algorithm in the case of rank-deficient data (e.g. presence of highly correlated sources or limited amount of data) for both narrowband and wideband sources. In the wideband scenario, they can also suppress the undesirable effects of spatial aliasing. However, DOA estimation with sparsity constraints has its limitations. The compressed sensing requirement of incoherent dictionaries for robust recovery sets limits to the resolution capabilities of the proposed method. On the other hand, the unknown parameters are continuous and therefore if the true DOAs do not belong to the predefined discrete set of potential locations the algorithms’ performance will degrade due to errors caused by mismatches. To overcome this limitation, an iterative alternating descent algorithm for the problem of off-grid DOA estimation is proposed that alternates between sparse recovery and dictionary update estimates. Simulations clearly illustrate the performance gain of the algorithm over the conventional sparsity approach and other existing off-grid DOA estimation algorithms.

[1]  I F Gorodnitsky,et al.  Neuromagnetic source imaging with FOCUSS: a recursive weighted minimum norm algorithm. , 1995, Electroencephalography and clinical neurophysiology.

[2]  Jos F. Sturm,et al.  A Matlab toolbox for optimization over symmetric cones , 1999 .

[3]  Gang Li,et al.  An approach of DOA estimation using noise subspace weighted ℓ1 minimization , 2011, 2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[4]  Volkan Cevher,et al.  Distributed target localization via spatial sparsity , 2008, 2008 16th European Signal Processing Conference.

[5]  Volkan Cevher,et al.  A compressive beamforming method , 2008, 2008 IEEE International Conference on Acoustics, Speech and Signal Processing.

[6]  Yonina C. Eldar,et al.  Robust Recovery of Signals From a Structured Union of Subspaces , 2008, IEEE Transactions on Information Theory.

[7]  Michael Zibulevsky,et al.  Underdetermined blind source separation using sparse representations , 2001, Signal Process..

[8]  Mike E. Davies,et al.  Sampling Theorems for Signals From the Union of Finite-Dimensional Linear Subspaces , 2009, IEEE Transactions on Information Theory.

[9]  Balas K. Natarajan,et al.  Sparse Approximate Solutions to Linear Systems , 1995, SIAM J. Comput..

[10]  T. Blumensath,et al.  Faster & greedier: algorithms for sparse reconstruction of large datasets , 2008, 2008 3rd International Symposium on Communications, Control and Signal Processing.

[11]  Albert Fannjiang,et al.  The MUSIC algorithm for sparse objects: a compressed sensing analysis , 2010, ArXiv.

[12]  Aris Gretsistas,et al.  An alternating descent algorithm for the off-grid DOA estimation problem with sparsity constraints , 2012, 2012 Proceedings of the 20th European Signal Processing Conference (EUSIPCO).

[13]  Mark D. Plumbley,et al.  A source localization approach based on structured sparsity for broadband far-field sources , 2011 .

[14]  R.G. Baraniuk,et al.  Compressive Sensing [Lecture Notes] , 2007, IEEE Signal Processing Magazine.

[15]  I. Bilik Spatial compressive sensing approach for field directionality estimation. , 2009, 2009 IEEE Radar Conference.

[16]  Bob L. Sturm,et al.  Cyclic pure greedy algorithms for recovering compressively sampled sparse signals , 2011, 2011 Conference Record of the Forty Fifth Asilomar Conference on Signals, Systems and Computers (ASILOMAR).

[17]  J. Capon High-resolution frequency-wavenumber spectrum analysis , 1969 .

[18]  Sven Treitel,et al.  Geophysical Signal Analysis , 2000 .

[19]  Emmanuel J. Candès,et al.  A Probabilistic and RIPless Theory of Compressed Sensing , 2010, IEEE Transactions on Information Theory.

[20]  D. Donoho For most large underdetermined systems of linear equations the minimal 𝓁1‐norm solution is also the sparsest solution , 2006 .

[21]  Rémi Gribonval,et al.  Sparse Representations in Audio and Music: From Coding to Source Separation , 2010, Proceedings of the IEEE.

[22]  Yonina C. Eldar,et al.  Compressed Sensing with Coherent and Redundant Dictionaries , 2010, ArXiv.

[23]  Emmanuel J. Candès,et al.  Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information , 2004, IEEE Transactions on Information Theory.

[24]  E. Candès,et al.  Sparsity and incoherence in compressive sampling , 2006, math/0611957.

[25]  Michael Elad,et al.  Sparse and Redundant Representations - From Theory to Applications in Signal and Image Processing , 2010 .

[26]  Gaël Richard,et al.  Union of MDCT Bases for Audio Coding , 2008, IEEE Transactions on Audio, Speech, and Language Processing.

[27]  Francis R. Bach,et al.  Structured sparsity-inducing norms through submodular functions , 2010, NIPS.

[28]  Upamanyu Madhow Fundamentals of Digital Communication: References , 2008 .

[29]  Geert Leus,et al.  Aliasing-Free Wideband Beamforming Using Sparse Signal Representation , 2011, IEEE Transactions on Signal Processing.

[30]  J. Shewchuk An Introduction to the Conjugate Gradient Method Without the Agonizing Pain , 1994 .

[31]  Jong Chul Ye,et al.  Compressive MUSIC: Revisiting the Link Between Compressive Sensing and Array Signal Processing , 2012, IEEE Transactions on Information Theory.

[32]  Shane F. Cotter Multiple snapshot matching pursuit for direction of arrival (DOA) estimation , 2007, 2007 15th European Signal Processing Conference.

[33]  Richard H. Byrd,et al.  A Trust Region Algorithm for Nonlinearly Constrained Optimization , 1987 .

[34]  E.J. Candes,et al.  An Introduction To Compressive Sampling , 2008, IEEE Signal Processing Magazine.

[35]  Aris Gretsistas,et al.  Gradient Polytope Faces Pursuit for large scale sparse recovery problems , 2010, 2010 IEEE International Conference on Acoustics, Speech and Signal Processing.

[36]  Massimo Fornasier,et al.  Compressive Sensing , 2015, Handbook of Mathematical Methods in Imaging.

[37]  L. Carin,et al.  On the Relationship Between Compressive Sensing and Random Sensor Arrays , 2009, IEEE Antennas and Propagation Magazine.

[38]  M. Yuan,et al.  Model selection and estimation in regression with grouped variables , 2006 .

[39]  Jean-Jacques Fuchs,et al.  On sparse representations in arbitrary redundant bases , 2004, IEEE Transactions on Information Theory.

[40]  Mark D. Plumbley Geometry and homotopy for l 1 sparse representations , 2005 .

[41]  Mike E. Davies,et al.  Gradient Pursuits , 2008, IEEE Transactions on Signal Processing.

[42]  Jean-Jacques Fuchs,et al.  On the application of the global matched filter to DOA estimation with uniform circular arrays , 2000, 2000 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings (Cat. No.00CH37100).

[43]  Edwin K. P. Chong,et al.  Sensitivity considerations in compressed sensing , 2011, 2011 Conference Record of the Forty Fifth Asilomar Conference on Signals, Systems and Computers (ASILOMAR).

[44]  Dmitry M. Malioutov,et al.  A sparse signal reconstruction perspective for source localization with sensor arrays , 2005, IEEE Transactions on Signal Processing.

[45]  Townsend Sturm,et al.  Sparse approximation and atomic decomposition: Considering atom interactions in evaluating and building signal representations , 2009 .

[46]  Bhiksha Raj,et al.  Joint sparsity models for wideband array processing , 2011, Optical Engineering + Applications.

[47]  Volkan Cevher,et al.  Recovery of compressible signals in unions of subspaces , 2009, 2009 43rd Annual Conference on Information Sciences and Systems.

[48]  S. Frick,et al.  Compressed Sensing , 2014, Computer Vision, A Reference Guide.

[49]  Jean-Luc Starck,et al.  Sparse Solution of Underdetermined Systems of Linear Equations by Stagewise Orthogonal Matching Pursuit , 2012, IEEE Transactions on Information Theory.

[50]  A. Willsky,et al.  Superresolution source localization through data-adaptive regularization , 2002, Sensor Array and Multichannel Signal Processing Workshop Proceedings, 2002.

[51]  M. H. Wright The interior-point revolution in optimization: History, recent developments, and lasting consequences , 2004 .

[52]  Yoram Bresler,et al.  Subspace Methods for Joint Sparse Recovery , 2010, IEEE Transactions on Information Theory.

[53]  Y. C. Pati,et al.  Orthogonal matching pursuit: recursive function approximation with applications to wavelet decomposition , 1993, Proceedings of 27th Asilomar Conference on Signals, Systems and Computers.

[54]  Krishnaraj M. Varma,et al.  Time-Delay-Estimate Based Direction-of-Arrival Estimation for Speech in Reverberant Environments , 2002 .

[55]  Rémi Gribonval,et al.  Harmonic decomposition of audio signals with matching pursuit , 2003, IEEE Trans. Signal Process..

[56]  M. Viberg,et al.  Two decades of array signal processing research: the parametric approach , 1996, IEEE Signal Process. Mag..

[57]  Richard A. Tapia,et al.  A trust region strategy for nonlinear equality constrained op-timization , 1984 .

[58]  Volkan Cevher,et al.  Model-Based Compressive Sensing , 2008, IEEE Transactions on Information Theory.

[59]  Rémi Gribonval,et al.  A survey of Sparse Component Analysis for blind source separation: principles, perspectives, and new challenges , 2006, ESANN.

[60]  R.G. Baraniuk,et al.  Distributed Compressed Sensing of Jointly Sparse Signals , 2005, Conference Record of the Thirty-Ninth Asilomar Conference onSignals, Systems and Computers, 2005..

[61]  Arkadi Nemirovski,et al.  On sparse representation in pairs of bases , 2003, IEEE Trans. Inf. Theory.

[62]  G. Giannakis,et al.  Sparse regularized total least squares for sensing applications , 2010, 2010 IEEE 11th International Workshop on Signal Processing Advances in Wireless Communications (SPAWC).

[63]  Jean-Jacques Fuchs,et al.  Detection and estimation of superimposed signals , 1998, Proceedings of the 1998 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP '98 (Cat. No.98CH36181).

[64]  Yonina C. Eldar,et al.  Rank Awareness in Joint Sparse Recovery , 2010, IEEE Transactions on Information Theory.

[65]  R. Tibshirani,et al.  Least angle regression , 2004, math/0406456.

[66]  Jie Chen,et al.  Theoretical Results on Sparse Representations of Multiple-Measurement Vectors , 2006, IEEE Transactions on Signal Processing.

[67]  Stéphane Mallat,et al.  A Wavelet Tour of Signal Processing - The Sparse Way, 3rd Edition , 2008 .

[68]  George Tzagkarakis,et al.  Multiple-measurement Bayesian compressed sensing using GSM priors for DOA estimation , 2010, 2010 IEEE International Conference on Acoustics, Speech and Signal Processing.

[69]  M. Kaveh,et al.  Sparse spectral fitting for Direction Of Arrival and power estimation , 2009, 2009 IEEE/SP 15th Workshop on Statistical Signal Processing.

[70]  R. Tibshirani Regression Shrinkage and Selection via the Lasso , 1996 .

[71]  Mark D. Plumbley On Polar Polytopes and the Recovery of Sparse Representations , 2007, IEEE Trans. Inf. Theory.

[72]  Cishen Zhang,et al.  Off-Grid Direction of Arrival Estimation Using Sparse Bayesian Inference , 2011, IEEE Transactions on Signal Processing.

[73]  Marco F. Duarte Localization and bearing estimation via structured sparsity models , 2012, 2012 IEEE Statistical Signal Processing Workshop (SSP).

[74]  Ping Feng,et al.  Spectrum-blind minimum-rate sampling and reconstruction of multiband signals , 1996, 1996 IEEE International Conference on Acoustics, Speech, and Signal Processing Conference Proceedings.

[75]  Michael Elad,et al.  Image Denoising Via Learned Dictionaries and Sparse representation , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[76]  A. Bruckstein,et al.  K-SVD : An Algorithm for Designing of Overcomplete Dictionaries for Sparse Representation , 2005 .

[77]  Alexander Schrijver,et al.  Theory of linear and integer programming , 1986, Wiley-Interscience series in discrete mathematics and optimization.

[78]  A. Robert Calderbank,et al.  Sensitivity to Basis Mismatch in Compressed Sensing , 2011, IEEE Trans. Signal Process..

[79]  Yonina C. Eldar,et al.  Blind Multiband Signal Reconstruction: Compressed Sensing for Analog Signals , 2007, IEEE Transactions on Signal Processing.

[80]  Joel A. Tropp,et al.  Greed is good: algorithmic results for sparse approximation , 2004, IEEE Transactions on Information Theory.

[81]  Yonina C. Eldar,et al.  Average Case Analysis of Multichannel Sparse Recovery Using Convex Relaxation , 2009, IEEE Transactions on Information Theory.

[82]  R. Chellappa,et al.  Compressive wireless arrays for bearing estimation , 2008, 2008 IEEE International Conference on Acoustics, Speech and Signal Processing.

[83]  Yoram Bresler,et al.  Subspace-augmented MUSIC for joint sparse recovery with any rank , 2010, 2010 IEEE Sensor Array and Multichannel Signal Processing Workshop.

[84]  David L. Donoho,et al.  Observed universality of phase transitions in high-dimensional geometry, with implications for modern data analysis and signal processing , 2009, Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[85]  Michael Elad,et al.  Analysis versus synthesis in signal priors , 2006, 2006 14th European Signal Processing Conference.

[86]  Matthew E. P. Davies,et al.  SMALLbox - An Evaluation Framework for Sparse Representations and Dictionary Learning Algorithms , 2010, LVA/ICA.

[87]  Mike E. Davies,et al.  Stagewise Weak Gradient Pursuits , 2009, IEEE Transactions on Signal Processing.

[88]  Michael Elad,et al.  On the Role of Sparse and Redundant Representations in Image Processing , 2010, Proceedings of the IEEE.

[89]  Yonina C. Eldar,et al.  Block sparsity and sampling over a union of subspaces , 2009, 2009 16th International Conference on Digital Signal Processing.

[90]  Yonina C. Eldar,et al.  Structured Compressed Sensing: From Theory to Applications , 2011, IEEE Transactions on Signal Processing.

[91]  Yousef Saad,et al.  Iterative methods for sparse linear systems , 2003 .

[92]  Mark D. Plumbley,et al.  Sparse reconstruction for compressed sensing using Stagewise Polytope Faces Pursuit , 2009, 2009 16th International Conference on Digital Signal Processing.

[93]  M. S. Babtlett Smoothing Periodograms from Time-Series with Continuous Spectra , 1948, Nature.

[94]  Igal Bilik,et al.  Spatial Compressive Sensing for Direction-of-Arrival Estimation of Multiple Sources using Dynamic Sensor Arrays , 2011, IEEE Transactions on Aerospace and Electronic Systems.

[95]  Kim-Chuan Toh,et al.  SDPT3 -- A Matlab Software Package for Semidefinite Programming , 1996 .

[96]  Lawrence Carin,et al.  Exploiting Structure in Wavelet-Based Bayesian Compressive Sensing , 2009, IEEE Transactions on Signal Processing.

[97]  Bhaskar D. Rao,et al.  Sparse signal reconstruction from limited data using FOCUSS: a re-weighted minimum norm algorithm , 1997, IEEE Trans. Signal Process..

[98]  Aris Gretsistas,et al.  A Multichannel Spatial Compressed Sensing Approach for Direction of Arrival Estimation , 2010, LVA/ICA.

[99]  Jean-Yves Audibert Optimization for Machine Learning , 1995 .

[100]  P. Bühlmann,et al.  The group lasso for logistic regression , 2008 .

[101]  T. Blumensath,et al.  Theory and Applications , 2011 .

[102]  Yonina C. Eldar,et al.  Block-Sparse Signals: Uncertainty Relations and Efficient Recovery , 2009, IEEE Transactions on Signal Processing.

[103]  Michael Elad,et al.  Optimally sparse representation in general (nonorthogonal) dictionaries via ℓ1 minimization , 2003, Proceedings of the National Academy of Sciences of the United States of America.

[104]  Georgios B. Giannakis,et al.  Sparsity-Cognizant Total Least-Squares for Perturbed Compressive Sampling , 2010, IEEE Transactions on Signal Processing.

[105]  Nikolaos Mitianoudis,et al.  Using beamforming in the audio source separation problem , 2003, Seventh International Symposium on Signal Processing and Its Applications, 2003. Proceedings..

[106]  Gongguo Tang,et al.  Support recovery for source localization based on overcomplete signal representation , 2010, 2010 IEEE International Conference on Acoustics, Speech and Signal Processing.

[107]  Stéphane Mallat,et al.  Matching pursuits with time-frequency dictionaries , 1993, IEEE Trans. Signal Process..

[108]  Mário A. T. Figueiredo,et al.  Gradient Projection for Sparse Reconstruction: Application to Compressed Sensing and Other Inverse Problems , 2007, IEEE Journal of Selected Topics in Signal Processing.

[109]  Aris Gretsistas,et al.  Group Polytope Faces Pursuit for Recovery of Block-Sparse Signals , 2012, LVA/ICA.

[110]  Michael Zibulevsky,et al.  Signal reconstruction in sensor arrays using sparse representations , 2006, Signal Process..

[111]  Michael Elad,et al.  Image Denoising Via Sparse and Redundant Representations Over Learned Dictionaries , 2006, IEEE Transactions on Image Processing.

[112]  E.J. Candes Compressive Sampling , 2022 .

[113]  Ralph Otto Schmidt,et al.  A signal subspace approach to multiple emitter location and spectral estimation , 1981 .

[114]  Rémi Gribonval,et al.  Sparse representations in unions of bases , 2003, IEEE Trans. Inf. Theory.

[115]  Babak Hassibi,et al.  On the Reconstruction of Block-Sparse Signals With an Optimal Number of Measurements , 2008, IEEE Transactions on Signal Processing.

[116]  Takuya Yoshioka,et al.  Blind Separation and Dereverberation of Speech Mixtures by Joint Optimization , 2011, IEEE Transactions on Audio, Speech, and Language Processing.

[117]  Mark D. Plumbley,et al.  Sparse representations of polyphonic music , 2006, Signal Process..

[118]  Michael A. Saunders,et al.  Atomic Decomposition by Basis Pursuit , 1998, SIAM J. Sci. Comput..

[119]  H. Rauhut,et al.  Atoms of All Channels, Unite! Average Case Analysis of Multi-Channel Sparse Recovery Using Greedy Algorithms , 2008 .

[120]  Richard M. Leahy,et al.  MEG-based imaging of focal neuronal current sources , 1996, IEEE Transactions on Medical Imaging.

[121]  Michael Elad,et al.  A generalized uncertainty principle and sparse representation in pairs of bases , 2002, IEEE Trans. Inf. Theory.

[122]  Rodney A. Kennedy,et al.  Effects of basis-mismatch in compressive sampling of continuous sinusoidal signals , 2010, 2010 2nd International Conference on Future Computer and Communication.

[123]  Xin Wang,et al.  Support recovery in compressive sensing for estimation of direction-of-arrival , 2011, 2011 Conference Record of the Forty Fifth Asilomar Conference on Signals, Systems and Computers (ASILOMAR).

[124]  M. Skolnik,et al.  Introduction to Radar Systems , 2021, Advances in Adaptive Radar Detection and Range Estimation.

[125]  Rayan Saab,et al.  Sparco: A Testing Framework for Sparse Reconstruction , 2007 .

[126]  Mark D. Plumbley Recovery of Sparse Representations by Polytope Faces Pursuit , 2006, ICA.

[127]  D. Donoho,et al.  Fast Solution of -Norm Minimization Problems When the Solution May Be Sparse , 2008 .

[128]  Emmanuel J. Candès,et al.  Decoding by linear programming , 2005, IEEE Transactions on Information Theory.

[129]  E. Candès,et al.  Stable signal recovery from incomplete and inaccurate measurements , 2005, math/0503066.

[130]  V.K. Goyal,et al.  Compressive Sampling and Lossy Compression , 2008, IEEE Signal Processing Magazine.

[131]  Mark D. Plumbley,et al.  Unsupervised analysis of polyphonic music by sparse coding , 2006, IEEE Transactions on Neural Networks.

[132]  Aboulnasr Hassanien Advanced array processing in the presence of complicated spatio-temporal sources , 2005 .

[133]  Eero P. Simoncelli,et al.  Recovery of Sparse Translation-Invariant Signals With Continuous Basis Pursuit , 2011, IEEE Transactions on Signal Processing.

[134]  M. Rudelson,et al.  Sparse reconstruction by convex relaxation: Fourier and Gaussian measurements , 2006, 2006 40th Annual Conference on Information Sciences and Systems.

[135]  Xiaoming Huo,et al.  Uncertainty principles and ideal atomic decomposition , 2001, IEEE Trans. Inf. Theory.