Robust methods for sensing and reconstructing sparse signals

Compressed sensing (CS) is an emerging signal acquisition framework that goes against the traditional Nyquist sampling paradigm. CS demonstrates that a sparse, or compressible, signal can be acquired using a low rate acquisition process. Since noise is always present in practical data acquisition systems, sensing and reconstruction methods are developed assuming a Gaussian (light-tailed) model for the corrupting noise. However, when the underlying signal and/or the measurements are corrupted by impulsive noise, commonly employed linear sampling operators, coupled with Gaussian-derived reconstruction algorithms, fail to recover a close approximation of the signal. This dissertation develops robust sampling and reconstruction methods for sparse signals in the presence of impulsive noise. To achieve this objective, we make use of robust statistics theory to develop appropriate methods addressing the problem of impulsive noise in CS systems. We develop a generalized Cauchy distribution (GCD) based theoretical approach that allows challenging problems to be formulated in a robust fashion. Robust sampling operators, together with robust reconstruction strategies are developed using the introduced GCD framework. To solve the problem of impulsive noise embedded in the underlying signal prior the measurement process, we propose a robust nonlinear measurement operator based on the weighed myriad estimator. To recover sparse signals from impulsive noise introduced in the measurement process, a geometric optimization problem based on L1 minimization employing a Lorentzian norm constraint on the

[1]  B. Ripley,et al.  Robust Statistics , 2018, Encyclopedia of Mathematical Geosciences.

[2]  Robert D. Nowak,et al.  Signal Reconstruction From Noisy Random Projections , 2006, IEEE Transactions on Information Theory.

[3]  Mike E. Davies,et al.  Normalized Iterative Hard Thresholding: Guaranteed Stability and Performance , 2010, IEEE Journal of Selected Topics in Signal Processing.

[4]  Robert D. Nowak,et al.  Finding needles in noisy haystacks , 2008, 2008 IEEE International Conference on Acoustics, Speech and Signal Processing.

[5]  Emmanuel J. Candès,et al.  Highly Robust Error Correction byConvex Programming , 2006, IEEE Transactions on Information Theory.

[6]  David L. Donoho,et al.  High-Dimensional Centrally Symmetric Polytopes with Neighborliness Proportional to Dimension , 2006, Discret. Comput. Geom..

[7]  Thierry Blu,et al.  Sampling signals with finite rate of innovation , 2002, IEEE Trans. Signal Process..

[8]  Gonzalo R. Arce,et al.  Optimality of the myriad filter in practical impulsive-noise environments , 2001, IEEE Trans. Signal Process..

[9]  Rayan Saab,et al.  Sparse Recovery by Non-convex Optimization -- Instance Optimality , 2008, ArXiv.

[10]  Michael Elad,et al.  Why Simple Shrinkage Is Still Relevant for Redundant Representations? , 2006, IEEE Transactions on Information Theory.

[11]  Gonzalo R. Arce,et al.  Robust techniques for wireless communications in non-gaussian environments , 1997 .

[12]  Richard G. Baraniuk,et al.  Sparse Signal Reconstruction from Noisy Compressive Measurements using Cross Validation , 2007, 2007 IEEE/SP 14th Workshop on Statistical Signal Processing.

[13]  Anna C. Gilbert,et al.  Improved time bounds for near-optimal sparse Fourier representations , 2005, SPIE Optics + Photonics.

[14]  Terence Tao,et al.  The Dantzig selector: Statistical estimation when P is much larger than n , 2005, math/0506081.

[15]  Ingrid Daubechies,et al.  Ten Lectures on Wavelets , 1992 .

[16]  Michael Elad,et al.  A wide-angle view at iterated shrinkage algorithms , 2007, SPIE Optical Engineering + Applications.

[17]  E. Gunawan,et al.  Performance analysis of OFDM systems for broadband power line communications under impulsive noise and multipath effects , 2005, IEEE Transactions on Power Delivery.

[18]  Kenneth E. Barner,et al.  Polynomial weighted median filtering , 2005, IEEE Transactions on Signal Processing.

[19]  John Wright,et al.  Dense Error Correction via L1-Minimization , 2008, 0809.0199.

[20]  Y. C. Pati,et al.  Orthogonal matching pursuit: recursive function approximation with applications to wavelet decomposition , 1993, Proceedings of 27th Asilomar Conference on Signals, Systems and Computers.

[21]  Sheng Chen,et al.  Orthogonal least squares methods and their application to non-linear system identification , 1989 .

[22]  Miin-Shen Yang,et al.  A similarity-based robust clustering method , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[23]  Volkan Cevher,et al.  Model-Based Compressive Sensing , 2008, IEEE Transactions on Information Theory.

[24]  Kenneth E. Barner,et al.  Robust sampling and reconstruction methods for compressed sensing , 2009, 2009 IEEE International Conference on Acoustics, Speech and Signal Processing.

[25]  Bin Dong,et al.  Randomized Iterative Hard Thresholding for Sparse Approximations , 2014, 2014 Data Compression Conference.

[26]  Kenneth E. Barner,et al.  Orthogonal Matching Pursuit based recovery for correlated sources with partially disjoint supports , 2010, 2010 44th Annual Conference on Information Sciences and Systems (CISS).

[27]  Kenneth E. Barner,et al.  Meridian Filtering for Robust Signal Processing , 2007, IEEE Transactions on Signal Processing.

[28]  P. R. Rider,et al.  Generalized cauchy distributions , 1957 .

[29]  Kenneth E. Barner,et al.  Robust Sampling and Reconstruction Methods for Sparse Signals in the Presence of Impulsive Noise , 2010, IEEE Journal of Selected Topics in Signal Processing.

[30]  G.R. Arce,et al.  Zero-Order Statistics: A Mathematical Framework for the Processing and Characterization of Very Impulsive Signals , 2006, IEEE Transactions on Signal Processing.

[31]  Wei Lu,et al.  Modified-CS: Modifying compressive sensing for problems with partially known support , 2009, 2009 IEEE International Symposium on Information Theory.

[32]  R. DeVore,et al.  Compressed sensing and best k-term approximation , 2008 .

[33]  Kenneth E. Barner,et al.  Generalized cauchy distribution based robust estimation , 2008, 2008 IEEE International Conference on Acoustics, Speech and Signal Processing.

[34]  Kenneth E. Barner,et al.  Lorentzian Iterative Hard Thresholding: Robust Compressed Sensing With Prior Information , 2013, IEEE Transactions on Signal Processing.

[35]  Vladimir N. Temlyakov,et al.  Nonlinear Methods of Approximation , 2003, Found. Comput. Math..

[36]  Ronald A. DeVore,et al.  Deterministic constructions of compressed sensing matrices , 2007, J. Complex..

[37]  E.J. Candes Compressive Sampling , 2022 .

[38]  Manuel Blanco-Velasco,et al.  Compressed sensing based method for ECG compression , 2011, 2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[39]  J. L. Nolan Stable Distributions. Models for Heavy Tailed Data , 2001 .

[40]  Fionn Murtagh,et al.  Fast communication , 2002 .

[41]  Emmanuel J. Candès,et al.  Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information , 2004, IEEE Transactions on Information Theory.

[42]  E. Candès,et al.  Sparsity and incoherence in compressive sampling , 2006, math/0611957.

[43]  Yaakov Tsaig,et al.  Extensions of compressed sensing , 2006, Signal Process..

[44]  David L. Donoho,et al.  Precise Undersampling Theorems , 2010, Proceedings of the IEEE.

[45]  Galen Reeves,et al.  Sampling bounds for sparse support recovery in the presence of noise , 2008, 2008 IEEE International Symposium on Information Theory.

[46]  Aggelos K. Katsaggelos,et al.  Non-convex priors in Bayesian compressed sensing , 2009, 2009 17th European Signal Processing Conference.

[47]  D. Donoho For most large underdetermined systems of linear equations the minimal 𝓁1‐norm solution is also the sparsest solution , 2006 .

[48]  Kenneth E. Barner,et al.  Lorentzian based iterative hard thresholding for compressed sensing , 2011, 2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[49]  J. Besag On the Statistical Analysis of Dirty Pictures , 1986 .

[50]  R. Tibshirani Regression Shrinkage and Selection via the Lasso , 1996 .

[51]  Kenneth E. Barner,et al.  A theoretical framework for problems requiring robust behavior , 2009, 2009 3rd IEEE International Workshop on Computational Advances in Multi-Sensor Adaptive Processing (CAMSAP).

[52]  Michael Elad,et al.  Stable recovery of sparse overcomplete representations in the presence of noise , 2006, IEEE Transactions on Information Theory.

[53]  Zhifeng Zhang,et al.  Adaptive time-frequency decompositions , 1994 .

[54]  Richard G. Baraniuk,et al.  Exact signal recovery from sparsely corrupted measurements through the Pursuit of Justice , 2009, 2009 Conference Record of the Forty-Third Asilomar Conference on Signals, Systems and Computers.

[55]  David L. Donoho,et al.  De-noising by soft-thresholding , 1995, IEEE Trans. Inf. Theory.

[56]  Volkan Cevher,et al.  Low-Dimensional Models for Dimensionality Reduction and Signal Recovery: A Geometric Perspective , 2010, Proceedings of the IEEE.

[57]  Javier Garcia-Frías,et al.  Exploiting Prior Knowledge in The Recovery of Signals from Noisy Random Projections , 2007, 2007 Data Compression Conference (DCC'07).

[58]  D. Ruppert Robust Statistics: The Approach Based on Influence Functions , 1987 .

[59]  Kenneth E. Barner,et al.  A Generalized Cauchy Distribution Framework for Problems Requiring Robust Behavior , 2010, EURASIP J. Adv. Signal Process..

[60]  Shie Qian,et al.  Signal representation using adaptive normalized Gaussian functions , 1994, Signal Process..

[61]  Yonina C. Eldar,et al.  Robust Recovery of Signals From a Structured Union of Subspaces , 2008, IEEE Transactions on Information Theory.

[62]  Joel A. Tropp,et al.  Sparse Approximation Via Iterative Thresholding , 2006, 2006 IEEE International Conference on Acoustics Speech and Signal Processing Proceedings.

[63]  Moncef Gabbouj,et al.  Weighted median filters: a tutorial , 1996 .

[64]  Bhaskar D. Rao,et al.  Variational EM Algorithms for Non-Gaussian Latent Variable Models , 2005, NIPS.

[65]  Michael Elad,et al.  Applications of Sparse Representation and Compressive Sensing , 2010, Proc. IEEE.

[66]  Stephen J. Wright,et al.  Toeplitz-Structured Compressed Sensing Matrices , 2007, 2007 IEEE/SP 14th Workshop on Statistical Signal Processing.

[67]  David P. Wipf,et al.  Iterative Reweighted 1 and 2 Methods for Finding Sparse Solutions , 2010, IEEE J. Sel. Top. Signal Process..

[68]  Patrick L. Combettes,et al.  Signal Recovery by Proximal Forward-Backward Splitting , 2005, Multiscale Model. Simul..

[69]  Emmanuel J. Cand The Restricted Isometry Property and Its Implications for Compressed Sensing , 2008 .

[70]  R. DeVore,et al.  A Simple Proof of the Restricted Isometry Property for Random Matrices , 2008 .

[71]  Martin Vetterli,et al.  Data Compression and Harmonic Analysis , 1998, IEEE Trans. Inf. Theory.

[72]  J. Astola,et al.  Optimal median type filters for exponential noise distributions , 1989 .

[73]  Michael A. Saunders,et al.  Atomic Decomposition by Basis Pursuit , 1998, SIAM J. Sci. Comput..

[74]  Gonzalo R. Arce,et al.  Statistically-Efficient Filtering in Impulsive Environments: Weighted Myriad Filters , 2002, EURASIP J. Adv. Signal Process..

[75]  Kenneth E. Barner,et al.  LLp metric based robust clustering , 2009, 2009 43rd Annual Conference on Information Sciences and Systems.

[76]  Joel A. Tropp,et al.  Signal Recovery From Random Measurements Via Orthogonal Matching Pursuit , 2007, IEEE Transactions on Information Theory.

[77]  Minh N. Do,et al.  Signal reconstruction using sparse tree representations , 2005, SPIE Optics + Photonics.

[78]  M. Wakin Manifold-Based Signal Recovery and Parameter Estimation from Compressive Measurements , 2010, 1002.1247.

[79]  Michael Gastpar,et al.  Differences Between Observation and Sampling Error in Sparse Signal Reconstruction , 2007, 2007 IEEE/SP 14th Workshop on Statistical Signal Processing.

[80]  Richard G. Baraniuk,et al.  Fast reconstruction of piecewise smooth signals from random projections , 2005 .

[81]  Bhaskar D. Rao,et al.  Sparse signal reconstruction from limited data using FOCUSS: a re-weighted minimum norm algorithm , 1997, IEEE Trans. Signal Process..

[82]  K.E. Barner,et al.  Hybrid Polynomial Filters for Gaussian and Non-Gaussian Noise Environments , 2006, IEEE Transactions on Signal Processing.

[83]  Z. D. Bai,et al.  On the maximum‐likelihood estimator for the location parameter of a cauchy distribution , 1987 .

[84]  Stéphane Mallat,et al.  Matching pursuits with time-frequency dictionaries , 1993, IEEE Trans. Signal Process..

[85]  R. Nowak,et al.  Fast wavelet-based image deconvolution using the EM algorithm , 2001, Conference Record of Thirty-Fifth Asilomar Conference on Signals, Systems and Computers (Cat.No.01CH37256).

[86]  Kenneth E. Barner,et al.  Iterative hard thresholding for compressed sensing with partially known support , 2011, 2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[87]  Mário A. T. Figueiredo,et al.  Gradient Projection for Sparse Reconstruction: Application to Compressed Sensing and Other Inverse Problems , 2007, IEEE Journal of Selected Topics in Signal Processing.

[88]  Antonin Chambolle,et al.  A l1-Unified Variational Framework for Image Restoration , 2004, ECCV.

[89]  Harry M. Hall A NEW MODEL FOR 'IMPULSIVE' PHENOMENA: APPLICATION TO ATMOSPHERIC-NOISE COMMUNICATION CHANNELS , 1966 .

[90]  Xiaoming Huo,et al.  Uncertainty principles and ideal atomic decomposition , 2001, IEEE Trans. Inf. Theory.

[91]  Deanna Needell,et al.  CoSaMP: Iterative signal recovery from incomplete and inaccurate samples , 2008, ArXiv.

[92]  Emmanuel J. Candès,et al.  Decoding by linear programming , 2005, IEEE Transactions on Information Theory.

[93]  E. Candès,et al.  Stable signal recovery from incomplete and inaccurate measurements , 2005, math/0503066.

[94]  G. Weiss,et al.  A First Course on Wavelets , 1996 .

[95]  Gonzalo R. Arce,et al.  Reconstruction of sparse signals from l1 dimensionality-reduced Cauchy random-projections , 2010, ICASSP.

[96]  Emmanuel J. Candès,et al.  Near-Optimal Signal Recovery From Random Projections: Universal Encoding Strategies? , 2004, IEEE Transactions on Information Theory.

[97]  Piotr Indyk,et al.  Combining geometry and combinatorics: A unified approach to sparse signal recovery , 2008, 2008 46th Annual Allerton Conference on Communication, Control, and Computing.

[98]  S. Mendelson,et al.  Uniform Uncertainty Principle for Bernoulli and Subgaussian Ensembles , 2006, math/0608665.

[99]  R. Chartrand,et al.  Restricted isometry properties and nonconvex compressive sensing , 2007 .

[100]  Justin K. Romberg,et al.  Compressive Sensing by Random Convolution , 2009, SIAM J. Imaging Sci..

[101]  Aggelos K. Katsaggelos,et al.  Fast bayesian compressive sensing using Laplace priors , 2009, 2009 IEEE International Conference on Acoustics, Speech and Signal Processing.

[102]  Laurent Jacques,et al.  A short note on compressed sensing with partially known signal support , 2009, Signal Process..

[103]  C. L. Nikias,et al.  Signal processing with fractional lower order moments: stable processes and their applications , 1993, Proc. IEEE.

[104]  Tuncer Can Aysal Filtering and estimation theory: First-order, polynomial and decentralized signal processing , 2007 .

[105]  Rajesh N. Davé,et al.  Characterization and detection of noise in clustering , 1991, Pattern Recognit. Lett..

[106]  Joel A. Tropp,et al.  Greed is good: algorithmic results for sparse approximation , 2004, IEEE Transactions on Information Theory.

[107]  G. McLachlan,et al.  The EM algorithm and extensions , 1996 .

[108]  John B. Thomas,et al.  Detectors for discrete-time signals in non-Gaussian noise , 1972, IEEE Trans. Inf. Theory.

[109]  Vahid Tarokh,et al.  A Frame Construction and a Universal Distortion Bound for Sparse Representations , 2008, IEEE Transactions on Signal Processing.

[110]  Yonina C. Eldar,et al.  Introduction to the Issue on Compressive Sensing , 2010, IEEE J. Sel. Top. Signal Process..

[111]  Ercan E. Kuruoglu Signal processing with heavy-tailed distributions , 2002, Signal Process..

[112]  David Middleton,et al.  Statistical-Physical Models of Electromagnetic Interference , 1977, IEEE Transactions on Electromagnetic Compatibility.

[113]  R. Tibshirani,et al.  Least angle regression , 2004, math/0406456.

[114]  I. Daubechies,et al.  An iterative thresholding algorithm for linear inverse problems with a sparsity constraint , 2003, math/0307152.

[115]  Richard G. Baraniuk,et al.  Wavelet-domain compressive signal reconstruction using a Hidden Markov Tree model , 2008, 2008 IEEE International Conference on Acoustics, Speech and Signal Processing.

[116]  Richard G. Baraniuk,et al.  Theory and Implementation of an Analog-to-Information Converter using Random Demodulation , 2007, 2007 IEEE International Symposium on Circuits and Systems.

[117]  Mike E. Davies,et al.  Iterative Hard Thresholding for Compressed Sensing , 2008, ArXiv.

[118]  Gonzalo R. Arce,et al.  Compressive Sensing Signal Reconstruction by Weighted Median Regression Estimates , 2011, IEEE Transactions on Signal Processing.

[119]  Kenneth E. Barner,et al.  Iterative algorithms for compressed sensing with partially known support , 2010, 2010 IEEE International Conference on Acoustics, Speech and Signal Processing.

[120]  D. Donoho,et al.  Neighborliness of randomly projected simplices in high dimensions. , 2005, Proceedings of the National Academy of Sciences of the United States of America.

[121]  Olgica Milenkovic,et al.  Weighted Superimposed Codes and Constrained Integer Compressed Sensing , 2008, IEEE Transactions on Information Theory.

[122]  D K Smith,et al.  Numerical Optimization , 2001, J. Oper. Res. Soc..

[123]  Robert D. Nowak,et al.  An EM algorithm for wavelet-based image restoration , 2003, IEEE Trans. Image Process..

[124]  Manuel Blanco-Velasco,et al.  Embedded filter bank-based algorithm for ECG compression , 2008, Signal Process..

[125]  Yonina C. Eldar Compressed Sensing of Analog Signals , 2008, ArXiv.

[126]  Rajesh N. Davé,et al.  Robust clustering methods: a unified view , 1997, IEEE Trans. Fuzzy Syst..

[127]  Kenneth E. Barner,et al.  Myriad-Type Polynomial Filtering , 2007, IEEE Transactions on Signal Processing.

[128]  Abdelhak M. Zoubir,et al.  The stability test for symmetric alpha-stable distributions , 2005, IEEE Transactions on Signal Processing.

[129]  T. Blumensath Compressed Sensing with Nonlinear Observations , 2010 .

[130]  S. Kirolos,et al.  DepartmentofElectricalandComputerEngineering,RiceUniversity,Houston,TX DepartmentsofMathematicsandEECS,UniversityofMichigan,AnnArbor,MI , 2022 .

[131]  Christian Jutten,et al.  A Fast Approach for Overcomplete Sparse Decomposition Based on Smoothed $\ell ^{0}$ Norm , 2008, IEEE Transactions on Signal Processing.

[132]  Gonzalo R. Arce,et al.  A general weighted median filter structure admitting negative weights , 1998, IEEE Trans. Signal Process..

[133]  S. Setzer,et al.  Signal recovery from incomplete measurements in the presence of outliers , 2007 .

[134]  Kenneth E. Barner,et al.  Bayesian compressed sensing using generalized Cauchy priors , 2010, 2010 IEEE International Conference on Acoustics, Speech and Signal Processing.

[135]  Aggelos K. Katsaggelos,et al.  Bayesian Compressive Sensing Using Laplace Priors , 2010, IEEE Transactions on Image Processing.

[136]  Stephen J. Wright,et al.  Computational Methods for Sparse Solution of Linear Inverse Problems , 2010, Proceedings of the IEEE.

[137]  Kenneth E. Barner,et al.  Constrained Decentralized Estimation Over Noisy Channels for Sensor Networks , 2008, IEEE Transactions on Signal Processing.

[138]  E.J. Candes,et al.  An Introduction To Compressive Sampling , 2008, IEEE Signal Processing Magazine.

[139]  George Eastman House,et al.  Sparse Bayesian Learning and the Relevan e Ve tor Ma hine , 2001 .

[140]  Volkan Cevher,et al.  Recovery of compressible signals in unions of subspaces , 2009, 2009 43rd Annual Conference on Information Sciences and Systems.

[141]  V. Zolotarev One-dimensional stable distributions , 1986 .

[142]  Volkan Cevher,et al.  Sparse Signal Recovery Using Markov Random Fields , 2008, NIPS.

[143]  Richard G. Baraniuk,et al.  Random Projections of Smooth Manifolds , 2009, Found. Comput. Math..

[144]  Kenneth E. Barner,et al.  Iteratively re-weighted least squares for sparse signal reconstruction from noisy measurements , 2009, 2009 43rd Annual Conference on Information Sciences and Systems.

[145]  Kenneth E. Barner,et al.  Blind decentralized estimation for bandwidth constrained wireless sensor networks , 2008, IEEE Transactions on Wireless Communications.

[146]  Martin J. Wainwright,et al.  High-dimensional subset recovery in noise: Sparse measurements and statistical efficiency , 2008, 2008 IEEE International Symposium on Information Theory.

[147]  G. Pólya,et al.  Inequalities (Cambridge Mathematical Library) , 1934 .

[148]  Ping Feng,et al.  Spectrum-blind minimum-rate sampling and reconstruction of multiband signals , 1996, 1996 IEEE International Conference on Acoustics, Speech, and Signal Processing Conference Proceedings.

[149]  D. Donoho,et al.  Counting faces of randomly-projected polytopes when the projection radically lowers dimension , 2006, math/0607364.