A greedy algorithm for the analysis transform domain

Many image processing applications benefited remarkably from the theory of sparsity. One model of sparsity is the cosparse analysis one. It was shown that using l_1-minimization one might stably recover a cosparse signal from a small set of random linear measurements if the operator is a frame. Another effort has provided guarantees for dictionaries that have a near optimal projection procedure using greedy-like algorithms. However, no claims have been given for frames. A common drawback of all these existing techniques is their high computational cost for large dimensional problems. In this work we propose a new greedy-like technique with theoretical recovery guarantees for frames as the analysis operator, closing the gap between greedy and relaxation techniques. Our results cover both the case of bounded adversarial noise, where we show that the algorithm provides us with a stable reconstruction, and the one of random Gaussian noise, for which we prove that it has a denoising effect, closing another gap in the analysis framework. Our proposed program, unlike the previous greedy-like ones that solely act in the signal domain, operates mainly in the analysis operator's transform domain. Besides the theoretical benefit, the main advantage of this strategy is its computational efficiency that makes it easily applicable to visually big data. We demonstrate its performance on several high dimensional images.

[1]  Terence Tao,et al.  The Dantzig selector: Statistical estimation when P is much larger than n , 2005, math/0506081.

[2]  Yoram Bresler,et al.  Sparsifying transform learning for Compressed Sensing MRI , 2013, 2013 IEEE 10th International Symposium on Biomedical Imaging.

[3]  Mohamed-Jalal Fadili,et al.  Robust Sparse Analysis Regularization , 2011, IEEE Transactions on Information Theory.

[4]  Tong Zhang,et al.  Sparse Recovery With Orthogonal Matching Pursuit Under RIP , 2010, IEEE Transactions on Information Theory.

[5]  Lie Wang,et al.  New Bounds for Restricted Isometry Constants , 2009, IEEE Transactions on Information Theory.

[6]  Xuelong Li,et al.  Ranking Graph Embedding for Learning to Rerank , 2013, IEEE Transactions on Neural Networks and Learning Systems.

[7]  Raja Giryes,et al.  Sampling in the Analysis Transform Domain , 2014, ArXiv.

[8]  Michael Elad,et al.  Multi-Scale Dictionary Learning Using Wavelets , 2011, IEEE Journal of Selected Topics in Signal Processing.

[9]  Y. C. Pati,et al.  Orthogonal matching pursuit: recursive function approximation with applications to wavelet decomposition , 1993, Proceedings of 27th Asilomar Conference on Signals, Systems and Computers.

[10]  Sheng Chen,et al.  Orthogonal least squares methods and their application to non-linear system identification , 1989 .

[11]  J. Tropp,et al.  CoSaMP: Iterative signal recovery from incomplete and inaccurate samples , 2008, Commun. ACM.

[12]  Emmanuel J. Candès,et al.  Modern statistical estimation via oracle inequalities , 2006, Acta Numerica.

[13]  Yanwei Pang,et al.  Learning Regularized LDA by Clustering , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[14]  Yonina C. Eldar,et al.  Compressed Sensing with Coherent and Redundant Dictionaries , 2010, ArXiv.

[15]  Raja Giryes,et al.  Greedy Signal Space Methods for incoherence and beyond , 2013, ArXiv.

[16]  L. Rudin,et al.  Nonlinear total variation based noise removal algorithms , 1992 .

[17]  Holger Rauhut,et al.  Analysis ℓ1-recovery with Frames and Gaussian Measurements , 2015, ArXiv.

[18]  Michael Elad,et al.  The Cosparse Analysis Model and Algorithms , 2011, ArXiv.

[19]  Michael Elad,et al.  Analysis versus synthesis in signal priors , 2006, 2006 14th European Signal Processing Conference.

[20]  Yoram Bresler,et al.  MR Image Reconstruction From Highly Undersampled k-Space Data by Dictionary Learning , 2011, IEEE Transactions on Medical Imaging.

[21]  Michael Elad,et al.  OMP with Highly Coherent Dictionaries , 2013 .

[22]  Raja Giryes,et al.  Near oracle performance and block analysis of signal space greedy methods , 2014, J. Approx. Theory.

[23]  Emmanuel J. Cand Modern statistical estimation via oracle inequalities , 2006 .

[24]  Michael Elad,et al.  Sparsity-Based Poisson Denoising With Dictionary Learning , 2013, IEEE Transactions on Image Processing.

[25]  Rémi Gribonval,et al.  Projection Onto The k-Cosparse Set is NP-Hard , 2013, ArXiv.

[26]  Deanna Needell,et al.  CoSaMP: Iterative signal recovery from incomplete and inaccurate samples , 2008, ArXiv.

[27]  Michael Elad,et al.  Performance Guarantees of the Thresholding Algorithm for the Cosparse Analysis Model , 2013, IEEE Transactions on Information Theory.

[28]  Emmanuel J. Candès,et al.  Decoding by linear programming , 2005, IEEE Transactions on Information Theory.

[29]  Rebecca Willett,et al.  Poisson Noise Reduction with Non-local PCA , 2012, Journal of Mathematical Imaging and Vision.

[30]  Michael Elad,et al.  Can we allow linear dependencies in the dictionary in the sparse synthesis framework? , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[31]  Deanna Needell,et al.  Stable Image Reconstruction Using Total Variation Minimization , 2012, SIAM J. Imaging Sci..

[32]  Stéphane Mallat,et al.  Matching pursuits with time-frequency dictionaries , 1993, IEEE Trans. Signal Process..

[33]  Michael Elad,et al.  From Sparse Solutions of Systems of Equations to Sparse Modeling of Signals and Images , 2009, SIAM Rev..

[34]  Thomas Strohmer,et al.  Compressed Remote Sensing of Sparse Objects , 2009, SIAM J. Imaging Sci..

[35]  D. Donoho,et al.  Sparse MRI: The application of compressed sensing for rapid MR imaging , 2007, Magnetic resonance in medicine.

[36]  Volkan Cevher,et al.  Recipes on hard thresholding methods , 2011, 2011 4th IEEE International Workshop on Computational Advances in Multi-Sensor Adaptive Processing (CAMSAP).

[37]  Emmanuel J. Candès,et al.  Near-Optimal Signal Recovery From Random Projections: Universal Encoding Strategies? , 2004, IEEE Transactions on Information Theory.

[38]  Rebecca Willett,et al.  This is SPIRAL-TAP: Sparse Poisson Intensity Reconstruction ALgorithms—Theory and Practice , 2010, IEEE Transactions on Image Processing.

[39]  Michael Elad,et al.  On Single Image Scale-Up Using Sparse-Representations , 2010, Curves and Surfaces.

[40]  S. Foucart Sparse Recovery Algorithms: Sufficient Conditions in Terms of RestrictedIsometry Constants , 2012 .

[41]  Yulong Liu,et al.  Compressed Sensing With General Frames via Optimal-Dual-Based $\ell _{1}$-Analysis , 2012, IEEE Transactions on Information Theory.

[42]  Holger Rauhut,et al.  Analysis $\ell_1$-recovery with frames and Gaussian measurements , 2013, 1306.1356.

[43]  Karen O. Egiazarian,et al.  Image denoising with block-matching and 3D filtering , 2006, Electronic Imaging.

[44]  D. Donoho,et al.  Counting faces of randomly-projected polytopes when the projection radically lowers dimension , 2006, math/0607364.

[45]  Michael Elad,et al.  Sparsity Based Methods for Overparametrized Variational Problems , 2014, SIAM J. Imaging Sci..

[46]  Simon Foucart,et al.  Hard Thresholding Pursuit: An Algorithm for Compressive Sensing , 2011, SIAM J. Numer. Anal..

[47]  Xuelong Li,et al.  Robust Tensor Analysis With L1-Norm , 2010, IEEE Transactions on Circuits and Systems for Video Technology.

[48]  Thomas Blumensath,et al.  Sampling and Reconstructing Signals From a Union of Linear Subspaces , 2009, IEEE Transactions on Information Theory.

[49]  P. Bickel,et al.  SIMULTANEOUS ANALYSIS OF LASSO AND DANTZIG SELECTOR , 2008, 0801.1095.

[50]  Deanna Needell,et al.  Signal Space CoSaMP for Sparse Recovery With Redundant Dictionaries , 2012, IEEE Transactions on Information Theory.

[51]  Raja Giryes,et al.  On the Effective Measure of Dimension in the Analysis Cosparse Model , 2014, IEEE Transactions on Information Theory.

[52]  Volkan Cevher,et al.  An ALPS view of sparse recovery , 2011, 2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[53]  Emmanuel J. Candès,et al.  Robust Subspace Clustering , 2013, ArXiv.

[54]  Olgica Milenkovic,et al.  Subspace Pursuit for Compressive Sensing Signal Reconstruction , 2008, IEEE Transactions on Information Theory.

[55]  M. Lustig,et al.  Compressed Sensing MRI , 2008, IEEE Signal Processing Magazine.

[56]  Michael Elad,et al.  Recovery of cosparse signals with Greedy Analysis Pursuit in the presence of noise , 2011, 2011 4th IEEE International Workshop on Computational Advances in Multi-Sensor Adaptive Processing (CAMSAP).

[57]  Michael Elad,et al.  Stable recovery of sparse overcomplete representations in the presence of noise , 2006, IEEE Transactions on Information Theory.

[58]  Yonina C. Eldar,et al.  Coherence-Based Performance Guarantees for Estimating a Sparse Vector Under Random Noise , 2009, IEEE Transactions on Signal Processing.

[59]  Michael Elad,et al.  RIP-Based Near-Oracle Performance Guarantees for SP, CoSaMP, and IHT , 2012, IEEE Transactions on Signal Processing.

[60]  S. Mallat,et al.  Adaptive greedy approximations , 1997 .

[61]  M. Davies,et al.  Greedy-like algorithms for the cosparse analysis model , 2012, 1207.2456.

[62]  Karen O. Egiazarian,et al.  BM3D Frames and Variational Image Deblurring , 2011, IEEE Transactions on Image Processing.

[63]  Mike E. Davies,et al.  Iterative Hard Thresholding for Compressed Sensing , 2008, ArXiv.