Improving A⋆ OMP: Theoretical and empirical analyses with a novel dynamic cost model

Best-first search has been recently utilized for compressed sensing (CS) by the A ? orthogonal matching pursuit ( A ? OMP ) algorithm. In this work, we concentrate on theoretical and empirical analyses of A ? OMP . We present a restricted isometry property (RIP) based general condition for exact recovery of sparse signals via A ? OMP . In addition, we develop online guarantees which promise improved recovery performance with the residue-based termination instead of the sparsity-based one. We demonstrate the recovery capabilities of A ? OMP with extensive recovery simulations using the adaptive-multiplicative (AMul) cost model, which effectively compensates for the path length differences in the search tree. The presented results, involving phase transitions for different nonzero element distributions as well as recovery rates and average error, reveal not only the superior recovery accuracy of A ? OMP , but also the improvements with the residue-based termination and the AMul cost model. Comparison of the run times indicates the speed up by the AMul cost model. We also demonstrate a hybrid of OMP and A ? OMP to accelerate the search further. Finally, we run A ? OMP on sparse images to illustrate its recovery performance for more realistic coefficient distributions. HighlightsWe analyze the empirical and theoretical performance of A ? OMP in sparse recovery.RIP-based theoretical analysis provides recovery guarantees for A ? OMP .The introduced novel dynamic cost model improves recovery performance significantly.Results of computationally expensive experiments are presented by phase transitions. A ? OMP outperforms conventional methods in a wide range of sparse recovery scenarios.

[1]  Y. C. Pati,et al.  Orthogonal matching pursuit: recursive function approximation with applications to wavelet decomposition , 1993, Proceedings of 27th Asilomar Conference on Signals, Systems and Computers.

[2]  E. Candès The restricted isometry property and its implications for compressed sensing , 2008 .

[3]  Olgica Milenkovic,et al.  Subspace Pursuit for Compressive Sensing Signal Reconstruction , 2008, IEEE Transactions on Information Theory.

[4]  Jian Wang,et al.  On the Recovery Limit of Sparse Signals Using Orthogonal Matching Pursuit , 2012, IEEE Transactions on Signal Processing.

[5]  Tong Zhang,et al.  Sparse Recovery With Orthogonal Matching Pursuit Under RIP , 2010, IEEE Transactions on Information Theory.

[6]  Michael A. Saunders,et al.  Atomic Decomposition by Basis Pursuit , 1998, SIAM J. Sci. Comput..

[7]  Deanna Needell,et al.  CoSaMP: Iterative signal recovery from incomplete and inaccurate samples , 2008, ArXiv.

[8]  Emmanuel J. Candès,et al.  Decoding by linear programming , 2005, IEEE Transactions on Information Theory.

[9]  Emmanuel J. Candès,et al.  Near-Optimal Signal Recovery From Random Projections: Universal Encoding Strategies? , 2004, IEEE Transactions on Information Theory.

[10]  Frederick Jelinek,et al.  Statistical methods for speech recognition , 1997 .

[11]  Mohammad Reza Mohammadi,et al.  Non-negative sparse decomposition based on constrained smoothed ℓ0 norm , 2014, Signal Process..

[12]  YinWotao,et al.  Sparse Signal Reconstruction via Iterative Support Detection , 2010 .

[13]  Hakan Erdogan,et al.  A comparison of termination criteria for A∗OMP , 2012, 2012 Proceedings of the 20th European Signal Processing Conference (EUSIPCO).

[14]  Christian Jutten,et al.  A Fast Approach for Overcomplete Sparse Decomposition Based on Smoothed $\ell ^{0}$ Norm , 2008, IEEE Transactions on Signal Processing.

[15]  Nazim Burak Karahanoglu,et al.  Search-based methods for the sparse signal recovery problem in compressed , 2013 .

[16]  Yi Shen,et al.  A Remark on the Restricted Isometry Property in Orthogonal Matching Pursuit , 2012, IEEE Transactions on Information Theory.

[17]  Bhaskar D. Rao,et al.  Application of tree-based searches to matching pursuit , 2001, 2001 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings (Cat. No.01CH37221).

[18]  Stéphane Mallat,et al.  Matching pursuits with time-frequency dictionaries , 1993, IEEE Trans. Signal Process..

[19]  Emmanuel J. Candès,et al.  Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information , 2004, IEEE Transactions on Information Theory.

[20]  Damiana Lazzaro,et al.  A fast algorithm for nonconvex approaches to sparse recovery problems , 2013, Signal Process..

[21]  Hakan Erdogan,et al.  Compressed sensing signal recovery via forward-backward pursuit , 2012, Digit. Signal Process..

[22]  P. Laguna,et al.  Signal Processing , 2002, Yearbook of Medical Informatics.

[23]  Mikael Skoglund,et al.  Distributed greedy pursuit algorithms , 2013, Signal Process..

[24]  Joel A. Tropp,et al.  Signal Recovery From Random Measurements Via Orthogonal Matching Pursuit , 2007, IEEE Transactions on Information Theory.

[25]  Jun Fang,et al.  Sparse signal recovery from one-bit quantized data: An iterative reweighted algorithm , 2014, Signal Process..

[26]  Zongben Xu,et al.  Restricted p-isometry properties of nonconvex block-sparse compressed sensing , 2014, Signal Process..

[27]  J. Tropp,et al.  CoSaMP: Iterative signal recovery from incomplete and inaccurate samples , 2008, Commun. ACM.

[28]  Mike E. Davies,et al.  Gradient Pursuits , 2008, IEEE Transactions on Signal Processing.

[29]  Jian Wang,et al.  Multipath Matching Pursuit , 2013, IEEE Transactions on Information Theory.

[30]  Wotao Yin,et al.  Sparse Signal Reconstruction via Iterative Support Detection , 2009, SIAM J. Imaging Sci..

[31]  I. Daubechies,et al.  Iteratively reweighted least squares minimization for sparse recovery , 2008, 0807.0575.

[32]  Mike E. Davies,et al.  Iterative Hard Thresholding for Compressed Sensing , 2008, ArXiv.

[33]  Rick Chartrand,et al.  Exact Reconstruction of Sparse Signals via Nonconvex Minimization , 2007, IEEE Signal Processing Letters.

[34]  Jian Wang,et al.  Improved Recovery Bounds of Orthogonal Matching Pursuit using Restricted Isometry Property , 2012, ArXiv.

[35]  KarahanogluNazim Burak,et al.  A* orthogonal matching pursuit , 2012 .

[36]  Michael B. Wakin,et al.  Analysis of Orthogonal Matching Pursuit Using the Restricted Isometry Property , 2009, IEEE Transactions on Information Theory.

[37]  J. CandesE.,et al.  Near-Optimal Signal Recovery From Random Projections , 2006 .

[38]  Deanna Needell,et al.  Signal Recovery From Incomplete and Inaccurate Measurements Via Regularized Orthogonal Matching Pursuit , 2007, IEEE Journal of Selected Topics in Signal Processing.

[39]  J. Tropp Algorithms for simultaneous sparse approximation. Part II: Convex relaxation , 2006, Signal Process..

[40]  Simon Foucart,et al.  Stability and Robustness of Weak Orthogonal Matching Pursuits , 2012 .

[41]  Yun Tian,et al.  Performance analysis of partial segmented compressed sampling , 2013, Signal Process..

[42]  Joel A. Tropp,et al.  Algorithms for simultaneous sparse approximation. Part I: Greedy pursuit , 2006, Signal Process..

[43]  Taner Ince,et al.  Nonconvex compressed sensing with partially known signal support , 2013, Signal Process..

[44]  Lawrence Carin,et al.  Bayesian Compressive Sensing , 2008, IEEE Transactions on Signal Processing.

[45]  Joel A. Tropp,et al.  Greed is good: algorithmic results for sparse approximation , 2004, IEEE Transactions on Information Theory.

[46]  Stephen P. Boyd,et al.  Enhancing Sparsity by Reweighted ℓ1 Minimization , 2007, 0711.1612.

[47]  S. Frick,et al.  Compressed Sensing , 2014, Computer Vision, A Reference Guide.

[48]  M. Rudelson,et al.  Sparse reconstruction by convex relaxation: Fourier and Gaussian measurements , 2006, 2006 40th Annual Conference on Information Sciences and Systems.

[49]  Arian Maleki,et al.  Optimally Tuned Iterative Reconstruction Algorithms for Compressed Sensing , 2009, IEEE Journal of Selected Topics in Signal Processing.

[50]  Joel A. Tropp,et al.  ALGORITHMS FOR SIMULTANEOUS SPARSE APPROXIMATION , 2006 .

[51]  P. Gács,et al.  Algorithms , 1992 .

[52]  Hakan Erdogan,et al.  A* orthogonal matching pursuit: Best-first search for compressed sensing signal recovery , 2010, Digit. Signal Process..

[53]  Aggelos K. Katsaggelos,et al.  Bayesian Compressive Sensing Using Laplace Priors , 2010, IEEE Transactions on Image Processing.

[54]  Bhaskar D. Rao,et al.  Latent Variable Bayesian Models for Promoting Sparsity , 2011, IEEE Transactions on Information Theory.

[55]  Nils J. Nilsson,et al.  A Formal Basis for the Heuristic Determination of Minimum Cost Paths , 1968, IEEE Trans. Syst. Sci. Cybern..