The Viterbi Algorithm for Subset Selection

We study the problem of sparse recovery in an overcomplete dictionary. This problem has attracted considerable attention in signal processing, statistics, and computer science, and a variety of algorithms have been developed to recover the sparse vector. We propose a new method based on the computationally efficient Viterbi algorithm which is shown to achieve better performance than competing algorithms such as Orthogonal Matching Pursuit (OMP), Orthogonal Least-Squares (OLS), Multi-Branch Matching Pursuit (MBMP), Iterative Hard Thresholding (IHT), and l1 minimization. We also explore the relationship of the Viterbi-based approach with OLS.

[1]  Y. C. Pati,et al.  Orthogonal matching pursuit: recursive function approximation with applications to wavelet decomposition , 1993, Proceedings of 27th Asilomar Conference on Signals, Systems and Computers.

[2]  Sheng Chen,et al.  Orthogonal least squares methods and their application to non-linear system identification , 1989 .

[3]  Deanna Needell,et al.  CoSaMP: Iterative signal recovery from incomplete and inaccurate samples , 2008, ArXiv.

[4]  Jr. G. Forney,et al.  Viterbi Algorithm , 1973, Encyclopedia of Machine Learning.

[5]  R. C. Davis,et al.  Coherent Demodulation of Continuous Phase Binary FSK Signals , 1971 .

[6]  Andrew J. Viterbi,et al.  Error bounds for convolutional codes and an asymptotically optimum decoding algorithm , 1967, IEEE Trans. Inf. Theory.

[7]  Charles Soussen,et al.  Joint K-Step Analysis of Orthogonal Matching Pursuit and Orthogonal Least Squares , 2011, IEEE Transactions on Information Theory.

[8]  J. Tropp,et al.  CoSaMP: Iterative signal recovery from incomplete and inaccurate samples , 2008, Commun. ACM.

[9]  Mike E. Davies,et al.  Gradient Pursuits , 2008, IEEE Transactions on Signal Processing.

[10]  T. Blumensath,et al.  Theory and Applications , 2011 .

[11]  Stéphane Mallat,et al.  Matching pursuits with time-frequency dictionaries , 1993, IEEE Trans. Signal Process..

[12]  Yonina C. Eldar,et al.  Multi-Branch Matching Pursuit with applications to MIMO radar , 2013, ArXiv.

[13]  David L. Neuhoff,et al.  The Viterbi algorithm as an aid in text recognition (Corresp.) , 1975, IEEE Trans. Inf. Theory.

[14]  G. David Forney,et al.  Maximum-likelihood sequence estimation of digital sequences in the presence of intersymbol interference , 1972, IEEE Trans. Inf. Theory.

[15]  Michael A. Saunders,et al.  Atomic Decomposition by Basis Pursuit , 1998, SIAM J. Sci. Comput..

[16]  Olgica Milenkovic,et al.  Subspace Pursuit for Compressive Sensing Signal Reconstruction , 2008, IEEE Transactions on Information Theory.