Matching Pursuits with random sequential subdictionaries

Matching Pursuits are a class of greedy algorithms commonly used in signal processing, for solving the sparse approximation problem. They rely on an atom selection step that requires the calculation of numerous projections, which can be computationally costly for large dictionaries and burdens their competitiveness in coding applications. We propose using a non-adaptive random sequence of subdictionaries in the decomposition process, thus parsing a large dictionary in a probabilistic fashion with no additional projection cost nor parameter estimation. A theoretical modeling based on order statistics is provided, along with experimental evidence showing that the novel algorithm can be efficiently used on sparse approximation problems. An application to audio signal compression with multiscale time-frequency dictionaries is presented, along with a discussion of the complexity and practical implementations.

[1]  Pierre Vandergheynst,et al.  On the exponential convergence of matching pursuits in quasi-incoherent dictionaries , 2006, IEEE Transactions on Information Theory.

[2]  John Princen,et al.  Analysis/Synthesis filter bank design based on time domain aliasing cancellation , 1986, IEEE Trans. Acoust. Speech Signal Process..

[3]  Martin Vetterli,et al.  Atomic signal models based on recursive filter banks , 1997, Conference Record of the Thirty-First Asilomar Conference on Signals, Systems and Computers (Cat. No.97CB36136).

[4]  Y. C. Pati,et al.  Orthogonal matching pursuit: recursive function approximation with applications to wavelet decomposition , 1993, Proceedings of 27th Asilomar Conference on Signals, Systems and Computers.

[5]  P. Jonathon Phillips.  Matching pursuit filters applied to face identification , 1998, IEEE Trans. Image Process..

[6]  Sacha Krstulovic,et al.  Mptk: Matching Pursuit Made Tractable , 2006, 2006 IEEE International Conference on Acoustics Speech and Signal Processing Proceedings.

[7]  A. J. Bernal,et al.  Probabilistic matching pursuit with Gabor dictionaries , 2000, Signal Process..

[8]  Robert C. Dixon,et al.  Spread Spectrum Systems with Commercial Applications , 2008 .

[9]  Terrence J. Sejnowski,et al.  Coding Time-Varying Signals Using Sparse, Shift-Invariant Representations , 1998, NIPS.

[10]  Michael S. Lewicki,et al.  Efficient auditory coding , 2006, Nature.

[11]  Pierre Vandergheynst,et al.  Universal and efficient compressed sensing by spread spectrum and application to realistic Fourier imaging techniques , 2011, EURASIP J. Adv. Signal Process..

[12]  Michael Elad,et al.  A Plurality of Sparse Representations Is Better Than the Sparsest One Alone , 2009, IEEE Transactions on Information Theory.

[13]  Pascal Frossard,et al.  Low-rate and flexible image coding with redundant representations , 2006, IEEE Transactions on Image Processing.

[14]  R. Tibshirani.  Regression Shrinkage and Selection via the Lasso , 1996 .

[15]  S. Jensen,et al.  The Cyclic Matching Pursuit and its Application to Audio Modeling and Coding , 2007, 2007 Conference Record of the Forty-First Asilomar Conference on Signals, Systems and Computers.

[16]  Bob L. Sturm,et al.  Cyclic matching pursuits with multiscale time-frequency dictionaries , 2010, 2010 Conference Record of the Forty Fourth Asilomar Conference on Signals, Systems and Computers.

[17]  Vladimir N. Temlyakov,et al.  A Criterion for Convergence of Weak Greedy Algorithms , 2002, Adv. Comput. Math..

[18]  Bob L. Sturm,et al.  Sparse Approximation and the Pursuit of Meaningful Signal Models With Interference Adaptation , 2010, IEEE Transactions on Audio, Speech, and Language Processing.

[19]  Rémi Gribonval,et al.  Fast matching pursuit with a multiscale dictionary of Gaussian chirps , 2001, IEEE Trans. Signal Process..

[20]  Bob L. Sturm,et al.  Dark Energy in Sparse Atomic Estimations , 2008, IEEE Transactions on Audio, Speech, and Language Processing.

[21]  Michael A. Saunders,et al.  Atomic Decomposition by Basis Pursuit , 1998, SIAM J. Sci. Comput..

[22]  Philip Schniter,et al.  Fast Bayesian Matching Pursuit: Model Uncertainty and Parameter Estimation for Sparse Linear Models , 2009 .

[23]  Christian Jutten,et al.  Bayesian Pursuit algorithm for sparse representation , 2009, 2009 IEEE International Conference on Acoustics, Speech and Signal Processing.

[24]  Meir Feder,et al.  On universal quantization by randomized uniform/lattice quantizers , 1992, IEEE Trans. Inf. Theory.

[25]  Xavier Rodet,et al.  Analysis of sound signals with high resolution matching pursuit , 1996, Proceedings of Third International Symposium on Time-Frequency and Time-Scale Analysis (TFTS-96).

[26]  Piotr J. Durka,et al.  Stochastic time-frequency dictionaries for matching pursuit , 2001, IEEE Trans. Signal Process..

[27]  Gregoire Nicolis,et al.  Stochastic resonance , 2007, Scholarpedia.

[28]  Gaël Richard,et al.  Audio Signal Representations for Indexing in the Transform Domain , 2010, IEEE Transactions on Audio, Speech, and Language Processing.

[29]  Mike E. Davies,et al.  Gradient Pursuits , 2008, IEEE Transactions on Signal Processing.

[30]  Gaël Richard,et al.  AUDIO SIGNAL REPRESENTATIONS FOR FACTORIZATION IN THE SPARSE DOMAIN , 2011, 2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[31]  Przemyslaw Dymarski,et al.  Greedy sparse decompositions: a comparative study , 2011, EURASIP J. Adv. Signal Process..

[32]  Pierre Vandergheynst,et al.  A low complexity Orthogonal Matching Pursuit for sparse signal approximation with shift-invariant dictionaries , 2009, 2009 IEEE International Conference on Acoustics, Speech and Signal Processing.

[33]  Mohammad Ahsanullah,et al.  Order Statistics for Discrete Distributions , 2013 .

[34]  Okan K. Ersoy,et al.  Probabilistic Matching Pursuit for Compressive Sensing , 2010 .

[35]  R.H. Pettit,et al.  Spread spectrum systems , 1977, Proceedings of the IEEE.

[36]  Joel A. Tropp,et al.  Signal Recovery From Random Measurements Via Orthogonal Matching Pursuit , 2007, IEEE Transactions on Information Theory.

[37]  P. Frossard,et al.  Tree-Based Pursuit: Algorithm and Properties , 2006, IEEE Transactions on Signal Processing.

[38]  Simon J. Godsill,et al.  Sparse Linear Regression With Structured Priors and Application to Denoising of Musical Audio , 2008, IEEE Transactions on Audio, Speech, and Language Processing.

[39]  Gaël Richard,et al.  Union of MDCT Bases for Audio Coding , 2008, IEEE Transactions on Audio, Speech, and Language Processing.

[40]  Shanti S. Gupta,et al.  On the order statistics from equally correlated normal random variables , 1973 .

[41]  A. Willsky,et al.  HIGH RESOLUTION PURSUIT FOR FEATURE EXTRACTION , 1998 .

[42]  Stéphane Mallat,et al.  Matching pursuits with time-frequency dictionaries , 1993, IEEE Trans. Signal Process..

[43]  Laurent Daudet,et al.  Sparse and structured decompositions of signals with the molecular matching pursuit , 2006, IEEE Transactions on Audio, Speech, and Language Processing.