Signal overcomplete representation and sparse decomposition based on redundant dictionaries

Decomposing a signal based upon redundant dictionaries is a new method for data representation on signal processing. It approximates a signal with an overcomplete system instead of an orthonormal basis to provide a sufficient choice for adaptive sparse decompositions. Replacing the original data with a sparse approximation can result in not only a higher compression ratio, but also greater flexibility in capturing the inherent structure of the natural signals with the redundancy of dictionaries. This paper gives an overview of a series of recent results in this field, and deals with the relationship between sparsity of signal decomposition and incoherence of dictionaries with BP and MP algorithms. The current and future challenges of the dictionary construction are discussed.

[1]  L. Jones.  On a conjecture of Huber concerning the convergence of projection pursuit regression , 1987 .

[2]  Emmanuel J. Candès,et al.  The curvelet transform for image denoising , 2002, IEEE Trans. Image Process..

[3]  Morten Nielsen,et al.  Approximation with highly redundant dictionaries , 2003, SPIE Optics + Photonics.

[4]  Michael Elad,et al.  A generalized uncertainty principle and sparse representation in pairs of bases , 2002, IEEE Trans. Inf. Theory.

[5]  J. Cheung,et al.  Minimum fuel neural networks and their applications to overcomplete signal representations , 2000 .

[6]  Alexander Schrijver,et al.  Theory of linear and integer programming , 1986, Wiley-Interscience series in discrete mathematics and optimization.

[7]  R. Gribonval,et al.  Nonlinear Approximation with Dictionaries. II. Inverse Estimates , 2006 .

[8]  Wasfy B. Mikhael,et al.  A survey of mixed transform techniques for speech and image coding , 1999, ISCAS'99. Proceedings of the 1999 IEEE International Symposium on Circuits and Systems VLSI (Cat. No.99CH36349).

[9]  Thomas Strohmer,et al.  GRASSMANNIAN FRAMES WITH APPLICATIONS TO CODING AND COMMUNICATION , 2003, math/0301135.

[10]  Ronald A. DeVore,et al.  Some remarks on greedy algorithms , 1996, Adv. Comput. Math..

[11]  J Barron.  Chapter 21 – Series and Transforms , 2007 .

[12]  E. Candès,et al.  New tight frames of curvelets and optimal representations of objects with piecewise C2 singularities , 2004 .

[13]  Ronald R. Coifman,et al.  Entropy-based algorithms for best basis selection , 1992, IEEE Trans. Inf. Theory.

[14]  E. Candès,et al.  Curvelets: A Surprisingly Effective Nonadaptive Representation for Objects with Edges , 2000 .

[15]  Rémi Gribonval,et al.  Sparse representations in unions of bases , 2003, IEEE Trans. Inf. Theory.

[16]  A. Calderbank,et al.  Z4‐Kerdock Codes, Orthogonal Spreads, and Extremal Euclidean Line‐Sets , 1997 .

[17]  Jean-Luc Starck,et al.  Image decomposition: separation of texture from piecewise smooth content , 2003, SPIE Optics + Photonics.

[18]  Victor E. DeBrunner,et al.  Lapped multiple bases algorithms for still image compression without blocking effect , 1997, IEEE Trans. Image Process..

[19]  Vladimir N. Temlyakov,et al.  Nonlinear Methods of Approximation , 2003, Found. Comput. Math..

[20]  Minh N. Do,et al.  Ieee Transactions on Image Processing the Contourlet Transform: an Efficient Directional Multiresolution Image Representation , 2022 .

[21]  Michael A. Saunders,et al.  Atomic Decomposition by Basis Pursuit , 1998, SIAM J. Sci. Comput..

[22]  S. Muthukrishnan,et al.  Approximation of functions over redundant dictionaries using coherence , 2003, SODA '03.

[23]  Arkadi Nemirovski,et al.  On sparse representation in pairs of bases , 2003, IEEE Trans. Inf. Theory.

[24]  Stéphane Mallat,et al.  Matching pursuits with time-frequency dictionaries , 1993, IEEE Trans. Signal Process..

[25]  Bhaskar D. Rao,et al.  Sparse channel estimation via matching pursuit with application to equalization , 2002, IEEE Trans. Commun..

[26]  S. Mallat,et al.  Adaptive greedy approximations , 1997 .

[27]  Stéphane Mallat,et al.  Sparse geometric image representations with bandelets , 2005, IEEE Transactions on Image Processing.

[28]  Michael Elad,et al.  Optimally sparse representation in general (nonorthogonal) dictionaries via ℓ1 minimization , 2003, Proceedings of the National Academy of Sciences of the United States of America.

[29]  J. Friedman,et al.  Projection Pursuit Regression , 1981 .

[30]  J. Tropp.  JUST RELAX: CONVEX PROGRAMMING METHODS FOR SUBSET SELECTION AND SPARSE APPROXIMATION , 2004 .

[31]  Xiaoming Huo,et al.  Uncertainty principles and ideal atomic decomposition , 2001, IEEE Trans. Inf. Theory.

[32]  Pascal Frossard,et al.  Efficient image representation by anisotropic refinement in matching pursuit , 2001, 2001 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings (Cat. No.01CH37221).

[33]  S. Muthukrishnan,et al.  Improved sparse approximation over quasiincoherent dictionaries , 2003, Proceedings 2003 International Conference on Image Processing (Cat. No.03CH37429).

[34]  Y. C. Pati,et al.  Orthogonal matching pursuit: recursive function approximation with applications to wavelet decomposition , 1993, Proceedings of 27th Asilomar Conference on Signals, Systems and Computers.

[35]  D. Donoho,et al.  Maximal Sparsity Representation via l 1 Minimization , 2002 .

[36]  Avideh Zakhor,et al.  Very low bit-rate video coding based on matching pursuits , 1997, IEEE Trans. Circuits Syst. Video Technol..

[37]  B. Golubov,et al.  Walsh Series and Transforms , 1991 .

[38]  David J. Field,et al.  Sparse coding with an overcomplete basis set: A strategy employed by V1? , 1997, Vision Research.

[39]  Vladimir N. Temlyakov,et al.  Weak greedy algorithms[*]This research was supported by National Science Foundation Grant DMS 9970326 and by ONR Grant N00014‐96‐1‐1003. , 2000, Adv. Comput. Math..

[40]  Pierre Vandergheynst,et al.  On the exponential convergence of matching pursuits in quasi-incoherent dictionaries , 2006, IEEE Transactions on Information Theory.

[41]  Ingrid Daubechies,et al.  Time-frequency localization operators: A geometric phase space approach , 1988, IEEE Trans. Inf. Theory.