Dictionary Learning for Sparse Representations: A Pareto Curve Root Finding Approach

A new dictionary learning method for exact sparse representation is presented in this paper. As the dictionary learning methods often iteratively update the sparse coefficients and dictionary, when the approximation error is small or zero, algorithm convergence will be slow or non-existent. The proposed framework can be used in such a setting by gradually increasing the fidelity of the approximation. This technique has previously been used for the convex sparse representations. It has been extended here to the non-convex dictionary learning problem by allowing the dictionary be modified.

[1]  Michael P. Friedlander,et al.  Probing the Pareto Frontier for Basis Pursuit Solutions , 2008, SIAM J. Sci. Comput..

[2]  Mike E. Davies,et al.  Parametric Dictionary Design for Sparse Coding , 2009, IEEE Transactions on Signal Processing.

[3]  Kjersti Engan,et al.  Method of optimal directions for frame design , 1999, 1999 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings. ICASSP99 (Cat. No.99CH36258).

[4]  Stéphane Mallat,et al.  Matching pursuits with time-frequency dictionaries , 1993, IEEE Trans. Signal Process..

[5]  A. Bruckstein,et al.  K-SVD : An Algorithm for Designing of Overcomplete Dictionaries for Sparse Representation , 2005 .

[6]  R. Tibshirani.  Regression Shrinkage and Selection via the Lasso , 1996 .

[7]  A. Bruckstein,et al.  On the uniqueness of overcomplete dictionaries, and a practical way to retrieve them , 2006 .

[8]  A. Goldstein.  Convex programming in Hilbert space , 1964 .

[9]  Terrence J. Sejnowski,et al.  Learning Overcomplete Representations , 2000, Neural Computation.

[10]  Mark D. Plumbley.  Dictionary Learning for L1-Exact Sparse Coding , 2007, ICA.

[11]  M. Elad,et al.  $rm K$-SVD: An Algorithm for Designing Overcomplete Dictionaries for Sparse Representation , 2006, IEEE Transactions on Signal Processing.

[12]  Mike E. Davies,et al.  Dictionary Learning for Sparse Approximations With the Majorization Method , 2009, IEEE Transactions on Signal Processing.

[13]  R. Gribonval,et al.  Some recovery conditions for basis learning by L1-minimization , 2008, 2008 3rd International Symposium on Communications, Control and Signal Processing.

[14]  Karin Schnass,et al.  Dictionary Identification—Sparse Matrix-Factorization via $\ell_1$ -Minimization , 2009, IEEE Transactions on Information Theory.

[15]  Robert B. Ash,et al.  Information Theory , 2020, The SAGE International Encyclopedia of Mass Media and Society.

[16]  David J. Field,et al.  Sparse coding with an overcomplete basis set: A strategy employed by V1? , 1997, Vision Research.

[17]  Michael A. Saunders,et al.  Atomic Decomposition by Basis Pursuit , 1998, SIAM J. Sci. Comput..

[18]  Fabian J. Theis,et al.  Sparse component analysis and blind source separation of underdetermined mixtures , 2005, IEEE Transactions on Neural Networks.