A Proximal Method for Convolutional Dictionary Learning with Convergence Property

The convolutional sparse coding (CSC) is superior in representing signals, and to obtain the best performance of CSC, the dictionary is usually learned from data. The so-called convolution dictionary learning (CDL) problem is therefore formulated for the purpose. Most of the solvers for CDL alternately update the coefficients and dictionary in an iterative manner, and as a consequence, numerous redundant iterations incur slow speed in achieving convergence. Moreover, their convergence properties can hardly be analyzed even though the $\ell_{0}$ sparse inducing function is approximated by the convex $\ell_{0}$ norm. In this article, we propose an algorithm which directly deals with the non-convex non-smooth $\ell_{0}$ constraint and provides a sound convergence property. The experimental results show that, the proposed method achieves the convergence point with less time and a smaller final function value compared to the existing convolutional dictionary learning algorithms.

[1]  Laurent Demanet,et al.  Fast Discrete Curvelet Transforms , 2006, Multiscale Model. Simul..

[2]  Marc Teboulle,et al.  A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems , 2009, SIAM J. Imaging Sci..

[3]  Quentin Barthelemy,et al.  Shift & 2D Rotation Invariant Sparse Coding for Multivariate Signals , 2012, IEEE Transactions on Signal Processing.

[4]  Simon Lucey,et al.  Optimization Methods for Convolutional Sparse Coding , 2014, ArXiv.

[5]  Gordon Wetzstein,et al.  Fast and flexible convolutional sparse coding , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[6]  José Carlos Príncipe,et al.  A fast proximal method for convolutional sparse coding , 2013, The 2013 International Joint Conference on Neural Networks (IJCNN).

[7]  Brendt Wohlberg,et al.  Efficient Algorithms for Convolutional Sparse Representations , 2016, IEEE Transactions on Image Processing.

[8]  Tong Zhang,et al.  Analysis of Multi-stage Convex Relaxation for Sparse Regularization , 2010, J. Mach. Learn. Res..

[9]  Yann LeCun,et al.  Convolutional Matching Pursuit and Dictionary Training , 2010, ArXiv.

[10]  Rajat Raina,et al.  Efficient sparse coding algorithms , 2006, NIPS.

[11]  Michael Elad,et al.  Working Locally Thinking Globally: Theoretical Guarantees for Convolutional Sparse Coding , 2017, IEEE Transactions on Signal Processing.

[12]  Benar Fux Svaiter,et al.  Convergence of descent methods for semi-algebraic and tame problems: proximal algorithms, forward–backward splitting, and regularized Gauss–Seidel methods , 2013, Math. Program..

[13]  Jeffrey A. Fessler,et al.  Convolutional Dictionary Learning: Acceleration and Convergence , 2017, IEEE Transactions on Image Processing.

[14]  S. Mallat A wavelet tour of signal processing , 1998 .

[15]  Xiaoming Huo,et al.  Uncertainty principles and ideal atomic decomposition , 2001, IEEE Trans. Inf. Theory.

[16]  Anders P. Eriksson,et al.  Fast Convolutional Sparse Coding , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.