Efficient Sum of Sparse Outer Products Dictionary Learning (SOUP-DIL)

The sparsity of natural signals in a transform domain or dictionary has been extensively exploited in several applications. More recently, the data-driven adaptation of synthesis dictionaries has shown promise in many applications compared to fixed or analytical dictionaries. However, dictionary learning problems are typically non-convex and NP-hard, and the usual alternating minimization approaches for these problems are often computationally expensive, with the computations dominated by the NP-hard synthesis sparse coding step. In this work, we investigate an efficient method for $\ell_{0}$ "norm"-based dictionary learning by first approximating the training data set with a sum of sparse rank-one matrices and then using a block coordinate descent approach to estimate the rank-one terms. The proposed algorithm involves efficient closed-form solutions. In particular, the sparse coding step involves a simple form of thresholding. We provide a convergence analysis for the proposed block coordinate descent method. Our experiments show the promising performance and significant speed-ups provided by our method over the classical K-SVD scheme in sparse signal representation and image denoising.