An efficient algorithm for dictionary learning with a mixed-norm regularizer for sparsity based on proximal operator

Dictionary learning often use ℓp-norm (p <; 2) for sparsity. In this paper, we use a different structure of regularizer, i.e. mixed ℓ1,2-norm regularizer for group sparsity. We propose a method based on a decomposition scheme and alternating optimization that can turn the whole problem into a set of subminimizations of univariate functions, each of which is dependent on only one dictionary atom or the coefficient vector. The ℓ1,2-norm regularizer induces sparsity over each coefficient vector rather than single coefficients. Although the subproblem with respect to the coefficient vector is still nonconvex, remarkably, it becomes much simpler and it has a closed-form solution by introducing a technique that is proximal operator. Hence, we propose a fast and efficient algorithm for learning overcomplete dictionary using the mixed ℓ1,2-norm regularizer. Due to the usage of the proximal operator, the proposed algorithm can achieve a nonliner convergence rate. The main advantages of the proposed algorithm is that, as suggested by the simulation study, it is faster and more efficient than state-of-the-art algorithms with different sparsity constraints.