Accurate Dictionary Learning with Direct Sparsity Control

Dictionary learning is a popular method for obtaining sparse linear representations for high dimensional data, with many applications in image classification, signal processing and machine learning. In this paper, we introduce a novel dictionary learning method based on a recent variable selection algorithm called Feature Selection with Annealing (FSA). Because FSA uses an $L_{0}$ constraint instead of the $L_{1}$ penalty, it does not introduce any bias in the coefficients and obtains a more accurate sparse representation. Furthermore, the $L_{0}$ constraint makes it easy to directly specify the desired sparsity level instead of indirectly through a $L_{1}$ penalty. Finally, experimental validation on real gray-scale images shows that the proposed method obtains higher accuracy and efficiency in dictionary learning compared to classical methods based on the $L_{1}$ penalty.