EK-SVD: Optimized dictionary design for sparse representations

Sparse representations using overcomplete dictionaries are used in a variety of field such as pattern recognition and compression. However, the size of dictionary is usually a tradeoff between approximation speed and accuracy. In this paper we propose a novel technique called the Enhanced K-SVD algorithm (EK-SVD), which finds a dictionary of optimized size-for a given dataset, without compromising its approximation accuracy. EK-SVD improves the K-SVD dictionary learning algorithm by introducing an optimized dictionary size discovery feature to K-SVD. Optimizing strict sparsity and MSE constraints, it starts with a large number of dictionary elements and gradually prunes the under-utilized or similar-looking elements to produce a well-trained dictionary that has no redundant elements. Experimental results show the optimized dictionaries learned using EK-SVD give the same accuracy as dictionaries learned using the K-SVD algorithm while substantially reducing the dictionary size by 60%.

[1]  Shahram Shirani,et al.  Matching Pursuit-Based Region-of-Interest Image Coding , 2007, IEEE Transactions on Image Processing.

[2]  Joseph N. Wilson,et al.  A matching pursuit based similarity measure for fuzzy clustering and classification of signals , 2008, 2008 IEEE International Conference on Fuzzy Systems (IEEE World Congress on Computational Intelligence).

[3]  A. Bruckstein,et al.  K-SVD : An Algorithm for Designing of Overcomplete Dictionaries for Sparse Representation , 2005 .

[4]  Hichem Frigui,et al.  Clustering by competitive agglomeration , 1997, Pattern Recognit..

[5]  Stéphane Mallat,et al.  Matching pursuits with time-frequency dictionaries , 1993, IEEE Trans. Signal Process..

[6]  Avideh Zakhor,et al.  Very low bit-rate video coding based on matching pursuits , 1997, IEEE Trans. Circuits Syst. Video Technol..

[7]  M. Elad,et al.  $rm K$-SVD: An Algorithm for Designing Overcomplete Dictionaries for Sparse Representation , 2006, IEEE Transactions on Signal Processing.