Dictionary Learning for Scalable Sparse Image Representation with Applications

This paper introduces a novel design for the dictionary learning algorithm, intended for scalable sparse representation of high motion video sequences and natural images. The proposed algorithm is built upon the foundation of the K-SVD framework originally designed to learn non-scalable dictionaries for natural images. Proposed design is mainly motivated by the main perception characteristic of the Human Visual System (HVS) mechanism. Specifically, its core structure relies on the exploitation of the high-frequency image components and contrast variations in order to achieve visual scene objects identification at all scalable levels. Proposed design is implemented by introducing a semi-random Morphological Component Analysis (MCA) based initialization of the K-SVD dictionary and the regularization of its atom’s update mechanism. In general, dictionary learning for sparse representations leads to state-of-the-art image restoration results for several different problems in the field of image processing. In experimental section we show that these are equally achievable by accommodating all dictionary elements to tailor the scalable data representation and reconstruction, hence modeling data that admit sparse representation in a novel manner. Performed simulations include scalable sparse recovery for representation of static and dynamic data changing over time (e.g., video) together with application to denoising and compressive sensing.

[1]  E.J. Candes,et al.  An Introduction To Compressive Sampling , 2008, IEEE Signal Processing Magazine.

[2]  Michael Elad,et al.  From Sparse Solutions of Systems of Equations to Sparse Modeling of Signals and Images , 2009, SIAM Rev..

[3]  Samuel Cheng,et al.  Compressive video sampling , 2008, 2008 16th European Signal Processing Conference.

[4]  S. Frick,et al.  Compressed Sensing , 2014, Computer Vision, A Reference Guide.

[5]  David L. Donoho,et al.  Sparse Solution Of Underdetermined Linear Equations By Stagewise Orthogonal Matching Pursuit , 2006 .

[6]  E Peli,et al.  Contrast sensitivity to patch stimuli: effects of spatial bandwidth and temporal presentation. , 1993, Spatial vision.

[7]  Guillermo Sapiro,et al.  Supervised Dictionary Learning , 2008, NIPS.

[8]  Zhifeng Zhang,et al.  Adaptive time-frequency decompositions , 1994 .

[9]  Allen Y. Yang,et al.  Robust Face Recognition via Sparse Representation , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  A. Bruckstein,et al.  K-SVD : An Algorithm for Designing of Overcomplete Dictionaries for Sparse Representation , 2005 .

[11]  Jianhong Shen,et al.  Deblurring images: Matrices, spectra, and filtering , 2007, Math. Comput..

[12]  Ting Sun,et al.  Single-pixel imaging via compressive sampling , 2008, IEEE Signal Process. Mag..

[13]  Michael Elad,et al.  Dictionaries for Sparse Representation Modeling , 2010, Proceedings of the IEEE.

[14]  Mohamed-Jalal Fadili,et al.  Sparsity and Morphological Diversity in Blind Source Separation , 2007, IEEE Transactions on Image Processing.

[15]  Vladimir Stankovic,et al.  Learning scalable dictionaries with application to scalable compressive sensing , 2012, 2012 Proceedings of the 20th European Signal Processing Conference (EUSIPCO).

[16]  David J. Field,et al.  Sparse coding with an overcomplete basis set: A strategy employed by V1? , 1997, Vision Research.

[17]  Michael Elad,et al.  Learning Multiscale Sparse Representations for Image and Video Restoration , 2007, Multiscale Model. Simul..

[18]  J. Robson,et al.  Application of fourier analysis to the visibility of gratings , 1968, The Journal of physiology.

[19]  Samuel Cheng,et al.  Scalable compressive video , 2011, 2011 18th IEEE International Conference on Image Processing.

[20]  H. Pashler STEVENS' HANDBOOK OF EXPERIMENTAL PSYCHOLOGY , 2002 .

[21]  J. Romberg,et al.  Imaging via Compressive Sampling , 2008, IEEE Signal Processing Magazine.

[22]  D. Donoho,et al.  Simultaneous cartoon and texture image inpainting using morphological component analysis (MCA) , 2005 .

[23]  Emmanuel J. Candès,et al.  Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information , 2004, IEEE Transactions on Information Theory.

[24]  Anat Levin,et al.  Natural image denoising: Optimality and inherent bounds , 2011, CVPR 2011.

[25]  Michael Elad,et al.  Sparse and Redundant Representations - From Theory to Applications in Signal and Image Processing , 2010 .

[26]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[27]  James A. Ferwerda,et al.  Elements of Early Vision for Computer Graphics (Tutorial) , 2001, IEEE Computer Graphics and Applications.

[28]  Thomas S. Huang,et al.  Image Super-Resolution Via Sparse Representation , 2010, IEEE Transactions on Image Processing.

[29]  Samuel Cheng,et al.  HVS based dictionary learning for scalable sparse image representation , 2012, 2012 Conference Record of the Forty Sixth Asilomar Conference on Signals, Systems and Computers (ASILOMAR).

[30]  Lei Zhang,et al.  Sparsity-based image denoising via dictionary learning and structural clustering , 2011, CVPR 2011.

[31]  Michael Elad,et al.  Sparse Representation for Color Image Restoration , 2008, IEEE Transactions on Image Processing.

[32]  K. Baker,et al.  Singular Value Decomposition Tutorial , 2013 .

[33]  Feng Liu,et al.  Image Denoising Via Sparse and Redundant Representations Over Learned Dictionaries in Wavelet Domain , 2009, 2009 Fifth International Conference on Image and Graphics.

[34]  Thong T. Do,et al.  Sparsity adaptive matching pursuit algorithm for practical compressed sensing , 2008, 2008 42nd Asilomar Conference on Signals, Systems and Computers.

[35]  Michael Elad,et al.  On Single Image Scale-Up Using Sparse-Representations , 2010, Curves and Surfaces.

[36]  Bojana Begovic Dictionary learning for scalable sparse image representation , 2016 .

[37]  Michael Elad,et al.  Image Denoising Via Sparse and Redundant Representations Over Learned Dictionaries , 2006, IEEE Transactions on Image Processing.

[38]  Guillermo Sapiro,et al.  Learning to Sense Sparse Signals: Simultaneous Sensing Matrix and Sparsifying Dictionary Optimization , 2009, IEEE Transactions on Image Processing.

[39]  Dianne P. O'Leary,et al.  Deblurring Images: Matrices, Spectra, and Filtering (Fundamentals of Algorithms 3) (Fundamentals of Algorithms) , 2006 .

[40]  Xuelong Li,et al.  Image quality assessment and human visual system , 2010, Visual Communications and Image Processing.

[41]  Hans-Peter Seidel,et al.  Visually significant edges , 2010, TAP.

[42]  Rajat Raina,et al.  Efficient sparse coding algorithms , 2006, NIPS.

[43]  Lie Wang,et al.  Orthogonal Matching Pursuit for Sparse Signal Recovery With Noise , 2011, IEEE Transactions on Information Theory.

[44]  M. Elad,et al.  $rm K$-SVD: An Algorithm for Designing Overcomplete Dictionaries for Sparse Representation , 2006, IEEE Transactions on Signal Processing.

[45]  Ke Huang,et al.  Sparse Representation for Signal Classification , 2006, NIPS.

[46]  Michael Elad,et al.  Optimized Projections for Compressed Sensing , 2007, IEEE Transactions on Signal Processing.

[47]  James A. Ferwerda,et al.  Fundamentals of Spatial Vision , 1998 .

[48]  Jamshed J. Bharucha,et al.  STEVENS’ HANDBOOK OF EXPERIMENTAL PSYCHOLOGY , 2002 .

[49]  Richard G. Baraniuk,et al.  Compressive imaging for video representation and coding , 2006 .