Boosted Dictionary Learning for Image Compression

Sparse representations over redundant dictionaries have shown to produce high-quality results in various signal and image processing tasks. Recent advancements in learning-based dictionary design have made image compression using data-adaptive learned dictionaries a promising field. In this paper, we present a boosted dictionary learning framework to construct an ensemble of complementary specialized dictionaries for sparse image representation. Boosted dictionaries along with a competitive sparse coding form our ensemble model which can provide us with more efficient sparse representations. The constituent dictionaries of the ensemble are obtained using a coherence regularized dictionary learning model for which two novel dictionary optimization algorithms are proposed. These algorithms improve the generalization properties of the trained dictionary compared with several incoherent dictionary learning methods. Based on the proposed ensemble model, we then develop a new image compression algorithm using boosted multi-scale dictionaries learned in the wavelet domain. Our algorithm is evaluated for the compression of natural images. Experimental results demonstrate that the proposed algorithm has better rate-distortion performance as compared with several competing compression methods, including analytic and learned dictionary schemes.

[1]  Michael Elad,et al.  Multi-Scale Dictionary Learning Using Wavelets , 2011, IEEE Journal of Selected Topics in Signal Processing.

[2]  Y. C. Pati,et al.  Orthogonal matching pursuit: recursive function approximation with applications to wavelet decomposition , 1993, Proceedings of 27th Asilomar Conference on Signals, Systems and Computers.

[3]  Ichiro Matsuda,et al.  Lossless coding using variable block-size adaptive prediction optimized for each image , 2005, 2005 13th European Signal Processing Conference.

[4]  R. Tibshirani,et al.  Least angle regression , 2004, math/0406456.

[5]  Michael Elad,et al.  Facial Image Compression using Patch-Ordering-Based Adaptive Wavelet Transform , 2014, IEEE Signal Processing Letters.

[6]  Pierre Vandergheynst,et al.  Dictionary Preconditioning for Greedy Algorithms , 2008, IEEE Transactions on Signal Processing.

[7]  Jianhua Lu,et al.  Compressibility Constrained Sparse Representation With Learnt Dictionary for Low Bit-Rate Image Compression , 2014, IEEE Transactions on Circuits and Systems for Video Technology.

[8]  Karthik S. Gurumoorthy,et al.  A Method for Compact Image Representation Using Sparse Matrix and Tensor Projections Onto Exemplar Orthonormal Bases , 2010, IEEE Transactions on Image Processing.

[9]  Michael Elad,et al.  Compression of facial images using the K-SVD algorithm , 2008, J. Vis. Commun. Image Represent..

[10]  I. Horev,et al.  Adaptive image compression using sparse dictionaries , 2012, 2012 19th International Conference on Systems, Signals and Image Processing (IWSSIP).

[11]  Guillermo Sapiro,et al.  Sparse Representation for Computer Vision and Pattern Recognition , 2010, Proceedings of the IEEE.

[12]  Mark D. Plumbley,et al.  INK-SVD: Learning incoherent dictionaries for sparse representations , 2012, 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[13]  A. Bruckstein,et al.  K-SVD : An Algorithm for Designing of Overcomplete Dictionaries for Sparse Representation , 2005 .

[14]  Mark D. Plumbley,et al.  Learning Incoherent Dictionaries for Sparse Approximation Using Iterative Projections and Rotations , 2013, IEEE Transactions on Signal Processing.

[15]  Alan C. Bovik,et al.  Image information and visual quality , 2006, IEEE Trans. Image Process..

[16]  T. Ebrahimi,et al.  The Jpeg Xr Image Coding Standard , .

[17]  Michael Elad,et al.  On the Role of Sparse and Redundant Representations in Image Processing , 2010, Proceedings of the IEEE.

[18]  Guillermo Sapiro,et al.  Online dictionary learning for sparse coding , 2009, ICML '09.

[19]  A. Said Introduction to Arithmetic Coding - Theory and Practice , 2023, ArXiv.

[20]  B. Ugur Töreyin,et al.  Sparse coding of hyperspectral imagery using online learning , 2015, Signal Image Video Process..

[21]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[22]  Touradj Ebrahimi,et al.  The JPEG 2000 still image compression standard , 2001, IEEE Signal Process. Mag..

[23]  Joan L. Mitchell,et al.  JPEG: Still Image Data Compression Standard , 1992 .

[24]  Michael Elad,et al.  From Sparse Solutions of Systems of Equations to Sparse Modeling of Signals and Images , 2009, SIAM Rev..

[25]  Chia-Chi Tsui,et al.  Solution Of Matrix Equation , 2003 .

[26]  Michael Elad,et al.  Dictionaries for Sparse Representation Modeling , 2010, Proceedings of the IEEE.

[27]  Luc Van Gool,et al.  Latent Dictionary Learning for Sparse Representation Based Classification , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[28]  Christine Guillemot,et al.  Image Compression Using Sparse Representations and the Iteration-Tuned and Aligned Dictionary , 2011, IEEE Journal of Selected Topics in Signal Processing.

[29]  John Wright,et al.  Complete Dictionary Recovery Using Nonconvex Optimization , 2015, ICML.

[30]  Jun Guo,et al.  Bases sorting: Generalizing the concept of frequency for over-complete dictionaries , 2013, Neurocomputing.

[31]  Guillermo Sapiro,et al.  Sparse Modeling with Universal Priors and Learned Incoherent Dictionaries(PREPRINT) , 2009 .

[32]  Guillermo Sapiro,et al.  Classification and clustering via dictionary learning with structured incoherence and shared features , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[33]  Karthikeyan Natesan Ramamurthy,et al.  Ensemble Sparse Models for Image Analysis , 2013, ArXiv.

[34]  Michael Elad,et al.  Sparse and Redundant Representations - From Theory to Applications in Signal and Image Processing , 2010 .

[35]  F. Dufaux,et al.  The JPEG XR image coding standard [Standards in a Nutshell] , 2009, IEEE Signal Processing Magazine.

[36]  Thomas G. Dietterich Multiple Classifier Systems , 2000, Lecture Notes in Computer Science.

[37]  William A. Pearlman,et al.  A new, fast, and efficient image codec based on set partitioning in hierarchical trees , 1996, IEEE Trans. Circuits Syst. Video Technol..

[38]  J. Magnus,et al.  Matrix Differential Calculus with Applications in Statistics and Econometrics , 1991 .

[39]  Guillermo Sapiro,et al.  Online Learning for Matrix Factorization and Sparse Coding , 2009, J. Mach. Learn. Res..

[40]  Joel A. Tropp,et al.  Greed is good: algorithmic results for sparse approximation , 2004, IEEE Transactions on Information Theory.

[41]  Richard H. Bartels,et al.  Algorithm 432 [C2]: Solution of the matrix equation AX + XB = C [F4] , 1972, Commun. ACM.

[42]  D. Shiva Rama Krishna,et al.  Fingerprint Compression Based on Sparse Representation , 2015 .

[43]  Christine Connolly,et al.  Handbook of Image and Video Processing 2nd Edition (Hardback) , 2006 .

[44]  Rong Zhang,et al.  Remote sensing image compression based on double-sparsity dictionary learning and universal trellis coded quantization , 2013, 2013 IEEE International Conference on Image Processing.

[45]  M. Elad,et al.  $rm K$-SVD: An Algorithm for Designing Overcomplete Dictionaries for Sparse Representation , 2006, IEEE Transactions on Signal Processing.

[46]  Jerry D. Gibson,et al.  Handbook of Image and Video Processing , 2000 .

[47]  Kjersti Engan,et al.  Method of optimal directions for frame design , 1999, 1999 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings. ICASSP99 (Cat. No.99CH36258).

[48]  M. T. Qureshi,et al.  Lyapunov Matrix Equation in System Stability and Control , 2008 .

[49]  Lap-Pui Chau,et al.  Expression-invariant and sparse representation for mesh-based compression for 3-D face models , 2013, 2013 Visual Communications and Image Processing (VCIP).

[50]  Kjersti Engan,et al.  Recursive Least Squares Dictionary Learning Algorithm , 2010, IEEE Transactions on Signal Processing.

[51]  David Zhang,et al.  FSIM: A Feature Similarity Index for Image Quality Assessment , 2011, IEEE Transactions on Image Processing.

[52]  Rong Zhang,et al.  SAR Image Compression Using Multiscale Dictionary Learning and Sparse Representation , 2013, IEEE Geoscience and Remote Sensing Letters.

[53]  Robert E. Schapire,et al.  The Boosting Approach to Machine Learning An Overview , 2003 .

[54]  Michael Elad,et al.  Double Sparsity: Learning Sparse Dictionaries for Sparse Signal Approximation , 2010, IEEE Transactions on Signal Processing.

[55]  Kjersti Engan,et al.  Image compression using learned dictionaries by RLS-DLA and compared with K-SVD , 2011, 2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[56]  Huan Wang,et al.  Exact Recovery of Sparsely-Used Dictionaries , 2012, COLT.

[57]  Julien Mairal,et al.  Optimization with Sparsity-Inducing Penalties , 2011, Found. Trends Mach. Learn..

[58]  Joachim M. Buhmann,et al.  Learning Dictionaries With Bounded Self-Coherence , 2012, IEEE Signal Processing Letters.

[59]  Azriel Rosenfeld,et al.  Robust regression methods for computer vision: A review , 1991, International Journal of Computer Vision.