Learning Multiscale Sparse Representations for Image and Video Restoration

Abstract : A framework for learning multiscale sparse representations of color images and video with over complete dictionaries is presented in this paper. Following the single-scale grayscale K-SVD algorithm introduced in [1], which formulates the sparse dictionary learning and image representation as an optimization problem efficiently solved via orthogonal matching pursuit and SVD, this proposed multiscale learned representation is obtained based on an efficient quadtree decomposition of the learned dictionary and overlapping image patches. The proposed framework provides an alternative to pre-defined dictionaries such as wavelets, and leads to state-of-the-art results in a number of image and video enhancement and restoration applications. The presentation of the framework here proposed is accompanied by numerous examples demonstrating its practical power.

[1]  Edward H. Adelson,et al.  The Design and Use of Steerable Filters , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[2]  Zhifeng Zhang,et al.  Adaptive time-frequency decompositions , 1994 .

[3]  J. Navarro-Pedreño Numerical Methods for Least Squares Problems , 1996 .

[4]  S. Mallat,et al.  Adaptive greedy approximations , 1997 .

[5]  Michael A. Saunders,et al.  Atomic Decomposition by Basis Pursuit , 1998, SIAM J. Sci. Comput..

[6]  S. Mallat A wavelet tour of signal processing , 1998 .

[7]  Ron Kimmel,et al.  Demosaicing: Image Reconstruction from Color CCD Samples , 1998, ECCV.

[8]  D. Donoho Wedgelets: nearly minimax estimation of edges , 1999 .

[9]  Guillermo Sapiro,et al.  Image inpainting , 2000, SIGGRAPH.

[10]  Taku Komura,et al.  Topology matching for fully automatic similarity estimation of 3D shapes , 2001, SIGGRAPH.

[11]  Jitendra Malik,et al.  A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[12]  Bruno A. Olshausen,et al.  Learning Sparse Multiscale Image Representations , 2002, NIPS.

[13]  E. Candès,et al.  Recovering edges in ill-posed inverse problems: optimality of curvelet frames , 2002 .

[14]  P. Laguna,et al.  Signal Processing , 2002, Yearbook of Medical Informatics.

[15]  Tommi S. Jaakkola,et al.  Weighted Low-Rank Approximations , 2003, ICML.

[16]  Martin J. Wainwright,et al.  Image denoising using scale mixtures of Gaussians in the wavelet domain , 2003, IEEE Trans. Image Process..

[17]  Minh N. Do,et al.  Framing pyramids , 2003, IEEE Trans. Signal Process..

[18]  Joel A. Tropp,et al.  Greed is good: algorithmic results for sparse approximation , 2004, IEEE Transactions on Information Theory.

[19]  Patrick Pérez,et al.  Region filling and object removal by exemplar-based image inpainting , 2004, IEEE Transactions on Image Processing.

[20]  Xin Li,et al.  Demosaicing by successive approximation , 2005, IEEE Transactions on Image Processing.

[21]  Michael J. Black,et al.  Fields of Experts: a framework for learning image priors , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[22]  Stéphane Mallat,et al.  Sparse geometric image representations with bandelets , 2005, IEEE Transactions on Image Processing.

[23]  Stéphane Mallat,et al.  Bandelet Image Approximation and Compression , 2005, Multiscale Model. Simul..

[24]  Thomas W. Parks,et al.  Demosaicing using optimal recovery , 2005, IEEE Transactions on Image Processing.

[25]  Jean-Michel Morel,et al.  A Review of Image Denoising Algorithms, with a New One , 2005, Multiscale Model. Simul..

[26]  A. Bruckstein,et al.  K-SVD : An Algorithm for Designing of Overcomplete Dictionaries for Sparse Representation , 2005 .

[27]  Michael Elad,et al.  Image Denoising Via Learned Dictionaries and Sparse representation , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[28]  Eero P. Simoncelli,et al.  Statistical Modeling of Images with Fields of Gaussian Scale Mixtures , 2006, NIPS.

[29]  Yuk-Hee Chan,et al.  Color Demosaicing Using Variance of Color Differences , 2006, IEEE Transactions on Image Processing.

[30]  Charles Kervrann,et al.  Optimal Spatial Adaptation for Patch-Based Image Denoising , 2006, IEEE Transactions on Image Processing.

[31]  Michael Elad,et al.  Stable recovery of sparse overcomplete representations in the presence of noise , 2006, IEEE Transactions on Information Theory.

[32]  Karen O. Egiazarian,et al.  Image denoising with block-matching and 3D filtering , 2006, Electronic Imaging.

[33]  M. Elad,et al.  $rm K$-SVD: An Algorithm for Designing Overcomplete Dictionaries for Sparse Representation , 2006, IEEE Transactions on Signal Processing.

[34]  Marc'Aurelio Ranzato,et al.  Efficient Learning of Sparse Representations with an Energy-Based Model , 2006, NIPS.

[35]  Joel A. Tropp,et al.  Just relax: convex programming methods for identifying sparse signals in noise , 2006, IEEE Transactions on Information Theory.

[36]  Michael Elad,et al.  Image Denoising Via Sparse and Redundant Representations Over Learned Dictionaries , 2006, IEEE Transactions on Image Processing.

[37]  Eli Shechtman,et al.  Space-Time Completion of Video , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[38]  Alessandro Foi,et al.  Image Denoising by Sparse 3-D Transform-Domain Collaborative Filtering , 2007, IEEE Transactions on Image Processing.

[39]  Guillermo Sapiro,et al.  Video Inpainting Under Constrained Camera Motion , 2007, IEEE Transactions on Image Processing.

[40]  William T. Freeman,et al.  What makes a good model of natural images? , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[41]  Michael Elad,et al.  Multiscale Sparse Image Representationwith Learned Dictionaries , 2007, 2007 IEEE International Conference on Image Processing.

[42]  Karen O. Egiazarian,et al.  Color Image Denoising via Sparse 3D Collaborative Filtering with Grouping Constraint in Luminance-Chrominance Space , 2007, 2007 IEEE International Conference on Image Processing.

[43]  Michael Elad,et al.  Sparse Representation for Color Image Restoration , 2008, IEEE Transactions on Image Processing.

[44]  Michael Elad,et al.  Image Sequence Denoising via Sparse and Redundant Representations , 2009, IEEE Transactions on Image Processing.