Convolutional neural networks for wavelet domain super resolution

Proposed a super resolution method with higher reconstruction accuracy than before.Cast super resolution as a problem of estimating sparse wavelet detail coefficients.Estimated sparse wavelet coefficients using a convolutional neural network (CNN).Trained our CNN with fewer samples than the one used for estimating pixels directly.Concluded that it is better to estimate wavelet coefficients rather than pixels. We present a single image super resolution technique in which we estimate wavelet detail coefficients of a desired high resolution (HR) image using a convolutional neural network (CNN) on the given low resolution (LR) image. Detail coefficients are necessarily sparse for natural images, unlike pixel intensities, and are thus better suited to be CNN output. This allows us to train a CNN with far fewer samples and lesser training time and yet achieve better reconstruction quality with lesser run time compared to a recent state-of-the-art technique that directly estimates the HR pixels using a CNN.

[1]  Michal Irani,et al.  Super-resolution from a single image , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[2]  Amit Sethi,et al.  A spatial neighbourhood based learning setup for super resolution , 2012, 2012 Annual IEEE India Conference (INDICON).

[3]  Quan Pan,et al.  Semi-coupled dictionary learning with applications to image super-resolution and photo-sketch synthesis , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[4]  Neeraj Kumar,et al.  Learning to predict super resolution wavelet coefficients , 2012, Proceedings of the 21st International Conference on Pattern Recognition (ICPR2012).

[5]  Michael Elad,et al.  On Single Image Scale-Up Using Sparse-Representations , 2010, Curves and Surfaces.

[6]  Aline Roumy,et al.  Low-Complexity Single-Image Super-Resolution based on Nonnegative Neighbor Embedding , 2012, BMVC.

[7]  Stphane Mallat,et al.  A Wavelet Tour of Signal Processing, Third Edition: The Sparse Way , 2008 .

[8]  William T. Freeman,et al.  Example-Based Super-Resolution , 2002, IEEE Computer Graphics and Applications.

[9]  Cataldo Guaragnella,et al.  Edge enhancement for subband coded images , 2001 .

[10]  Justin K. Romberg,et al.  Bayesian tree-structured image modeling using wavelet-domain hidden Markov models , 2001, IEEE Trans. Image Process..

[11]  Kwang In Kim,et al.  Single-Image Super-Resolution Using Sparse Regression and Natural Image Prior , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[12]  Raymond H. Chan,et al.  Wavelet Algorithms for High-Resolution Image Reconstruction , 2002, SIAM J. Sci. Comput..

[13]  Jing Tian,et al.  Wavelet-Based Image Interpolation Using a Three-Component Exponential Mixture Model , 2008, 2008 Congress on Image and Signal Processing.

[14]  Stéphane Mallat,et al.  Super-Resolution With Sparse Mixing Estimators , 2010, IEEE Transactions on Image Processing.

[15]  Yuan Xie,et al.  Image Super-Resolution Using Deep Belief Networks , 2014, ICIMCS '14.

[16]  Michal Irani,et al.  Internal statistics of a single natural image , 2011, CVPR 2011.

[17]  Chih-Yuan Yang,et al.  Learning a No-Reference Quality Metric for Single-Image Super-Resolution , 2016, Comput. Vis. Image Underst..

[18]  Zhou Wang,et al.  Image Quality Assessment: From Error Measurement to Structural Similarity , 2004 .

[19]  Thomas S. Huang,et al.  Image Super-Resolution Via Sparse Representation , 2010, IEEE Transactions on Image Processing.

[20]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[21]  Rob Fergus,et al.  Visualizing and Understanding Convolutional Networks , 2013, ECCV.

[22]  Mei Han,et al.  SoftCuts: A Soft Edge Smoothness Prior for Color Image Super-Resolution , 2009, IEEE Transactions on Image Processing.

[23]  Takeo Kanade,et al.  Limits on super-resolution and how to break them , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[24]  Robert D. Nowak,et al.  Wavelet-based statistical signal processing using hidden Markov models , 1998, IEEE Trans. Signal Process..

[25]  Thomas W. Parks,et al.  Image interpolation using wavelet based hidden Markov trees , 2001, 2001 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings (Cat. No.01CH37221).

[26]  Xiaoou Tang,et al.  Image Super-Resolution Using Deep Convolutional Networks , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[27]  Il Kyu Eom,et al.  Image Interpolation Based on Statistical Relationship Between Wavelet Subbands , 2007, 2007 IEEE International Conference on Multimedia and Expo.

[28]  Michael Elad,et al.  A Statistical Prediction Model Based on Sparse Representations for Single Image Super-Resolution , 2014, IEEE Transactions on Image Processing.

[29]  Charless C. Fowlkes,et al.  Contour Detection and Hierarchical Image Segmentation , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[30]  Stéphane Mallat,et al.  A Wavelet Tour of Signal Processing - The Sparse Way, 3rd Edition , 2008 .

[31]  Luc Van Gool,et al.  A+: Adjusted Anchored Neighborhood Regression for Fast Super-Resolution , 2014, ACCV.