High Resolution Local Structure-Constrained Image Upsampling

With the development of ultra-high-resolution display devices, the visual perception of fine texture details is becoming more and more important. A method of high-quality image upsampling with a low cost is greatly needed. In this paper, we propose a fast and efficient image upsampling method that makes use of high-resolution local structure constraints. The average local difference is used to divide a bicubic-interpolated image into a sharp edge area and a texture area, and these two areas are reconstructed separately with specific constraints. For reconstruction of the sharp edge area, a high-resolution gradient map is estimated as an extra constraint for the recovery of sharp and natural edges; for the reconstruction of the texture area, a high-resolution local texture structure map is estimated as an extra constraint to recover fine texture details. These two reconstructed areas are then combined to obtain the final high-resolution image. The experimental results demonstrated that the proposed method recovered finer pixel-level texture details and obtained top-level objective performance with a low time cost compared with state-of-the-art methods.

[1]  Michal Irani,et al.  Motion Analysis for Image Enhancement: Resolution, Occlusion, and Transparency , 1993, J. Vis. Commun. Image Represent..

[2]  D C Van Essen,et al.  Information processing in the primate visual system: an integrated systems perspective. , 1992, Science.

[3]  Chunhong Pan,et al.  Fast Image Upsampling via the Displacement Field , 2014, IEEE Transactions on Image Processing.

[4]  Michal Irani,et al.  Super-resolution from a single image , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[5]  Matti Pietikäinen,et al.  Multiresolution Gray-Scale and Rotation Invariant Texture Classification with Local Binary Patterns , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[6]  Xuelong Li,et al.  Single Image Super-Resolution With Non-Local Means and Steering Kernel Regression , 2012, IEEE Transactions on Image Processing.

[7]  Michael Elad,et al.  On Single Image Scale-Up Using Sparse-Representations , 2010, Curves and Surfaces.

[8]  Michael Elad,et al.  A Statistical Prediction Model Based on Sparse Representations for Single Image Super-Resolution , 2014, IEEE Transactions on Image Processing.

[9]  Guangtao Zhai,et al.  Single Image Super-resolution With Detail Enhancement Based on Local Fractal Analysis of Gradient , 2013, IEEE Transactions on Circuits and Systems for Video Technology.

[10]  Patrick Pérez,et al.  Reconstructing an image from its local descriptors , 2011, CVPR 2011.

[11]  William T. Freeman,et al.  Learning low-level vision , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[12]  Nanning Zheng,et al.  Image hallucination with primal sketch priors , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[13]  Anat Levin,et al.  Accurate Blur Models vs. Image Priors in Single Image Super-resolution , 2013, 2013 IEEE International Conference on Computer Vision.

[14]  Luc Van Gool,et al.  Anchored Neighborhood Regression for Fast Example-Based Super-Resolution , 2013, 2013 IEEE International Conference on Computer Vision.

[15]  Michal Irani,et al.  Nonparametric Blind Super-resolution , 2013, 2013 IEEE International Conference on Computer Vision.

[16]  Alexandre Alahi,et al.  From Bits to Images: Inversion of Local Binary Descriptors , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[17]  Xiaoyang Tan,et al.  Enhanced Local Texture Feature Sets for Face Recognition Under Difficult Lighting Conditions , 2007, IEEE Transactions on Image Processing.

[18]  Cao Ming-min Super Resolution Through Neighbor Embedding with Adaptive Neighbor Selection , 2015 .

[19]  Raanan Fattal,et al.  Image upsampling via imposed edge statistics , 2007, ACM Trans. Graph..

[20]  Lei Zhang,et al.  Image Deblurring and Super-Resolution by Adaptive Sparse Domain Selection and Adaptive Regularization , 2010, IEEE Transactions on Image Processing.

[21]  M. Tech,et al.  Real Time Artifact-Free Image Upscaling , 2012 .

[22]  Rabab Kreidieh Ward,et al.  Contrast enhancement for enlarged images based on edge sharpening , 2005, IEEE International Conference on Image Processing 2005.

[23]  Thomas S. Huang,et al.  Bilevel sparse coding for coupled feature spaces , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[24]  Rabab Kreidieh Ward,et al.  A New Orientation-Adaptive Interpolation Method , 2007, IEEE Transactions on Image Processing.

[25]  Jitendra Malik,et al.  A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[26]  Mei Han,et al.  Soft Edge Smoothness Prior for Alpha Channel Super Resolution , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[27]  Yong Haur Tay,et al.  Enhanced Image Super-Resolution Technique Using Convolutional Neural Network , 2013, IVIC.

[28]  Xuelong Li,et al.  Learning Multiple Linear Mappings for Efficient Single Image Super-Resolution , 2015, IEEE Transactions on Image Processing.

[29]  Thomas Martin Deserno,et al.  Survey: interpolation methods in medical image processing , 1999, IEEE Transactions on Medical Imaging.

[30]  Philip J. Willis,et al.  Image Interpolation by Pixel‐Level Data‐Dependent Triangulation , 2004, Comput. Graph. Forum.

[31]  Sung-Jea Ko,et al.  Edge-oriented two-step interpolation based on training set , 2010, IEEE Transactions on Consumer Electronics.

[32]  Zhenhua Guo,et al.  A Completed Modeling of Local Binary Pattern Operator for Texture Classification , 2010, IEEE Transactions on Image Processing.

[33]  R. Keys Cubic convolution interpolation for digital image processing , 1981 .

[34]  Chi-Keung Tang,et al.  Fast image/video upsampling , 2008, SIGGRAPH 2008.

[35]  Xuelong Li,et al.  Single Image Super-Resolution With Multiscale Similarity Learning , 2013, IEEE Transactions on Neural Networks and Learning Systems.

[36]  Wen Gao,et al.  Image interpolation via regularized local linear regression , 2011, 28th Picture Coding Symposium.

[37]  Kin-Man Lam,et al.  Face hallucination based on sparse local-pixel structure , 2014, Pattern Recognit..

[38]  Raanan Fattal,et al.  Image upsampling via texture hallucination , 2010, 2010 IEEE International Conference on Computational Photography (ICCP).

[39]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[40]  Gaofeng Meng,et al.  Edge-Directed Single-Image Super-Resolution Via Adaptive Gradient Magnitude Self-Interpolation , 2013, IEEE Transactions on Circuits and Systems for Video Technology.

[41]  Lei Zhang,et al.  Nonlocally Centralized Sparse Representation for Image Restoration , 2013, IEEE Transactions on Image Processing.

[42]  Thomas S. Huang,et al.  Image Super-Resolution Via Sparse Representation , 2010, IEEE Transactions on Image Processing.

[43]  Xuelong Li,et al.  A Comprehensive Survey to Face Hallucination , 2013, International Journal of Computer Vision.

[44]  Russell Zaretzki,et al.  Beta Process Joint Dictionary Learning for Coupled Feature Spaces with Application to Single Image Super-Resolution , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[45]  David H. Frakes,et al.  Segment Adaptive Gradient Angle Interpolation , 2013, IEEE Transactions on Image Processing.

[46]  Raanan Fattal,et al.  Image and video upscaling from local self-examples , 2011, TOGS.

[47]  Huifang Sun,et al.  Image interpolation via graph-based Bayesian label propagation. , 2014, IEEE transactions on image processing : a publication of the IEEE Signal Processing Society.