Texture enhancement for improving single-image super-resolution performance

Although various single-image super-resolution algorithms have been developed to increase image resolution, they still do not provide adequate performance in the texture region due to the lack of fine textures in the processed image. In this paper, we present a novel texture enhancement strategy in order to improve the super-resolution performance in the texture region. For texture enhancement, we extract a low-resolution texture layer from an input image and generate a high-resolution texture layer by applying the proposed texture synthesis algorithm. A texture enhanced high-resolution image is then obtained by properly combining the generated high-resolution texture layer with an image obtained by using an existing single-image super-resolution algorithm. Experimental results show that the proposed texture enhancement strategy provides sharper and more natural looking textures compared with the existing super-resolution algorithms. This paper addresses a key remaining issue, texture SR, which is yet unresolved.Patch-based texture synthesis is performed by using a LR texture layer.A HR texture patch is synthesized based on both AR and reconstruction models.The proposed algorithm provides realistic SR images with sharp and fine textures.

[1]  Chris Damkat,et al.  Single image super-resolution using self-examples and texture synthesis , 2011, Signal Image Video Process..

[2]  William T. Freeman,et al.  Example-Based Super-Resolution , 2002, IEEE Computer Graphics and Applications.

[3]  Xuelong Li,et al.  Single Image Super-Resolution With Non-Local Means and Steering Kernel Regression , 2012, IEEE Transactions on Image Processing.

[4]  Christine Guillemot,et al.  Single image super-resolution using sparse representations with structure constraints , 2014, 2014 IEEE International Conference on Image Processing (ICIP).

[5]  Yehoshua Y. Zeevi,et al.  Single-Image Superresolution of Natural Stochastic Textures Based on Fractional Brownian Motion , 2014, IEEE Transactions on Image Processing.

[6]  Thomas S. Huang,et al.  Image Super-Resolution Via Sparse Representation , 2010, IEEE Transactions on Image Processing.

[7]  Lei Zhang,et al.  Sparse Representation Based Image Interpolation With Nonlocal Autoregressive Modeling , 2013, IEEE Transactions on Image Processing.

[8]  Thomas B. Moeslund,et al.  Super-resolution: a comprehensive survey , 2014, Machine Vision and Applications.

[9]  Yehoshua Y. Zeevi,et al.  Single-image superresolution of self-similar textures , 2013, 2013 IEEE International Conference on Image Processing.

[10]  Chih-Yuan Yang,et al.  Fast Direct Super-Resolution by Simple Functions , 2013, 2013 IEEE International Conference on Computer Vision.

[11]  Guangtao Zhai,et al.  Single Image Super-resolution With Detail Enhancement Based on Local Fractal Analysis of Gradient , 2013, IEEE Transactions on Circuits and Systems for Video Technology.

[12]  Chih-Yuan Yang,et al.  Single-Image Super-Resolution: A Benchmark , 2014, ECCV.

[13]  Kwang In Kim,et al.  Single-Image Super-Resolution Using Sparse Regression and Natural Image Prior , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[14]  Michal Irani,et al.  Super-resolution from a single image , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[15]  Lei Zhang,et al.  Image Deblurring and Super-Resolution by Adaptive Sparse Domain Selection and Adaptive Regularization , 2010, IEEE Transactions on Image Processing.

[16]  Yi Sun,et al.  Context-aware single image super-resolution using sparse representation and cross-scale similarity , 2015, Signal Process. Image Commun..

[17]  Raanan Fattal,et al.  Image upsampling via texture hallucination , 2010, 2010 IEEE International Conference on Computational Photography (ICCP).

[18]  Wan-Chi Siu,et al.  Single image super-resolution using Gaussian process regression , 2011, CVPR 2011.

[19]  D. Yeung,et al.  Super-resolution through neighbor embedding , 2004, CVPR 2004.

[20]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[21]  Michael Elad,et al.  A Statistical Prediction Model Based on Sparse Representations for Single Image Super-Resolution , 2014, IEEE Transactions on Image Processing.

[22]  Moshe Porat,et al.  On texture and image interpolation using Markov models , 2009, Signal Process. Image Commun..

[23]  Li Xu,et al.  Structure extraction from texture via relative total variation , 2012, ACM Trans. Graph..

[24]  Raanan Fattal,et al.  Image upsampling via imposed edge statistics , 2007, ACM Trans. Graph..

[25]  Xuelong Li,et al.  Image Super-Resolution With Sparse Neighbor Embedding , 2012, IEEE Transactions on Image Processing.

[26]  Luc Van Gool,et al.  A+: Adjusted Anchored Neighborhood Regression for Fast Super-Resolution , 2014, ACCV.

[27]  Jiejie Zhu,et al.  Context-constrained hallucination for image super-resolution , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[28]  Aline Roumy,et al.  Neighbor embedding based single-image super-resolution using Semi-Nonnegative Matrix Factorization , 2012, 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[29]  Xiangjun Zhang,et al.  Image Interpolation by Adaptive 2-D Autoregressive Modeling and Soft-Decision Estimation , 2008, IEEE Transactions on Image Processing.

[30]  Licheng Yu,et al.  Self-example based super-resolution with fractal-based gradient enhancement , 2013, 2013 IEEE International Conference on Multimedia and Expo Workshops (ICMEW).

[31]  Xiaoou Tang,et al.  Learning a Deep Convolutional Network for Image Super-Resolution , 2014, ECCV.

[32]  Alessandro Foi,et al.  Image Denoising by Sparse 3-D Transform-Domain Collaborative Filtering , 2007, IEEE Transactions on Image Processing.

[33]  Harry Shum,et al.  Gradient Profile Prior and Its Applications in Image Super-Resolution and Enhancement , 2011, IEEE Transactions on Image Processing.

[34]  Jong Beom Ra,et al.  Example-Based Super-Resolution via Structure Analysis of Patches , 2013, IEEE Signal Processing Letters.

[35]  Chun Qi,et al.  Low-Rank Neighbor Embedding for Single Image Super-Resolution , 2014, IEEE Signal Processing Letters.

[36]  Raanan Fattal,et al.  Image and video upscaling from local self-examples , 2011, TOGS.

[37]  Chi-Keung Tang,et al.  Fast image/video upsampling , 2008, SIGGRAPH 2008.

[38]  Chinh T. Dang,et al.  Image Super-Resolution via Local Self-Learning Manifold Approximation , 2014, IEEE Signal Processing Letters.

[39]  Aline Roumy,et al.  Single-Image Super-Resolution via Linear Mapping of Interpolated Self-Examples , 2014, IEEE Transactions on Image Processing.

[40]  Stephen Lin,et al.  Super resolution using edge prior and single image detail synthesis , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[41]  Chih-Yuan Yang,et al.  Exploiting Self-similarities for Single Frame Super-Resolution , 2010, ACCV.