Edge and color preserving single image superresolution

Abstract. Most existing superresolution (SR) techniques focus primarily on improving the quality in the luminance component of SR images, while paying less attention to the chrominance component. We present an edge and color preserving image SR approach. First, for the luminance channel, a heavy-tailed gradient distribution of natural images is investigated as an image prior. Then, an efficient optimization algorithm is developed to recover the latent high-resolution (HR) luminance component. Second, for the chrominance channels, we propose a two-stage framework for luminance-guided chrominance SR. In the first stage, since most of the shape and structural information is contained in the luminance channel, a simple Markov random field formulation is introduced to search the optimal direction for color local interpolation guided by HR luminance components. To further improve the quality of the chrominance channels, in the second stage, a nonlocal auto regression model is utilized to refine the initial HR chrominance. Finally, we combine the SR reconstructed luminance components with the generated HR chrominance maps to get the final SR color image. Systematic experimental results demonstrated that our method outperforms some state-of-the-art methods in terms of the peak signal-to-noise ratio, structural similarity, feature similarity, and the mean color errors.

[1]  Xiangjun Zhang,et al.  Image Interpolation by Adaptive 2-D Autoregressive Modeling and Soft-Decision Estimation , 2008, IEEE Transactions on Image Processing.

[2]  Dani Lischinski,et al.  Colorization using optimization , 2004, ACM Trans. Graph..

[3]  Xuelong Li,et al.  Single Image Super-Resolution With Non-Local Means and Steering Kernel Regression , 2012, IEEE Transactions on Image Processing.

[4]  Frédo Durand,et al.  Image and depth from a conventional camera with a coded aperture , 2007, SIGGRAPH 2007.

[5]  Thomas S. Huang,et al.  Collaborative and compressive high-resolution imaging , 2012, Proceedings of the 21st International Conference on Pattern Recognition (ICPR2012).

[6]  Mark D. Fairchild,et al.  A top down description of S-CIELAB and CIEDE2000 , 2003 .

[7]  Robert L. Stevenson,et al.  Extraction of high-resolution frames from video sequences , 1996, IEEE Trans. Image Process..

[8]  Guillermo Sapiro,et al.  On the level lines and geometry of vector-valued images , 2000, IEEE Signal Processing Letters.

[9]  David Zhang,et al.  FSIM: A Feature Similarity Index for Image Quality Assessment , 2011, IEEE Transactions on Image Processing.

[10]  Hsieh Hou,et al.  Cubic splines for image interpolation and digital filtering , 1978 .

[11]  William T. Freeman,et al.  Comparison of graph cuts with belief propagation for stereo, using identical MRF parameters , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[12]  Mei Han,et al.  Soft Edge Smoothness Prior for Alpha Channel Super Resolution , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[13]  Moon Gi Kang,et al.  Super-resolution image reconstruction: a technical overview , 2003, IEEE Signal Process. Mag..

[14]  Junfeng Yang,et al.  A New Alternating Minimization Algorithm for Total Variation Image Reconstruction , 2008, SIAM J. Imaging Sci..

[15]  Rob Fergus,et al.  Fast Image Deconvolution using Hyper-Laplacian Priors , 2009, NIPS.

[16]  Jean-Michel Morel,et al.  A non-local algorithm for image denoising , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[17]  Lei Zhang,et al.  Image Deblurring and Super-Resolution by Adaptive Sparse Domain Selection and Adaptive Regularization , 2010, IEEE Transactions on Image Processing.

[18]  Guillermo Sapiro,et al.  Inpainting the colors , 2005, IEEE International Conference on Image Processing 2005.

[19]  Thomas S. Huang,et al.  Image super-resolution as sparse representation of raw image patches , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[20]  Rob Fergus,et al.  Blind deconvolution using a normalized sparsity measure , 2011, CVPR 2011.

[21]  William T. Freeman,et al.  Example-Based Super-Resolution , 2002, IEEE Computer Graphics and Applications.

[22]  Truong Q. Nguyen,et al.  Markov Random Field Model-Based Edge-Directed Image Interpolation , 2007, IEEE Transactions on Image Processing.

[23]  Wencheng Wu,et al.  The CIEDE2000 color-difference formula: Implementation notes, supplementary test data, and mathematical observations , 2005 .

[24]  Kwang In Kim,et al.  Example-Based Learning for Single-Image Super-Resolution , 2008, DAGM-Symposium.

[25]  Thomas S. Huang,et al.  Image Super-Resolution Via Sparse Representation , 2010, IEEE Transactions on Image Processing.

[26]  Thomas S. Huang,et al.  Close the loop: Joint blind image restoration and recognition with sparse representation prior , 2011, 2011 International Conference on Computer Vision.

[27]  Jean-Michel Morel,et al.  Geometry and Color in Natural Images , 2002, Journal of Mathematical Imaging and Vision.

[28]  Alessandro Foi,et al.  Image Denoising by Sparse 3-D Transform-Domain Collaborative Filtering , 2007, IEEE Transactions on Image Processing.

[29]  Guillermo Sapiro,et al.  Non-local sparse models for image restoration , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[30]  Michael Elad,et al.  Generalizing the Nonlocal-Means to Super-Resolution Reconstruction , 2009, IEEE Transactions on Image Processing.

[31]  Nadine Gottschalk Colorization For Single Image Super Resolution , 2016 .

[32]  Marc Teboulle,et al.  A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems , 2009, SIAM J. Imaging Sci..

[33]  Chi-Keung Tang,et al.  Fast image/video upsampling , 2008, SIGGRAPH 2008.

[34]  Ron Kimmel,et al.  A Natural Norm for Color Processing , 1998, ACCV.

[35]  Peyman Milanfar,et al.  Kernel Regression for Image Processing and Reconstruction , 2007, IEEE Transactions on Image Processing.

[36]  L. Rudin,et al.  Nonlinear total variation based noise removal algorithms , 1992 .