Color Image Super-Resolution and Enhancement with Inter-Channel Details at Trivial Cost

Image super-resolution is essential for a variety of applications such as medical imaging, surveillance imaging, and satellite imaging, among others. Traditionally, the most popular color image super-resolution is performed in each color channel independently. In this paper, we show that the super-resolution quality can be further enhanced by exploiting the cross-channel correlation. Inspired by the High-Quality Linear Interpolation (HQLI) demosaicking algorithm by Malvar et al., we design an image super-resolution scheme that integrates intra-channel interpolation with cross-channel details by isotropic linear combinations. Despite its simplicity, our super-resolution method achieves the accuracy comparable with the existing fastest state-of-the-art super-resolution algorithm at 20 times faster speed. It is well applicable to applications that adopt traditional interpolations, for improved visual quality at trivial computation cost. Our comparative study verifies the effectiveness and efficiency of the proposed super-resolution algorithm.

[1]  Christian Ledig,et al.  Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[2]  Kwang In Kim,et al.  Single-Image Super-Resolution Using Sparse Regression and Natural Image Prior , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[3]  Michal Irani,et al.  Super-resolution from a single image , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[4]  Mingli Zhang,et al.  Image Smoothing Based on Image Decomposition and Sparse High Frequency Gradient , 2018, Journal of Computer Science and Technology.

[5]  Pong C. Yuen,et al.  Very low resolution face recognition problem , 2010, 2010 Fourth IEEE International Conference on Biometrics: Theory, Applications and Systems (BTAS).

[6]  Heng Lian,et al.  Variational Local Structure Estimation for Image Super-Resolution , 2006, 2006 International Conference on Image Processing.

[7]  Richard Szeliski,et al.  Computer Vision - Algorithms and Applications , 2011, Texts in Computer Science.

[8]  Thomas S. Huang,et al.  Coupled Dictionary Training for Image Super-Resolution , 2012, IEEE Transactions on Image Processing.

[9]  Luc Van Gool,et al.  Anchored Neighborhood Regression for Fast Example-Based Super-Resolution , 2013, 2013 IEEE International Conference on Computer Vision.

[10]  Jitendra Malik,et al.  A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[11]  Jihong Ouyang,et al.  Low Cost Edge Sensing for High Quality Demosaicking , 2018, IEEE Transactions on Image Processing.

[12]  Narendra Ahuja,et al.  Single image super-resolution from transformed self-exemplars , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[13]  Bryan S. Morse,et al.  Image magnification using level-set reconstruction , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[14]  William T. Freeman,et al.  Example-Based Super-Resolution , 2002, IEEE Computer Graphics and Applications.

[15]  Vishal Monga,et al.  Sparsity-Based Color Image Super Resolution via Exploiting Cross Channel Constraints , 2016, IEEE Transactions on Image Processing.

[16]  Thomas S. Huang,et al.  Image Super-Resolution Via Sparse Representation , 2010, IEEE Transactions on Image Processing.

[17]  Kyoung Mu Lee,et al.  Deeply-Recursive Convolutional Network for Image Super-Resolution , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[18]  Xiaoou Tang,et al.  Learning a Deep Convolutional Network for Image Super-Resolution , 2014, ECCV.

[19]  Caiming Zhang,et al.  Single-Image Super-Resolution Based on Rational Fractal Interpolation , 2018, IEEE Transactions on Image Processing.

[20]  Philip J. Willis,et al.  Image Interpolation by Pixel‐Level Data‐Dependent Triangulation , 2004, Comput. Graph. Forum.

[21]  Shuai Liu,et al.  Adaptive deep residual network for single image super-resolution , 2019, Computational Visual Media.

[22]  Henrique S. Malvar,et al.  High-quality linear interpolation for demosaicing of Bayer-patterned color images , 2004, 2004 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[23]  Gregory Shakhnarovich,et al.  Deep Back-Projection Networks for Super-Resolution , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[24]  Kyoung Mu Lee,et al.  Enhanced Deep Residual Networks for Single Image Super-Resolution , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[25]  Peter M. Atkinson,et al.  Sub‐pixel mapping of rural land cover objects from fine spatial resolution satellite sensor imagery using super‐resolution pixel‐swapping , 2006 .

[26]  Jan Flusser,et al.  Image registration methods: a survey , 2003, Image Vis. Comput..

[27]  Ingmar Lissner,et al.  Image-Difference Prediction: From Grayscale to Color , 2013, IEEE Transactions on Image Processing.

[28]  Luc Van Gool,et al.  NTIRE 2017 Challenge on Single Image Super-Resolution: Methods and Results , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[29]  Sanjit K. Mitra,et al.  EDGE-ENHANCED IMAGE ZOOMING , 1996 .

[30]  Nicola Asuni,et al.  Submitted to Ieee Transactions on Image Processing 1 Real Time Artifact-free Image Upscaling , 2022 .

[31]  Michael Elad,et al.  On Single Image Scale-Up Using Sparse-Representations , 2010, Curves and Surfaces.

[32]  Mei Han,et al.  SoftCuts: A Soft Edge Smoothness Prior for Color Image Super-Resolution , 2009, IEEE Transactions on Image Processing.

[33]  Eirikur Agustsson,et al.  NTIRE 2017 Challenge on Single Image Super-Resolution: Dataset and Study , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[34]  K. Mardia,et al.  A review of image-warping methods , 1998 .

[35]  Na Ding,et al.  Single Image Super-Resolution via Dynamic Lightweight Database with Local-Feature Based Interpolation , 2019, Journal of Computer Science and Technology.

[36]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[37]  Aline Roumy,et al.  Low-Complexity Single-Image Super-Resolution based on Nonnegative Neighbor Embedding , 2012, BMVC.

[38]  Hayit Greenspan,et al.  Super-Resolution in Medical Imaging , 2009, Comput. J..