Joint Back Projection and Residual Networks for Efficient Image Super-Resolution

Benefiting from the great power of graphic processor units, researchers can now come up with more and more sophisticated and complex deep learning structures to solve computer vision problems in various fields with excellent results. However, real-time performance is the bottleneck for deep learning in some applications, like image super-resolution. In this paper, we propose an image super-resolution making use of both the advantages of Back Projection and Residual Networks (BPRN). It generalizes the residual networks as a hierarchical back projection process. We use both convolution and deconvolution to down- and up-sample images to feedback the residues for super-resolution. Furthermore, we come up with a Lighter BPRN (L-BPRN) model to achieve similar state-of-the-art PSNR but fewer network parameters. The testing process is much faster and also accurate for image super-resolution with different scaling factors. Compared with recent deep learning based image super-resolution approaches, experimental results show that our proposed methods can achieve the state-of-the-art PSNR and SSIM performance as well as fast realization.

[1]  Charless C. Fowlkes,et al.  Contour Detection and Hierarchical Image Segmentation , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[2]  Jian Yang,et al.  Image Super-Resolution via Deep Recursive Residual Network , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[3]  Michal Irani,et al.  Improving resolution by image registration , 1991, CVGIP Graph. Model. Image Process..

[4]  Luc Van Gool,et al.  A+: Adjusted Anchored Neighborhood Regression for Fast Super-Resolution , 2014, ACCV.

[5]  Christian Ledig,et al.  Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[6]  Sunghyun Cho,et al.  Good Image Priors for Non-blind Deconvolution - Generic vs. Specific , 2014, ECCV.

[7]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[8]  Daniel Rueckert,et al.  Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[9]  Yui-Lam Chan,et al.  Fast image super-resolution via Randomized Multi-split Forests , 2017, 2017 IEEE International Symposium on Circuits and Systems (ISCAS).

[10]  Thomas S. Huang,et al.  Image Super-Resolution Via Sparse Representation , 2010, IEEE Transactions on Image Processing.

[11]  Xiaoou Tang,et al.  Accelerating the Super-Resolution Convolutional Neural Network , 2016, ECCV.

[12]  Wan-Chi Siu,et al.  Learning Hierarchical Decision Trees for Single-Image Super-Resolution , 2017, IEEE Transactions on Circuits and Systems for Video Technology.

[13]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[14]  Kyoung Mu Lee,et al.  Accurate Image Super-Resolution Using Very Deep Convolutional Networks , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[15]  Kyoung Mu Lee,et al.  Deeply-Recursive Convolutional Network for Image Super-Resolution , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[16]  Michal Irani,et al.  Motion Analysis for Image Enhancement: Resolution, Occlusion, and Transparency , 1993, J. Vis. Commun. Image Represent..

[17]  Jian Sun,et al.  Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[18]  Kyoung Mu Lee,et al.  Enhanced Deep Residual Networks for Single Image Super-Resolution , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[19]  Luc Van Gool,et al.  NTIRE 2017 Challenge on Single Image Super-Resolution: Methods and Results , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[20]  Wan-Chi Siu,et al.  Single image super-resolution using Gaussian process regression , 2011, CVPR 2011.

[21]  Wan-Chi Siu,et al.  Cascaded Random Forests for Fast Image Super-Resolution , 2018, 2018 25th IEEE International Conference on Image Processing (ICIP).

[22]  Luc Van Gool,et al.  Anchored Neighborhood Regression for Fast Example-Based Super-Resolution , 2013, 2013 IEEE International Conference on Computer Vision.

[23]  Gregory Shakhnarovich,et al.  Deep Back-Projection Networks for Super-Resolution , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[24]  Xiaoou Tang,et al.  Image Super-Resolution Using Deep Convolutional Networks , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[25]  Wan-Chi Siu,et al.  Image super-resolution via weighted random forest , 2017, 2017 IEEE International Conference on Industrial Technology (ICIT).

[26]  Narendra Ahuja,et al.  Deep Laplacian Pyramid Networks for Fast and Accurate Super-Resolution , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[27]  Narendra Ahuja,et al.  Single image super-resolution from transformed self-exemplars , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).