Cross-SRN: Structure-Preserving Super-Resolution Network With Cross Convolution

It is challenging to restore low-resolution (LR) images to super-resolution (SR) images with correct and clear details. Existing deep learning works almost neglect the inherent structural information of images, which acts as an important role for visual perception of SR results. In this paper, we design a hierarchical feature exploitation network to probe and preserve structural information in a multi-scale feature fusion manner. First, we propose a cross convolution upon traditional edge detectors to localize and represent edge features. Then, cross convolution blocks (CCBs) are designed with feature normalization and channel attention to consider the inherent correlations of features. Finally, we leverage multi-scale feature fusion group (MFFG) to embed the cross convolution blocks and develop the relations of structural features in different scales hierarchically, invoking a lightweight structure-preserving network named as Cross-SRN. Experimental results demonstrate the Cross-SRN achieves competitive or superior restoration performances against the stateof-the-art methods with accurate and clear structural details. Moreover, we set a criterion to select images with rich structural textures. The proposed Cross-SRN outperforms the state-of-theart methods on the selected benchmark, which demonstrates that our network has a significant advantage in preserving edges.

[1]  Kyung-Ah Sohn,et al.  Fast, Accurate, and, Lightweight Super-Resolution with Cascading Residual Network , 2018, ECCV.

[2]  Jian Yang,et al.  MemNet: A Persistent Memory Network for Image Restoration , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[3]  Xiang Bai,et al.  Richer Convolutional Features for Edge Detection , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[4]  Haoqian Wang,et al.  Wide Weighted Attention Multi-Scale Network for Accurate MR Image Super-Resolution , 2022, IEEE Transactions on Circuits and Systems for Video Technology.

[5]  Jian Yang,et al.  Image Super-Resolution via Deep Recursive Residual Network , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[6]  Narendra Ahuja,et al.  Fast and Accurate Image Super-Resolution with Deep Laplacian Pyramid Networks , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[7]  Aline Roumy,et al.  Low-Complexity Single-Image Super-Resolution based on Nonnegative Neighbor Embedding , 2012, BMVC.

[8]  Kyoung Mu Lee,et al.  Accurate Image Super-Resolution Using Very Deep Convolutional Networks , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[9]  Xiaoou Tang,et al.  Image Super-Resolution Using Deep Convolutional Networks , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  Weili Zeng,et al.  Fast Single-Image Super-Resolution via Deep Network With Component Learning , 2019, IEEE Transactions on Circuits and Systems for Video Technology.

[11]  Ming Yang,et al.  BDCN: Bi-Directional Cascade Network for Perceptual Edge Detection , 2020, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[12]  Jitendra Malik,et al.  A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[13]  Thomas S. Huang,et al.  Balanced Two-Stage Residual Networks for Image Super-Resolution , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[14]  Kiyoharu Aizawa,et al.  Sketch-based manga retrieval using manga109 dataset , 2015, Multimedia Tools and Applications.

[15]  Mingkui Tan,et al.  Closed-Loop Matters: Dual Regression Networks for Single Image Super-Resolution , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[16]  Kyoung Mu Lee,et al.  Deeply-Recursive Convolutional Network for Image Super-Resolution , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[17]  Xiaochun Cao,et al.  Correction to: Single Image Super-Resolution via a Holistic Attention Network , 2020, ECCV.

[18]  John F. Canny,et al.  A Computational Approach to Edge Detection , 1986, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[19]  Nariman Farvardin,et al.  A perceptually motivated three-component image model-Part I: description of the model , 1995, IEEE Trans. Image Process..

[20]  Xiaoou Tang,et al.  Accelerating the Super-Resolution Convolutional Neural Network , 2016, ECCV.

[21]  Richard O. Duda,et al.  Pattern classification and scene analysis , 1974, A Wiley-Interscience publication.

[22]  Yun Fu,et al.  Residual Dense Network for Image Restoration , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[23]  Jun Sun,et al.  Robust Single-Image Super-Resolution Based on Adaptive Edge-Preserving Smoothing Regularization , 2018, IEEE Transactions on Image Processing.

[24]  Yao Zhao,et al.  FilterNet: Adaptive Information Filtering Network for Accurate and Fast Image Super-Resolution , 2020, IEEE Transactions on Circuits and Systems for Video Technology.

[25]  Jie Liu,et al.  Residual Feature Distillation Network for Lightweight Image Super-Resolution , 2020, ECCV Workshops.

[26]  Kangfu Mei,et al.  Multi-scale Residual Network for Image Super-Resolution , 2018, ECCV.

[27]  Tong Tong,et al.  Image Super-Resolution Using Dense Skip Connections , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[28]  Tieyong Zeng,et al.  Soft-Edge Assisted Network for Single Image Super-Resolution , 2020, IEEE Transactions on Image Processing.

[29]  Shu-Tao Xia,et al.  Second-Order Attention Network for Single Image Super-Resolution , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[30]  Gaofeng Meng,et al.  Edge-Directed Single-Image Super-Resolution Via Adaptive Gradient Magnitude Self-Interpolation , 2013, IEEE Transactions on Circuits and Systems for Video Technology.

[31]  Kyoung Mu Lee,et al.  Enhanced Deep Residual Networks for Single Image Super-Resolution , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[32]  Yanyun Qu,et al.  LatticeNet: Towards Lightweight Image Super-Resolution with Lattice Block , 2020, ECCV.

[33]  Eirikur Agustsson,et al.  NTIRE 2017 Challenge on Single Image Super-Resolution: Dataset and Study , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[34]  Shuicheng Yan,et al.  Deep Edge Guided Recurrent Residual Learning for Image Super-Resolution , 2016, IEEE Transactions on Image Processing.

[35]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[36]  Michael Elad,et al.  On Single Image Scale-Up Using Sparse-Representations , 2010, Curves and Surfaces.

[37]  Yun Fu,et al.  Image Super-Resolution Using Very Deep Residual Channel Attention Networks , 2018, ECCV.

[38]  Yanpeng Cao,et al.  MRFN: Multi-Receptive-Field Network for Fast and Accurate Single Image Super-Resolution , 2020, IEEE Transactions on Multimedia.

[39]  Jungwon Lee,et al.  Image Super Resolution Based on Fusing Multiple Convolution Neural Networks , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[40]  Peihua Li,et al.  Resolution-Aware Network for Image Super-Resolution , 2019, IEEE Transactions on Circuits and Systems for Video Technology.

[41]  Srimanta Mandal,et al.  Edge preserving single image super resolution in sparse environment , 2013, 2013 IEEE International Conference on Image Processing.

[42]  Enhua Wu,et al.  Squeeze-and-Excitation Networks , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[43]  Wenhan Yang,et al.  Neighborhood regression for edge-preserving image super-resolution , 2015, 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[44]  Jie Zhou,et al.  Structure-Preserving Super Resolution With Gradient Guidance , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[45]  R. Keys Cubic convolution interpolation for digital image processing , 1981 .

[46]  Sergey Ioffe,et al.  Rethinking the Inception Architecture for Computer Vision , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[47]  Narendra Ahuja,et al.  Single image super-resolution from transformed self-exemplars , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[48]  Jae-Seok Choi,et al.  A Deep Convolutional Neural Network with Selection Units for Super-Resolution , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[49]  Kurt Keutzer,et al.  SqueezeNext: Hardware-Aware Neural Network Design , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[50]  Xinbo Gao,et al.  Lightweight Image Super-Resolution with Information Multi-distillation Network , 2019, ACM Multimedia.

[51]  Wen Gao,et al.  Iterative Network for Image Super-Resolution , 2020, ArXiv.

[52]  Xinbo Gao,et al.  Fast and Accurate Single Image Super-Resolution via Information Distillation Network , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[53]  Huizhu Jia,et al.  Digital Retina: A Way to Make the City Brain More Efficient by Visual Coding , 2021, IEEE Transactions on Circuits and Systems for Video Technology.

[54]  Narendra Ahuja,et al.  Deep Laplacian Pyramid Networks for Fast and Accurate Super-Resolution , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[55]  Jeremy R. Cooperstock,et al.  Preconditioning for Edge-Preserving Image Super Resolution , 2012, IEEE Transactions on Image Processing.