ISTA-Inspired Network for Image Super-Resolution

—Deep learning for image super-resolution (SR) has been investigated by numerous researchers in recent years. Most of the works concentrate on effective block designs and improve the network representation but lack interpretation. There are also iterative optimization-inspired networks for image SR, which take the solution step as a whole without giving an explicit optimization step. This paper proposes an unfolding iterative shrinkage thresholding algorithm (ISTA) inspired network for interpretable image SR. Specifically, we analyze the problem of image SR and propose a solution based on the ISTA method. Inspired by the mathematical analysis, the ISTA block is devel- oped to conduct the optimization in an end-to-end manner. To make the exploration more effective, a multi-scale exploitation block and multi-scale attention mechanism are devised to build the ISTA block. Experimental results show the proposed ISTA- inspired restoration network (ISTAR) achieves competitive or better performances than other optimization-inspired works with fewer parameters and lower computation complexity.

[1]  W. Gao,et al.  Sequential Hierarchical Learning with Distribution Transformation for Image Super-Resolution , 2020, ACM Trans. Multim. Comput. Commun. Appl..

[2]  Yu Sang,et al.  Med-SRNet: GAN-Based Medical Image Super-Resolution via High-Resolution Representation Learning , 2022, Computational intelligence and neuroscience.

[3]  Shu-Qi Lei,et al.  Super Resolution Image Visual Quality Assessment Based on Feature Optimization , 2022, Computational intelligence and neuroscience.

[4]  W. Gao,et al.  Hierarchical Similarity Learning for Aliasing Suppression Image Super-Resolution , 2022, IEEE Transactions on Neural Networks and Learning Systems.

[5]  Siwei Ma,et al.  Cross-SRN: Structure-Preserving Super-Resolution Network With Cross Convolution , 2022, IEEE Transactions on Circuits and Systems for Video Technology.

[6]  Shiqi Wang,et al.  Iterative Network for Image Super-Resolution , 2020, IEEE Transactions on Multimedia.

[7]  Jiayi Ma,et al.  Deep Unfolding Network for Spatiospectral Image Super-Resolution , 2022, IEEE Transactions on Computational Imaging.

[8]  Yin Zhang,et al.  Super-Resolution Imaging for Real Aperture Radar by Two-Dimensional Deconvolution , 2021, 2021 IEEE International Geoscience and Remote Sensing Symposium IGARSS.

[9]  Xinfeng Zhang,et al.  Spatial-Temporal Correlation Learning for Real-Time Video Deinterlacing , 2021, 2021 IEEE International Conference on Multimedia and Expo (ICME).

[10]  Yan Wang,et al.  SAR Parametric Super-Resolution Image Reconstruction Methods Based on ADMM and Deep Neural Network , 2021, IEEE Transactions on Geoscience and Remote Sensing.

[11]  Yun Fu,et al.  Residual Dense Network for Image Restoration , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[12]  Jie Liu,et al.  Residual Feature Distillation Network for Lightweight Image Super-Resolution , 2020, ECCV Workshops.

[13]  Xin Wang,et al.  High-Resolution Radar Target Recognition via Inception-Based VGG (IVGG) Networks , 2020, Comput. Intell. Neurosci..

[14]  Wei Gao,et al.  Low-rate Image Compression with Super-resolution Learning , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[15]  Jie Tang,et al.  Residual Feature Aggregation Network for Image Super-Resolution , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[16]  Yao Zhao,et al.  FilterNet: Adaptive Information Filtering Network for Accurate and Fast Image Super-Resolution , 2020, IEEE Transactions on Circuits and Systems for Video Technology.

[17]  Yanpeng Cao,et al.  MRFN: Multi-Receptive-Field Network for Fast and Accurate Single Image Super-Resolution , 2020, IEEE Transactions on Multimedia.

[18]  Luc Van Gool,et al.  Deep Unfolding Network for Image Super-Resolution , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[19]  Tieyong Zeng,et al.  Soft-Edge Assisted Network for Single Image Super-Resolution , 2020, IEEE Transactions on Image Processing.

[20]  Enhua Wu,et al.  Squeeze-and-Excitation Networks , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[21]  Weili Zeng,et al.  Fast Single-Image Super-Resolution via Deep Network With Component Learning , 2019, IEEE Transactions on Circuits and Systems for Video Technology.

[22]  Xinbo Gao,et al.  Lightweight Image Super-Resolution with Information Multi-distillation Network , 2019, ACM Multimedia.

[23]  Peihua Li,et al.  Resolution-Aware Network for Image Super-Resolution , 2019, IEEE Transactions on Circuits and Systems for Video Technology.

[24]  Lei Zhang,et al.  Deep Plug-And-Play Super-Resolution for Arbitrary Blur Kernels , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[25]  Gregory Shakhnarovich,et al.  Deep Back-Projection Networks for Single Image Super-resolution , 2019, ArXiv.

[26]  Kangfu Mei,et al.  Multi-scale Residual Network for Image Super-Resolution , 2018, ECCV.

[27]  Yun Fu,et al.  Image Super-Resolution Using Very Deep Residual Channel Attention Networks , 2018, ECCV.

[28]  Kyung-Ah Sohn,et al.  Fast, Accurate, and, Lightweight Super-Resolution with Cascading Residual Network , 2018, ECCV.

[29]  Tong Tong,et al.  Image Super-Resolution Using Dense Skip Connections , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[30]  Jian Yang,et al.  MemNet: A Persistent Memory Network for Image Restoration , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[31]  Jae-Seok Choi,et al.  A Deep Convolutional Neural Network with Selection Units for Super-Resolution , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[32]  Jungwon Lee,et al.  Image Super Resolution Based on Fusing Multiple Convolution Neural Networks , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[33]  Eirikur Agustsson,et al.  NTIRE 2017 Challenge on Single Image Super-Resolution: Dataset and Study , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[34]  Jian Yang,et al.  Image Super-Resolution via Deep Recursive Residual Network , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[35]  Kyoung Mu Lee,et al.  Enhanced Deep Residual Networks for Single Image Super-Resolution , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[36]  Yun Chen,et al.  Fuzzy Classification of High Resolution Remote Sensing Scenes Using Visual Attention Features , 2017, Comput. Intell. Neurosci..

[37]  Thomas S. Huang,et al.  Balanced Two-Stage Residual Networks for Image Super-Resolution , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[38]  Narendra Ahuja,et al.  Deep Laplacian Pyramid Networks for Fast and Accurate Super-Resolution , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[39]  Wangmeng Zuo,et al.  Learning Deep CNN Denoiser Prior for Image Restoration , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[40]  Kilian Q. Weinberger,et al.  Densely Connected Convolutional Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[41]  Stanley H. Chan,et al.  Plug-and-Play ADMM for Image Restoration: Fixed-Point Convergence and Applications , 2016, IEEE Transactions on Computational Imaging.

[42]  Shuicheng Yan,et al.  Deep Edge Guided Recurrent Residual Learning for Image Super-Resolution , 2016, IEEE Transactions on Image Processing.

[43]  Jian Sun,et al.  Deep ADMM-Net for Compressive Sensing MRI , 2016, NIPS.

[44]  Xiaoou Tang,et al.  Accelerating the Super-Resolution Convolutional Neural Network , 2016, ECCV.

[45]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[46]  Kyoung Mu Lee,et al.  Accurate Image Super-Resolution Using Very Deep Convolutional Networks , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[47]  Kyoung Mu Lee,et al.  Deeply-Recursive Convolutional Network for Image Super-Resolution , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[48]  Kiyoharu Aizawa,et al.  Sketch-based manga retrieval using manga109 dataset , 2015, Multimedia Tools and Applications.

[49]  Xiaoou Tang,et al.  Image Super-Resolution Using Deep Convolutional Networks , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[50]  Narendra Ahuja,et al.  Single image super-resolution from transformed self-exemplars , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[51]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[52]  Aline Roumy,et al.  Low-Complexity Single-Image Super-Resolution based on Nonnegative Neighbor Embedding , 2012, BMVC.

[53]  Djemel Ziou,et al.  Image Quality Metrics: PSNR vs. SSIM , 2010, 2010 20th International Conference on Pattern Recognition.

[54]  Michael Elad,et al.  On Single Image Scale-Up Using Sparse-Representations , 2010, Curves and Surfaces.

[55]  Pavel Sovka,et al.  High-Resolution Movement EEG Classification , 2007, Comput. Intell. Neurosci..

[56]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[57]  Jitendra Malik,et al.  A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.