Super-resolution-based GAN for image processing: Recent advances and future trends

Abstract Humans can analyze the relationship of data collected from different domains without any supervision, but automatic learning of discovering data is a challenging task. Image processing is one of the examples where conversion from day to night and night to day is difficult. To avoid drawbacks in image processing, Generative adversarial network (GAN) came into existence, which helps to generate networks without losing properties of attributes such as face identity and orientation. Again, super-resolution generative adversarial network (SRGAN) is a formative task that is capable of producing realistic textures in single image super-resolution. Still, hallucinated descriptions are generally attained with no desirable artifacts. Furthermore, to improve visual quality, we significantly studied the prime components of SRGAN—architecture, perceptual loss, adversarial loss, and training algorithm. Moreover, various potential open challenges, along with the future scope, are also discussed.

[1]  Wei Xu,et al.  EESR: Edge Enhanced Super-Resolution , 2018, 2018 14th IEEE International Conference on Solid-State and Integrated Circuit Technology (ICSICT).

[2]  Yi Sun,et al.  FG-SRGAN: A Feature-Guided Super-Resolution Generative Adversarial Network for Unpaired Image Super-Resolution , 2019, ISNN.

[3]  Bernhard Schölkopf,et al.  EnhanceNet: Single Image Super-Resolution Through Automated Texture Synthesis , 2016, 2017 IEEE International Conference on Computer Vision (ICCV).

[4]  Ningfang Song,et al.  White-Light Interference Microscopy Image Super-Resolution Using Generative Adversarial Networks , 2020, IEEE Access.

[5]  Chao Dong,et al.  Recovering Realistic Texture in Image Super-Resolution by Deep Spatial Feature Transform , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[6]  Xiaoou Tang,et al.  Accelerating the Super-Resolution Convolutional Neural Network , 2016, ECCV.

[7]  Christian Ledig,et al.  Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[8]  Ji-Hwan Kim,et al.  Super-Resolution of License Plate Images via Character-Based Perceptual Loss , 2020, 2020 IEEE International Conference on Big Data and Smart Computing (BigComp).

[9]  Wangs-Su Jeon,et al.  Single Image Super Resolution using Residual Learning , 2019, 2019 International Conference on Fuzzy Theory and Its Applications (iFUZZY).

[10]  Jiang Wenjie,et al.  Research on Super-resolution Reconstruction Algorithm of Remote Sensing Image Based on Generative Adversarial Networks , 2019, 2019 IEEE 2nd International Conference on Automation, Electronics and Electrical Engineering (AUTEEE).

[11]  Junjun Jiang,et al.  Edge-Enhanced GAN for Remote Sensing Image Superresolution , 2019, IEEE Transactions on Geoscience and Remote Sensing.

[12]  Yi Fang,et al.  GraspFusionNet: a two-stage multi-parameter grasp detection network based on RGB–XYZ fusion in dense clutter , 2020, Machine Vision and Applications.

[13]  Nanning Zheng,et al.  Incorporating image priors with deep convolutional neural networks for image super-resolution , 2016, Neurocomputing.

[14]  Yunsong Li,et al.  Hyperspectral image super-resolution using deep convolutional neural network , 2017, Neurocomputing.

[15]  Tong Tong,et al.  Image Super-Resolution Using Dense Skip Connections , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[16]  Kyoung Mu Lee,et al.  Accurate Image Super-Resolution Using Very Deep Convolutional Networks , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[17]  Peng Wang,et al.  Adaptive Importance Learning for Improving Lightweight Image Super-Resolution Network , 2018, International Journal of Computer Vision.

[18]  Muwei Jian,et al.  Improved face super-resolution generative adversarial networks , 2020, Machine Vision and Applications.

[19]  Christoph Busch,et al.  Impact and Detection of Facial Beautification in Face Recognition: An Overview , 2019, IEEE Access.

[20]  Ramesh Nallapati,et al.  OCGAN: One-Class Novelty Detection Using GANs With Constrained Latent Representations , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[21]  Jianmin Jiang,et al.  Deep learning based image Super-resolution for nonlinear lens distortions , 2018, Neurocomputing.

[22]  Mario Fritz,et al.  Attributing Fake Images to GANs: Learning and Analyzing GAN Fingerprints , 2018, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[23]  Pingping Wei,et al.  CT Image Super Resolution Based On Improved SRGAN , 2020, 2020 5th International Conference on Computer and Communication Systems (ICCCS).

[24]  Mirabela Rusu,et al.  Anisotropic Super Resolution In Prostate Mri Using Super Resolution Generative Adversarial Networks , 2019, 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019).

[25]  Luc Van Gool,et al.  NTIRE 2017 Challenge on Single Image Super-Resolution: Methods and Results , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[26]  Robert L. Stevenson,et al.  Super-resolution from image sequences-a review , 1998, 1998 Midwest Symposium on Circuits and Systems (Cat. No. 98CB36268).

[27]  Narendra Ahuja,et al.  Fast and Accurate Image Super-Resolution with Deep Laplacian Pyramid Networks , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[28]  Yurong Qian,et al.  Single Image Super-Resolution Reconstruction based on the ResNeXt Network , 2020, Multimedia Tools and Applications.

[29]  Yury Yanovich,et al.  Converging blockchain and next-generation artificial intelligence technologies to decentralize and accelerate biomedical research and healthcare , 2015, Oncotarget.

[30]  Kyoung Mu Lee,et al.  Enhanced Deep Residual Networks for Single Image Super-Resolution , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[31]  Esa Rahtu,et al.  ICface: Interpretable and Controllable Face Reenactment Using GANs , 2019, 2020 IEEE Winter Conference on Applications of Computer Vision (WACV).

[32]  Tao Zhang,et al.  Channel Splitting Network for Single MR Image Super-Resolution , 2018, IEEE Transactions on Image Processing.

[33]  M. S. Anuar,et al.  Cardinality improvement of Zero Cross Correlation (ZCC) code for OCDMA visible light communication system utilizing catenated-OFDM modulation scheme , 2018, Optik.

[34]  Xinying Xu,et al.  Deep Learning Based Single Image Super-resolution: A Survey , 2018, International Journal of Automation and Computing.

[35]  Heung-Kyu Lee,et al.  Median Filtered Image Restoration and Anti-Forensics Using Adversarial Networks , 2018, IEEE Signal Processing Letters.

[36]  Narendra Ahuja,et al.  Deep Laplacian Pyramid Networks for Fast and Accurate Super-Resolution , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[37]  Xueying Qin,et al.  Deep Depth Super-Resolution: Learning Depth Super-Resolution Using Deep Convolutional Neural Network , 2016, ACCV.

[38]  Wen Zhang,et al.  Deep learning methods in real-time image super-resolution: a survey , 2019, Journal of Real-Time Image Processing.

[39]  Hamid Krim,et al.  Sparse Generative Adversarial Network , 2019, 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW).

[40]  Luc Van Gool,et al.  Jointly Optimized Regressors for Image Super‐resolution , 2015, Comput. Graph. Forum.

[41]  Rob Fergus,et al.  Deep Generative Image Models using a Laplacian Pyramid of Adversarial Networks , 2015, NIPS.

[42]  Jie Li,et al.  Image super-resolution: The techniques, applications, and future , 2016, Signal Process..

[43]  Daniel Rueckert,et al.  Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[44]  Soo-Chang Pei,et al.  Image Super-Resolution Using Complex Dense Block on Generative Adversarial Networks , 2019, 2019 IEEE International Conference on Image Processing (ICIP).

[45]  Yun Fu,et al.  Image Super-Resolution Using Very Deep Residual Channel Attention Networks , 2018, ECCV.