Video Super-Resolution Based on Generative Adversarial Network and Edge Enhancement

With the help of deep neural networks, video super-resolution (VSR) has made a huge breakthrough. However, these deep learning-based methods are rarely used in specific situations. In addition, training sets may not be suitable because many methods only assume that under ideal circumstances, low-resolution (LR) datasets are downgraded from high-resolution (HR) datasets in a fixed manner. In this paper, we proposed a model based on Generative Adversarial Network (GAN) and edge enhancement to perform super-resolution (SR) reconstruction for LR and blur videos, such as closed-circuit television (CCTV). The adversarial loss allows discriminators to be trained to distinguish between SR frames and ground truth (GT) frames, which is helpful to produce realistic and highly detailed results. The edge enhancement function uses the Laplacian edge module to perform edge enhancement on the intermediate result, which helps further improve the final results. In addition, we add the perceptual loss to the loss function to obtain a higher visual experience. At the same time, we also tried training network on different datasets. A large number of experiments show that our method has advantages in the Vid4 dataset and other LR videos.

[1]  William T. Freeman,et al.  Example-Based Super-Resolution , 2002, IEEE Computer Graphics and Applications.

[2]  Donglian Qi,et al.  Hybrid dual-complementary metal-oxide-semiconductor/memristor synapse-based neural network with its applications in image super-resolution , 2019, IET Circuits Devices Syst..

[3]  Aggelos K. Katsaggelos,et al.  Video Super-Resolution With Convolutional Neural Networks , 2016, IEEE Transactions on Computational Imaging.

[4]  Chao Ren,et al.  Video Super-Resolution via Residual Learning , 2018, IEEE Access.

[5]  Ainuddin Wahid Abdul Wahab,et al.  Chroma key background detection for digital video using statistical correlation of blurring artifact , 2016, Digit. Investig..

[6]  R. Heintzmann,et al.  Superresolution by localization of quantum dots using blinking statistics. , 2005, Optics express.

[7]  Thomas S. Huang,et al.  Image Super-Resolution Via Sparse Representation , 2010, IEEE Transactions on Image Processing.

[8]  Junjun Jiang,et al.  Edge-Enhanced GAN for Remote Sensing Image Superresolution , 2019, IEEE Transactions on Geoscience and Remote Sensing.

[9]  Lei Zhang,et al.  Nonlocally Centralized Sparse Representation for Image Restoration , 2013, IEEE Transactions on Image Processing.

[10]  Ainuddin Wahid Abdul Wahab,et al.  Digital video inpainting detection using correlation of hessian matrix , 2016 .

[11]  Shukai Duan,et al.  A general memristor-based pulse coupled neural network with variable linking coefficient for multi-focus image fusion , 2018, Neurocomputing.

[12]  Marc Levoy,et al.  Handheld multi-frame super-resolution , 2019, ACM Trans. Graph..

[13]  Nils Thuerey,et al.  Learning temporal coherence via self-supervision for GAN-based video generation , 2020, ACM Trans. Graph..

[14]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[15]  Xiaoou Tang,et al.  Learning a Deep Convolutional Network for Image Super-Resolution , 2014, ECCV.

[16]  Ning Xu,et al.  Wide Activation for Efficient and Accurate Image Super-Resolution , 2018, ArXiv.

[17]  Stanley H. Chan,et al.  Plug-and-Play ADMM for Image Restoration: Fixed-Point Convergence and Applications , 2016, IEEE Transactions on Computational Imaging.

[18]  Jianmin Jiang,et al.  Video Super Resolution via Deep Global-Aware Network , 2019, IEEE Access.

[19]  Juan Lin,et al.  Trinity of Pixel Enhancement: a Joint Solution for Demosaicking, Denoising and Super-Resolution , 2019, ArXiv.