Image Denoising Using Generative Adversarial Network

Image denoising is one of the most important and fundamental research areas in the digital image-processing field. A noisy image can mislead image processing-based research. Therefore, image denoising is a critical area of research. In the recent advancement of computer vision, deep learning becomes most powerful tool. Deep learning is solving most of the problems, usually, which were earlier solved by various conventional techniques. The progress of deep learning encourages researchers to apply deep learning-based methods into image denoising also. In recent years, generative adversarial network (GAN) becomes a new avenue in computer vision research. The GANs are adversarial networks with generative capability, and the network has a very vast area of applications. In this chapter, we concentrate on a specific area of application of GAN—image denoising. At first, the traditional denoising techniques are highlighted. Then, we state the underlying architecture of GAN and its modifications. Then, we discuss the way GANs are applied in the area of image denoising. We survey all recent works of GANs in image denoising and categories those work according to the type of input images. In the end, we propose some research directions in this area. The compilation and discussions presented in this chapter regarding image denoising using GAN are new inclusion, and similar survey work is not available for the community.

[1]  Antonia Creswell,et al.  Denoising Adversarial Autoencoders , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[2]  Vishal M. Patel,et al.  Image De-Raining Using a Conditional Generative Adversarial Network , 2017, IEEE Transactions on Circuits and Systems for Video Technology.

[3]  Sung-Jea Ko,et al.  Adversarial Context Aggregation Network for Low-Light Image Enhancement , 2018, 2018 Digital Image Computing: Techniques and Applications (DICTA).

[4]  Michael Elad,et al.  Image Denoising Via Sparse and Redundant Representations Over Learned Dictionaries , 2006, IEEE Transactions on Image Processing.

[5]  Xuanqin Mou,et al.  Low-Dose CT Image Denoising Using a Generative Adversarial Network With Wasserstein Distance and Perceptual Loss , 2017, IEEE Transactions on Medical Imaging.

[6]  Kavita Chachadi,et al.  Modeling and Performance Evaluation of Generative adversarial network for image denoising , 2018, 2018 International Conference on Computational Techniques, Electronics and Mechanical Systems (CTEMS).

[7]  Nalin Kumar,et al.  Noise Removal and Filtering Techniques used in Medical Images , 2017 .

[8]  Jae-Seok Choi,et al.  High-Resolution Image Dehazing with Respect to Training Losses and Receptive Field Sizes , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[9]  Dong Liu,et al.  Generative Adversarial Networks with Dense Connection for Optical Coherence Tomography Images Denoising , 2018, 2018 11th International Congress on Image and Signal Processing, BioMedical Engineering and Informatics (CISP-BMEI).

[10]  J Hennig,et al.  RARE imaging: A fast imaging method for clinical MR , 1986, Magnetic resonance in medicine.

[11]  Aaron C. Courville,et al.  Improved Training of Wasserstein GANs , 2017, NIPS.

[12]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[13]  Yang Yang,et al.  Micro-Doppler Spectrogram Denoising Based on Generative Adversarial Network , 2018, 2018 48th European Microwave Conference (EuMC).

[14]  Alessandro Foi,et al.  Image Denoising by Sparse 3-D Transform-Domain Collaborative Filtering , 2007, IEEE Transactions on Image Processing.

[15]  Aaron C. Courville,et al.  Adversarially Learned Inference , 2016, ICLR.

[16]  Soumith Chintala,et al.  Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks , 2015, ICLR.

[17]  Angus G. Forbes,et al.  Image Denoising Using A Generative Adversarial Network , 2019, 2019 IEEE 2nd International Conference on Information and Computer Technologies (ICICT).

[18]  Michael Elad,et al.  On Single Image Scale-Up Using Sparse-Representations , 2010, Curves and Surfaces.

[19]  Wenzhong Guo,et al.  Deep Multimodal Representation Learning: A Survey , 2019, IEEE Access.

[20]  Lei Zhang,et al.  Color demosaicking by local directional interpolation and nonlocal adaptive thresholding , 2011, J. Electronic Imaging.

[21]  Sun Yi,et al.  A New Generative Adversarial Network for Texture Preserving Image Denoising , 2018, ICIP 2018.

[22]  Roberto Manduchi,et al.  Bilateral filtering for gray and color images , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[23]  L. Shao,et al.  From Heuristic Optimization to Dictionary Learning: A Review and Comprehensive Comparison of Image Denoising Algorithms , 2014, IEEE Transactions on Cybernetics.

[24]  Vishal M. Patel,et al.  Generative adversarial network-based restoration of speckled SAR images , 2017, 2017 IEEE 7th International Workshop on Computational Advances in Multi-Sensor Adaptive Processing (CAMSAP).

[25]  Ming Yang,et al.  Image Blind Denoising with Generative Adversarial Network Based Noise Modeling , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[26]  Sergios Theodoridis,et al.  Adaptive Kernel-Based Image Denoising Employing Semi-Parametric Regularization , 2010, IEEE Transactions on Image Processing.

[27]  A. Baudes,et al.  A Nonlocal Algorithm for Image Denoising , 2005, CVPR 2005.

[28]  Yuhui Zheng,et al.  Recent Progress on Generative Adversarial Networks (GANs): A Survey , 2019, IEEE Access.

[29]  Bernard Widrow,et al.  Least-mean-square adaptive filters , 2003 .

[30]  Stefan Roth,et al.  Benchmarking Denoising Algorithms with Real Photographs , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[31]  Tingtian Li,et al.  Single-Image Reflection Removal via a Two-Stage Background Recovery Process , 2019, IEEE Signal Processing Letters.

[32]  Jenq-Neng Hwang,et al.  Single Image Snow Removal via Composition Generative Adversarial Networks , 2019, IEEE Access.

[33]  Wei Jiang,et al.  Motion-Blind Blur Removal for CT Images with Wasserstein Generative Adversarial Networks , 2018, 2018 11th International Congress on Image and Signal Processing, BioMedical Engineering and Informatics (CISP-BMEI).

[34]  Tae Hyun Kim,et al.  Deep Multi-scale Convolutional Neural Network for Dynamic Scene Deblurring , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[35]  Jin Keun Seo,et al.  Unpaired Image Denoising Using a Generative Adversarial Network in X-Ray CT , 2019, IEEE Access.

[36]  Lei Zhang,et al.  Sparsity-based image denoising via dictionary learning and structural clustering , 2011, CVPR 2011.

[37]  Simon Osindero,et al.  Conditional Generative Adversarial Nets , 2014, ArXiv.

[38]  Michael S. Bernstein,et al.  ImageNet Large Scale Visual Recognition Challenge , 2014, International Journal of Computer Vision.

[39]  Vishal M. Patel,et al.  Densely Connected Pyramid Dehazing Network , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[40]  Jing Liu,et al.  Recurrent Conditional Generative Adversarial Network for Image Deblurring , 2019, IEEE Access.

[41]  Norbert Wiener,et al.  Extrapolation, Interpolation, and Smoothing of Stationary Time Series , 1964 .

[42]  Aggelos K. Katsaggelos,et al.  Noise reduction filters for dynamic image sequences: a review , 1995, Proc. IEEE.

[43]  Vladlen Koltun,et al.  Multi-Scale Context Aggregation by Dilated Convolutions , 2015, ICLR.

[44]  Hu Chen,et al.  Visual Attention Network for Low-Dose CT , 2018, IEEE Signal Processing Letters.

[45]  Guang-Zhong Yang,et al.  Structure adaptive anisotropic filtering , 1995 .

[46]  Liujuan Cao,et al.  Joint Denoising and Super-Resolution via Generative Adversarial Training , 2018, 2018 24th International Conference on Pattern Recognition (ICPR).

[47]  Yi Chen,et al.  Motion Deblurring via Using Generative Adversarial Networks for Space-Based Imaging , 2018, 2018 IEEE 16th International Conference on Software Engineering Research, Management and Applications (SERA).

[48]  Navdeep Jaitly,et al.  Adversarial Autoencoders , 2015, ArXiv.

[49]  Léon Bottou,et al.  Wasserstein Generative Adversarial Networks , 2017, ICML.

[50]  Guang Yang,et al.  DAGAN: Deep De-Aliasing Generative Adversarial Networks for Fast Compressed Sensing MRI Reconstruction , 2018, IEEE Transactions on Medical Imaging.

[51]  Yu Qiao,et al.  RDS-Denoiser: a Detail-preserving Convolutional Neural Network for Image Denoising , 2018, 2018 IEEE International Conference on Cyborg and Bionic Systems (CBS).

[52]  Martin J. Wainwright,et al.  Image denoising using scale mixtures of Gaussians in the wavelet domain , 2003, IEEE Trans. Image Process..

[53]  David Zhang,et al.  Two-stage image denoising by principal component analysis with local pixel grouping , 2010, Pattern Recognit..

[54]  Pierrick Coupé,et al.  An Optimized Blockwise Nonlocal Means Denoising Filter for 3-D Magnetic Resonance Images , 2008, IEEE Transactions on Medical Imaging.

[55]  Andrea Vedaldi,et al.  It Takes (Only) Two: Adversarial Generator-Encoder Networks , 2017, AAAI.

[56]  Yann LeCun,et al.  Energy-based Generative Adversarial Network , 2016, ICLR.

[57]  Michael J. Black,et al.  Fields of Experts , 2009, International Journal of Computer Vision.

[58]  Kunal Swami,et al.  CANDY: Conditional Adversarial Networks based End-to-End System for Single Image Haze Removal , 2018, 2018 24th International Conference on Pattern Recognition (ICPR).

[59]  Gerard de Haan,et al.  An Overview and Performance Evaluation of Classification-Based Least Squares Trained Filters , 2008, IEEE Transactions on Image Processing.

[60]  Peyman Milanfar,et al.  Kernel Regression for Image Processing and Reconstruction , 2007, IEEE Transactions on Image Processing.

[61]  Luc Van Gool,et al.  NTIRE 2017 Challenge on Single Image Super-Resolution: Methods and Results , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[62]  Xiang Zhu,et al.  Automatic Parameter Selection for Denoising Algorithms Using a No-Reference Measure of Image Content , 2010, IEEE Transactions on Image Processing.

[63]  Jean-Michel Morel,et al.  The Noise Clinic: a Blind Image Denoising Algorithm , 2015, Image Process. Line.

[64]  Joachim Weickert,et al.  Rotationally invariant similarity measures for nonlocal image denoising , 2011, J. Vis. Commun. Image Represent..

[65]  Denis Lukovnikov,et al.  On the regularization of Wasserstein GANs , 2017, ICLR.

[66]  Trevor Darrell,et al.  Adversarial Feature Learning , 2016, ICLR.

[67]  Luc Brun,et al.  Non-local image smoothing by applying anisotropic diffusion PDE's in the space of patches , 2009, 2009 16th IEEE International Conference on Image Processing (ICIP).

[68]  Van-Giang Nguyen,et al.  Joint Image Deblurring and Binarization for License Plate Images using Deep Generative Adversarial Networks , 2018, 2018 5th NAFOSTED Conference on Information and Computer Science (NICS).

[69]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[70]  Linda G. Shapiro,et al.  Computer Vision , 2001 .

[71]  Guillermo Sapiro,et al.  Non-local sparse models for image restoration , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[72]  Thierry Blu,et al.  A New SURE Approach to Image Denoising: Interscale Orthonormal Wavelet Thresholding , 2007, IEEE Transactions on Image Processing.

[73]  Santanu Chaudhury,et al.  Ultrasound Image Enhancement Using Structure Oriented Adversarial Network , 2018, IEEE Signal Processing Letters.

[74]  David Pfau,et al.  Unrolled Generative Adversarial Networks , 2016, ICLR.