Joint Denoising and Super-Resolution via Generative Adversarial Training

Single image denoising and super-resolution are sitting in the core of various image processing and pattern recognition applications. Typically, these two tasks are handled separately, without regarding to joint reinforcement and learning. The former deals with equal-size pixel-to-pixel translation, while the latter deals with scaling up amount of input pixels. In this paper, we propose a Generative Adversarial Network(GAN) towards joint learning of single image denoising and super-resolution. In principle, our design allows both tasks to share several common building blocks, with the linking between both outputs to reinforce each other. Such a reinforcement is accomplished via designing a novel generative network through optimizing a novel loss function to achieve both denoising and super-resolution. Quantitatively comparing to a set of alternative approaches and baselines, the experiment demonstrated superior performance our method in denoising and super-resolution with high upscaling factors.

[1]  Lei Zhang,et al.  Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising , 2016, IEEE Transactions on Image Processing.

[2]  Jean-Michel Morel,et al.  A non-local algorithm for image denoising , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[3]  Christian Ledig,et al.  Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[4]  Daniel Rueckert,et al.  Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[5]  Michael S. Bernstein,et al.  ImageNet Large Scale Visual Recognition Challenge , 2014, International Journal of Computer Vision.

[6]  Andrew L. Maas Rectifier Nonlinearities Improve Neural Network Acoustic Models , 2013 .

[7]  Xiaoou Tang,et al.  Single Image Haze Removal Using Dark Channel Prior , 2011 .

[8]  Richard A. Haddad,et al.  Adaptive median filters: new algorithms and results , 1995, IEEE Trans. Image Process..

[9]  Yicong Zhou,et al.  A new weighted mean filter with a two-phase detector for removing impulse noise , 2015, Inf. Sci..

[10]  R. Flowerdew,et al.  Using areal interpolation methods in geographic information systems , 1991 .

[11]  Yonggang Zhang,et al.  A robust Gaussian approximate filter for nonlinear systems with heavy tailed measurement noises , 2016, 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[12]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[13]  Leon A. Gatys,et al.  Texture Synthesis Using Convolutional Neural Networks , 2015, NIPS.

[14]  Narendra Ahuja,et al.  Super-resolving Noisy Images , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[15]  Zhe Zhou,et al.  Cognition and Removal of Impulse Noise With Uncertainty , 2012, IEEE Transactions on Image Processing.

[16]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[17]  Dacheng Tao,et al.  DehazeNet: An End-to-End System for Single Image Haze Removal , 2016, IEEE Transactions on Image Processing.

[18]  Guangchun Luo,et al.  A logarithm-based image denoising method for a mixture of Gaussian white noise and signal dependent noise , 2017, J. Intell. Fuzzy Syst..

[19]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[20]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[21]  Yong Cheng,et al.  Modified Adaptive Gaussian Filter for Removal of Salt and Pepper Noise , 2015, KSII Trans. Internet Inf. Syst..

[22]  Hossein Nezamabadi-pour,et al.  A Fast Adaptive Salt and Pepper Noise Reduction Method in Images , 2013, Circuits Syst. Signal Process..

[23]  Donald G. Bailey,et al.  A novel approach to real-time bilinear interpolation , 2004, Proceedings. DELTA 2004. Second IEEE International Workshop on Electronic Design, Test and Applications.

[24]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[25]  Alessandro Foi,et al.  Image Denoising by Sparse 3-D Transform-Domain Collaborative Filtering , 2007, IEEE Transactions on Image Processing.

[26]  Truong Q. Nguyen,et al.  An Adaptable $k$ -Nearest Neighbors Algorithm for MMSE Image Interpolation , 2009, IEEE Transactions on Image Processing.

[27]  Cem Kalyoncu,et al.  A weighted mean filter with spatial-bias elimination for impulse noise removal , 2015, Digit. Signal Process..

[28]  Fang Li,et al.  A New Adaptive Weighted Mean Filter for Removing Salt-and-Pepper Noise , 2014, IEEE Signal Processing Letters.

[29]  Zhongliang Jing,et al.  A Regular k-Shrinkage Thresholding Operator for the Removal of Mixed Gaussian-Impulse Noise , 2017, Appl. Comput. Intell. Soft Comput..

[30]  Jian Sun,et al.  Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[31]  Stefan Harmeling,et al.  Image denoising: Can plain neural networks compete with BM3D? , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[32]  R. E. Carlson,et al.  Monotone Piecewise Cubic Interpolation , 1980 .

[33]  Li Fei-Fei,et al.  Perceptual Losses for Real-Time Style Transfer and Super-Resolution , 2016, ECCV.