CA-GAN: Class-Condition Attention GAN for Underwater Image Enhancement

Underwater images suffer from serious color distortion and detail loss because of the wavelength-dependent light absorption and scattering, which seriously influences the subsequent underwater object detection and recognition. The latest methods for underwater image enhancement are based on deep models, which focus on finding a mapping function from the underwater image subspace to a ground-truth image subspace. They neglect the diversity of underwater conditions which leads to different background colors of underwater images. In this paper, we propose a Class-condition Attention Generative Adversarial Network (CA-GAN) to enhance an underwater image. We build an underwater image dataset which contains ten categories generated by the simulator with different water attenuation coefficient and depth. Relying on the underwater image classes, CA-GAN creates a many-to-one mapping function for an underwater image. Moreover, in order to generate the realistic image, attention mechanism is utilized. In the channel attention block, the feature maps in the front-end layers and the back-end layers are fused along channels, and in the spatial attention block, feature maps are pixel-wise fused. Extensive experiments are conducted on synthetic and real underwater images. The experimental results demonstrate that CA-GAN can effectively recover color and detail of various scenes of underwater images and is superior to the state-of-the-art methods.

[1]  Yuichi Yoshida,et al.  Spectral Normalization for Generative Adversarial Networks , 2018, ICLR.

[2]  B. L. McGlamery,et al.  A Computer Model For Underwater Camera Systems , 1980, Other Conferences.

[3]  Pei-Yin Chen,et al.  Low Complexity Underwater Image Enhancement Based on Dark Channel Prior , 2011, 2011 Second International Conference on Innovations in Bio-inspired Computing and Applications.

[4]  Ian D. Reid,et al.  Learning Depth from Single Monocular Images Using Deep Convolutional Neural Fields , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[5]  Radu Timofte,et al.  2018 PIRM Challenge on Perceptual Image Super-resolution , 2018, ArXiv.

[6]  Dan Feng,et al.  Benchmarking Single-Image Dehazing and Beyond , 2017, IEEE Transactions on Image Processing.

[7]  Codruta O. Ancuti,et al.  Enhancing underwater images and videos by fusion , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[8]  Ming Zhu,et al.  Real-World Underwater Enhancement: Challenges, Benchmarks, and Solutions Under Natural Light , 2019, IEEE Transactions on Circuits and Systems for Video Technology.

[9]  Eric O. Postma,et al.  A Learned Representation of Artist-Specific Colourisation , 2017, 2017 IEEE International Conference on Computer Vision Workshops (ICCVW).

[10]  Takayuki Hamamoto,et al.  Underwater Image Color Correction using Exposure-Bracketing Imaging , 2018, IEEE Signal Processing Letters.

[11]  Jing-Yu Yang,et al.  Underwater image enhancement based on structure-texture decomposition , 2017, 2017 IEEE International Conference on Image Processing (ICIP).

[12]  Dacheng Tao,et al.  An Underwater Image Enhancement Benchmark Dataset and Beyond , 2019, IEEE Transactions on Image Processing.

[13]  Jie Li,et al.  WaterGAN: Unsupervised Generative Network to Enable Real-Time Color Correction of Monocular Underwater Images , 2017, IEEE Robotics and Automation Letters.

[14]  Xiaoli Yu,et al.  Underwater-GAN: Underwater Image Restoration via Conditional Generative Adversarial Network , 2018, CVAUI/IWCF/MIPPSNA@ICPR.

[15]  Xiao-Ping Zhang,et al.  A retinex-based enhancing approach for single underwater image , 2014, 2014 IEEE International Conference on Image Processing (ICIP).

[16]  Jonathon Shlens,et al.  Conditional Image Synthesis with Auxiliary Classifier GANs , 2016, ICML.

[17]  Mario Fernando Montenegro Campos,et al.  Underwater Depth Estimation and Image Restoration Based on Single Images , 2016, IEEE Computer Graphics and Applications.

[18]  Md Jahidul Islam,et al.  Enhancing Underwater Imagery Using Generative Adversarial Networks , 2018, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[19]  Jiri Matas,et al.  DeblurGAN: Blind Motion Deblurring Using Conditional Adversarial Networks , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[20]  Pamela C. Cosman,et al.  Underwater Image Restoration Based on Image Blurriness and Light Absorption , 2017, IEEE Transactions on Image Processing.

[21]  Y.Y. Schechner,et al.  Recovery of underwater visibility and structure by polarization analysis , 2005, IEEE Journal of Oceanic Engineering.

[22]  Chunle Guo,et al.  Emerging From Water: Underwater Image Color Correction Based on Weakly Supervised Color Transfer , 2017, IEEE Signal Processing Letters.

[23]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[24]  S. Avidan,et al.  Diving into Haze-Lines: Color Restoration of Underwater Images , 2017 .

[25]  C. Mobley Light and Water: Radiative Transfer in Natural Waters , 1994 .

[26]  Vishal M. Patel,et al.  Densely Connected Pyramid Dehazing Network , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[27]  Jeff Donahue,et al.  Large Scale GAN Training for High Fidelity Natural Image Synthesis , 2018, ICLR.

[28]  Runmin Cong,et al.  Underwater Image Enhancement by Dehazing With Minimum Information Loss and Histogram Distribution Prior , 2016, IEEE Transactions on Image Processing.

[29]  Vishal M. Patel,et al.  Image De-Raining Using a Conditional Generative Adversarial Network , 2017, IEEE Transactions on Circuits and Systems for Video Technology.

[30]  Yang Wang,et al.  A deep CNN method for underwater image enhancement , 2017, 2017 IEEE International Conference on Image Processing (ICIP).

[31]  N Carlevaris-Bianco,et al.  Initial results in underwater single image dehazing , 2010, OCEANS 2010 MTS/IEEE SEATTLE.

[32]  Li Fei-Fei,et al.  ImageNet: A large-scale hierarchical image database , 2009, CVPR.

[33]  David Dagan Feng,et al.  Single Image Dehazing with White Balance Correction and Image Decomposition , 2012, 2012 International Conference on Digital Image Computing Techniques and Applications (DICTA).

[34]  Lei Zhang,et al.  Learning Aggregated Transmission Propagation Networks for Haze Removal and Beyond , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[35]  Shai Avidan,et al.  Underwater Single Image Color Restoration Using Haze-Lines and a New Quantitative Dataset , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.