Jointly adversarial networks for wavelength compensation and dehazing of underwater images

Severe color casts, low contrast and blurriness of underwater images caused by light absorption and scattering result in a difficult task for exploring underwater environments. Different from most of previous underwater image enhancement methods that compute light attenuation along object-camera path through hazy image formation model, we propose a novel jointly wavelength compensation and dehazing network (JWCDN) that takes into account the wavelength attenuation along surface-object path and the scattering along object-camera path simultaneously. By embedding a simplified underwater formation model into generative adversarial network, we can jointly estimates the transmission map, wavelength attenuation and background light via different network modules, and uses the simplified underwater image formation model to recover degraded underwater images. Especially, a multi-scale densely connected encoder-decoder network is proposed to leverage features from multiple layers for estimating the transmission map. To further improve the recovered image, we use an edge preserving network module to enhance the detail of the recovered image. Moreover, to train the proposed network, we propose a novel underwater image synthesis method that generates underwater images with inherent optical properties of different water types. The synthesis method can simulate the color, contrast and blurriness appearance of real-world underwater environments simultaneously. Extensive experiments on synthetic and real-world underwater images demonstrate that the proposed method yields comparable or better results on both subjective and objective assessments, compared with several state-of-the-art methods.

[1]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[2]  Jie Li,et al.  WaterGAN: Unsupervised Generative Network to Enable Real-Time Color Correction of Monocular Underwater Images , 2017, IEEE Robotics and Automation Letters.

[3]  Adrian Galdran,et al.  Automatic Red-Channel underwater image restoration , 2015, J. Vis. Commun. Image Represent..

[4]  Dacheng Tao,et al.  DehazeNet: An End-to-End System for Single Image Haze Removal , 2016, IEEE Transactions on Image Processing.

[5]  Runmin Cong,et al.  A hybrid method for underwater image correction , 2017, Pattern Recognit. Lett..

[6]  Xiao-Ping Zhang,et al.  A retinex-based enhancing approach for single underwater image , 2014, 2014 IEEE International Conference on Image Processing (ICIP).

[7]  Xiaogang Wang,et al.  Pyramid Scene Parsing Network , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[8]  Zhou Wang,et al.  A Patch-Structure Representation Method for Quality Assessment of Contrast Changed Images , 2015, IEEE Signal Processing Letters.

[9]  Derek Hoiem,et al.  Indoor Segmentation and Support Inference from RGBD Images , 2012, ECCV.

[10]  Md Jahidul Islam,et al.  Enhancing Underwater Imagery Using Generative Adversarial Networks , 2018, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[11]  Xiaoou Tang,et al.  Single Image Haze Removal Using Dark Channel Prior , 2011 .

[12]  B. L. McGlamery,et al.  A Computer Model For Underwater Camera Systems , 1980, Other Conferences.

[13]  Yi Zhu,et al.  DenseNet for dense flow , 2017, 2017 IEEE International Conference on Image Processing (ICIP).

[14]  Curtis D Mobley,et al.  Inherent optical properties of Jerlov water types. , 2015, Applied optics.

[15]  Jules S. Jaffe,et al.  Computer modeling and the design of optimal underwater imaging systems , 1990 .

[16]  Tao Jin,et al.  Deriving inherent optical properties from background color and underwater image enhancement , 2015 .

[17]  Chunle Guo,et al.  Emerging From Water: Underwater Image Color Correction Based on Weakly Supervised Color Transfer , 2017, IEEE Signal Processing Letters.

[18]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[19]  Chong-Yi Li,et al.  Underwater Image Enhancement by Dehazing With Minimum Information Loss and Histogram Distribution Prior. , 2016, IEEE transactions on image processing : a publication of the IEEE Signal Processing Society.

[20]  Jian Sun,et al.  Single image haze removal using dark channel prior , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[21]  Patricia Ladret,et al.  The blur effect: perception and estimation with a new no-reference perceptual blur metric , 2007, Electronic Imaging.

[22]  Fatih Murat Porikli,et al.  Deep Underwater Image Enhancement , 2018, ArXiv.

[23]  拓海 杉山,et al.  “Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks”の学習報告 , 2017 .

[24]  Yoav Y. Schechner,et al.  The Next Best Underwater View , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[25]  Nor Ashidi Mat Isa,et al.  Enhancement of low quality underwater image through integrated global and local contrast correction , 2015, Appl. Soft Comput..

[26]  Nor Ashidi Mat Isa,et al.  Underwater image quality enhancement through integrated color model with Rayleigh distribution , 2015, Appl. Soft Comput..

[27]  Pamela C. Cosman,et al.  Underwater Image Restoration Based on Image Blurriness and Light Absorption , 2017, IEEE Transactions on Image Processing.

[28]  Raimondo Schettini,et al.  Underwater Image Processing: State of the Art of Restoration and Image Enhancement Methods , 2010, EURASIP J. Adv. Signal Process..

[29]  Pamela C. Cosman,et al.  Generalization of the Dark Channel Prior for Single Image Restoration , 2018, IEEE Transactions on Image Processing.

[30]  Li Fei-Fei,et al.  Perceptual Losses for Real-Time Style Transfer and Super-Resolution , 2016, ECCV.

[31]  Shai Avidan,et al.  Underwater Single Image Color Restoration Using Haze-Lines and a New Quantitative Dataset , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[32]  Chen Gao,et al.  Human-Visual-System-Inspired Underwater Image Quality Measures , 2016, IEEE Journal of Oceanic Engineering.

[33]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[34]  Yoshua Bengio,et al.  The One Hundred Layers Tiramisu: Fully Convolutional DenseNets for Semantic Segmentation , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[35]  Antoni Burguera,et al.  Imaging Systems for Advanced Underwater Vehicles , 2011 .

[36]  Codruta O. Ancuti,et al.  Enhancing underwater images and videos by fusion , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[37]  Hanumant Singh,et al.  Applications of Geo-Referenced Underwater Photo Mosaics in Marine Biology and Archaeology , 2007 .

[38]  Kilian Q. Weinberger,et al.  Densely Connected Convolutional Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[39]  Xiaochun Cao,et al.  Single Image Dehazing via Multi-scale Convolutional Neural Networks , 2016, ECCV.

[40]  Vishal M. Patel,et al.  Densely Connected Pyramid Dehazing Network , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[41]  Ying-Ching Chen,et al.  Underwater Image Enhancement by Wavelength Compensation and Dehazing , 2012, IEEE Transactions on Image Processing.

[42]  Codruta O. Ancuti,et al.  Color Balance and Fusion for Underwater Image Enhancement , 2018, IEEE Transactions on Image Processing.

[43]  Yoav Y. Schechner,et al.  Clear underwater vision , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[44]  Mario Fernando Montenegro Campos,et al.  Underwater Depth Estimation and Image Restoration Based on Single Images , 2016, IEEE Computer Graphics and Applications.

[45]  Alexei A. Efros,et al.  Image-to-Image Translation with Conditional Adversarial Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[46]  Jizheng Xu,et al.  An All-in-One Network for Dehazing and Beyond , 2017, ArXiv.

[47]  Thomas Brox,et al.  DeMoN: Depth and Motion Network for Learning Monocular Stereo , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).