Multi-scale adversarial network for underwater image restoration

Abstract Underwater image restoration, which is the keystone to the underwater vision research, is still a challenging work. The key point of underwater image restoration work is how to remove the turbidity and the color distortion caused by the underwater environment. In this paper, we propose an underwater image restoration method based on transferring an underwater style image into a recovered style using Multi-Scale Cycle Generative Adversarial Network (MCycle GAN) System. We include a Structural Similarity Index Measure loss (SSIM loss), which can provide more flexibility to model the detail structural to improve the image restoration performance. We use dark channel prior (DCP) algorithm to get the transmission map of an image and design an adaptive SSIM loss to improve underwater image quality. We input this information into the network for multi-scale calculation on the images, which achieves the combination of DCP algorithm and Cycle-Consistent Adversarial Networks (CycleGAN). By compared the quantitative and qualitative with existing state-of-the-art approaches, our method shows a pleasing performance on the underwater image dataset.

[1]  Nan Wang,et al.  Underwater Image Restoration via Maximum Attenuation Identification , 2017, IEEE Access.

[2]  Alexei A. Efros,et al.  Image-to-Image Translation with Conditional Adversarial Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[3]  Weilin Hou,et al.  Automated underwater image restoration and retrieval of related optical properties , 2007, 2007 IEEE International Geoscience and Remote Sensing Symposium.

[4]  Huimin Lu,et al.  Multisensor Image Fusion and Enhancement in Spectral Total Variation Domain , 2018, IEEE Transactions on Multimedia.

[5]  Huimin Lu,et al.  Underwater Optical Image Processing: a Comprehensive Review , 2017, Mob. Networks Appl..

[6]  Adrian Galdran,et al.  Automatic Red-Channel underwater image restoration , 2015, J. Vis. Commun. Image Represent..

[7]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[8]  Chuan Li,et al.  Precomputed Real-Time Texture Synthesis with Markovian Generative Adversarial Networks , 2016, ECCV.

[9]  Shai Avidan,et al.  Non-local Image Dehazing , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[10]  Huchuan Lu,et al.  Deep visual tracking: Review and experimental comparison , 2018, Pattern Recognit..

[11]  X. Cufi,et al.  On the way to solve lighting problems in underwater imaging , 2002, OCEANS '02 MTS/IEEE.

[12]  Alexei A. Efros,et al.  Unpaired Image-to-Image Translation Using Cycle-Consistent Adversarial Networks , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[13]  Yung-Cheng Liu,et al.  Automatic white balance for digital still camera , 1995 .

[14]  Jan Kautz,et al.  Loss Functions for Neural Networks for Image Processing , 2015, ArXiv.

[15]  Robert A. Hummel,et al.  Image Enhancement by Histogram transformation , 1975 .

[16]  Chen Gao,et al.  Human-Visual-System-Inspired Underwater Image Quality Measures , 2016, IEEE Journal of Oceanic Engineering.

[17]  John N. Lygouras,et al.  THETIS: an underwater remotely operated vehicle for water pollution measurements , 1998, Microprocess. Microsystems.

[18]  Peter Gerstoft,et al.  Source localization in an ocean waveguide using supervised machine learning , 2017, The Journal of the Acoustical Society of America.

[19]  Christian Ledig,et al.  Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[20]  Ling Shao,et al.  A Fast Single Image Haze Removal Algorithm Using Color Attenuation Prior , 2015, IEEE Transactions on Image Processing.

[21]  Dacheng Tao,et al.  DehazeNet: An End-to-End System for Single Image Haze Removal , 2016, IEEE Transactions on Image Processing.

[22]  Jian Wang,et al.  Single underwater image restoration by blue-green channels dehazing and red channel correction , 2016, 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[23]  R. A. Salam,et al.  Underwater Image Enhancement Using an Integrated Colour Model , 2007 .

[24]  Li Fei-Fei,et al.  Perceptual Losses for Real-Time Style Transfer and Super-Resolution , 2016, ECCV.

[25]  Yu Nakagawa,et al.  Super Resolving of the Depth Map for 3D Reconstruction of Underwater Terrain Using Kinect , 2016, 2016 IEEE 22nd International Conference on Parallel and Distributed Systems (ICPADS).

[26]  Younggun Cho,et al.  Estimation of ambient light and transmission map with common convolutional architecture , 2016, OCEANS 2016 MTS/IEEE Monterey.

[27]  Chunle Guo,et al.  Emerging From Water: Underwater Image Color Correction Based on Weakly Supervised Color Transfer , 2017, IEEE Signal Processing Letters.

[28]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[29]  Karel J. Zuiderveld,et al.  Contrast Limited Adaptive Histogram Equalization , 1994, Graphics Gems.

[30]  Gregory Dudek,et al.  Multi-domain monitoring of marine environments using a heterogeneous robot team , 2012, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[31]  Hanumant Singh,et al.  Advances in Underwater Robot Vehicles for Deep Ocean Exploration: Navigation, Control, and Survey Operations , 2000 .

[32]  Jian-Ru Lin,et al.  3D underwater scene reconstruction through descattering and colour correction , 2016, Int. J. Comput. Sci. Eng..

[33]  Raimondo Schettini,et al.  Underwater Image Processing: State of the Art of Restoration and Image Enhancement Methods , 2010, EURASIP J. Adv. Signal Process..

[34]  Alessandro Rizzi,et al.  Underwater color constancy: enhancement of automatic live fish recognition , 2003, IS&T/SPIE Electronic Imaging.

[35]  Joost van de Weijer,et al.  Author Manuscript, Published in "ieee Transactions on Image Processing Edge-based Color Constancy , 2022 .

[36]  Ying-Ching Chen,et al.  Underwater Image Enhancement by Wavelength Compensation and Dehazing , 2012, IEEE Transactions on Image Processing.

[37]  Li Fei-Fei,et al.  ImageNet: A large-scale hierarchical image database , 2009, CVPR.

[38]  Huimin Lu,et al.  Turbidity Underwater Image Restoration Using Spectral Properties and Light Compensation , 2016, IEICE Trans. Inf. Syst..

[39]  Jichang Guo,et al.  Underwater image enhancement by dehazing and color correction , 2015, J. Electronic Imaging.

[40]  Nan Wang,et al.  Underwater Inherent Optical Properties Estimation Using a Depth Aided Deep Neural Network , 2017, Comput. Intell. Neurosci..

[41]  Ping Shi,et al.  The recognition of Chinese caption text in news video using convolutional neural network , 2016, 2016 IEEE Advanced Information Management, Communicates, Electronic and Automation Control Conference (IMCEC).

[42]  Jules S. Jaffe,et al.  Computer modeling and the design of optimal underwater imaging systems , 1990 .

[43]  Jian Sun,et al.  Single image haze removal using dark channel prior , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[44]  Jie Li,et al.  WaterGAN: Unsupervised Generative Network to Enable Real-Time Color Correction of Monocular Underwater Images , 2017, IEEE Robotics and Automation Letters.

[45]  Jean-Michel Morel,et al.  Fast implementation of color constancy algorithms , 2009, Electronic Imaging.

[46]  Nor Ashidi Mat Isa,et al.  Underwater image quality enhancement through integrated color model with Rayleigh distribution , 2015, Appl. Soft Comput..