A Survey of Restoration and Enhancement for Underwater Images

Images captured underwater usually suffer from color distortion, detail blurring, low contrast, and a bluish or greenish tone due to light scattering and absorption in the underwater medium, which in turn the visibility is adversely affected by these factors seriously. Over the last decades, various image restoration and enhancement methods have been developed by many researchers to improve the quality (visibility and highlight richer details) of underwater images. This paper introduces the overview of state-of-the-art underwater image restoration and enhancement techniques and classifies the approaches in two categories: image restoration (physical-based model) and image enhancement (nonphysical-based model). Furthermore, the classification of these two methods is elaborated. Then, the typical underwater image restoration and enhancement methods are discussed in detail, as well as a comprehensive study and fair evaluation of the methods is carried out from both qualitative and quantitative perspectives. Finally, the research process of underwater image restoration and enhancement is summarized and the suggestions for future research are prospected.

[1]  Shree K. Nayar,et al.  Structured light in scattering media , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[2]  Jian-Ru Lin,et al.  Underwater image enhancement method using weighted guided trigonometric filtering and artificial light correction , 2016, J. Vis. Commun. Image Represent..

[3]  Leandro dos Santos Coelho,et al.  Bio-inspired optimization algorithms for real underwater image restoration , 2019, Signal Process. Image Commun..

[4]  Dacheng Tao,et al.  An Underwater Image Enhancement Benchmark Dataset and Beyond , 2019, IEEE Transactions on Image Processing.

[5]  Tao Jin,et al.  Deriving inherent optical properties from background color and underwater image enhancement , 2015 .

[6]  Nor Ashidi Mat Isa,et al.  Automatic system for improving underwater image contrast and color through recursive adaptive histogram modification , 2017, Comput. Electron. Agric..

[7]  Codruta O. Ancuti,et al.  Color Balance and Fusion for Underwater Image Enhancement , 2018, IEEE Transactions on Image Processing.

[8]  Yiguang Liu,et al.  Adaptive Deep Convolutional Neural Networks for Scene-Specific Object Detection , 2019, IEEE Transactions on Circuits and Systems for Video Technology.

[9]  Shahriar Negahdaripour,et al.  Improved Stereo Matching in Scattering Media by Incorporating a Backscatter Cue , 2014, IEEE Transactions on Image Processing.

[10]  Mario Fernando Montenegro Campos,et al.  Underwater Depth Estimation and Image Restoration Based on Single Images , 2016, IEEE Computer Graphics and Applications.

[11]  Zhenbing Liu,et al.  Accurate segmentation of nuclei in pathological images via sparse reconstruction and deep convolutional networks , 2017, Neurocomputing.

[12]  Huimin Lu,et al.  Underwater Image Super-Resolution by Descattering and Fusion , 2017, IEEE Access.

[13]  Daniele Nardi,et al.  Enhancing Automatic Maritime Surveillance Systems With Visual Information , 2017, IEEE Transactions on Intelligent Transportation Systems.

[14]  Francisco R. Rubio,et al.  Formation Control of Autonomous Underwater Vehicles Subject to Communication Delays , 2014 .

[15]  Arcot Sowmya,et al.  An Underwater Color Image Quality Evaluation Metric , 2015, IEEE Transactions on Image Processing.

[16]  Mohua Zhang,et al.  Underwater Image Restoration Based on A New Underwater Image Formation Model , 2018, IEEE Access.

[17]  Xinghao Ding,et al.  Two-step approach for single underwater image enhancement , 2017, 2017 International Symposium on Intelligent Signal Processing and Communication Systems (ISPACS).

[18]  T. Sree Sharmila,et al.  A wavelet transform based contrast enhancement method for underwater acoustic images , 2018, Multidimens. Syst. Signal Process..

[19]  Ayman Alfalou,et al.  Enhancing underwater optical imaging by using a low-pass polarization filter. , 2019, Optics express.

[20]  Jian Liang,et al.  Underwater polarimetric imaging for visibility enhancement utilizing active unpolarized illumination , 2019, Optics Communications.

[21]  Wolfgang Heidrich,et al.  HDR-VDP-2: a calibrated visual metric for visibility and quality predictions in all luminance conditions , 2011, SIGGRAPH 2011.

[22]  Fatih Murat Porikli,et al.  Deep Underwater Image Enhancement , 2018, ArXiv.

[23]  Yu Wang,et al.  Efficient underwater image and video enhancement based on Retinex , 2019, Signal Image Video Process..

[24]  Daniel E. Clark,et al.  Particle PHD filter multiple target tracking in sonar image , 2007, IEEE Transactions on Aerospace and Electronic Systems.

[25]  Nan Wang,et al.  Underwater Image Restoration via Maximum Attenuation Identification , 2017, IEEE Access.

[26]  Lizhong Xu,et al.  Region-specialized underwater image restoration in inhomogeneous optical environments , 2014 .

[27]  Md Jahidul Islam,et al.  Enhancing Underwater Imagery Using Generative Adversarial Networks , 2018, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[28]  Ke Hu,et al.  An In-Depth Survey of Underwater Image Enhancement and Restoration , 2019, IEEE Access.

[29]  G. Buchsbaum A spatial processor model for object colour perception , 1980 .

[30]  Anuj Srivastava,et al.  Underwater Minefield Detection in Clutter Data Using Spatial Point-Process Models , 2016, IEEE Journal of Oceanic Engineering.

[31]  Bin Wang,et al.  Detecting Infrared Maritime Targets Overwhelmed in Sun Glitters by Antijitter Spatiotemporal Saliency , 2019, IEEE Transactions on Geoscience and Remote Sensing.

[32]  Simon X. Yang,et al.  Contrast Limited Adaptive Histogram Equalization-Based Fusion in YIQ and HSI Color Spaces for Underwater Image Enhancement , 2017, Int. J. Pattern Recognit. Artif. Intell..

[33]  M. Schaepman,et al.  Review of constituent retrieval in optically deep and complex waters from satellite imagery , 2012 .

[34]  N. C. Flemming,et al.  Archaeology of the continental shelf: Marine resources, submerged landscapes and underwater archaeology , 2008 .

[35]  E. Land The retinex theory of color vision. , 1977, Scientific American.

[36]  M. Wang,et al.  Removal of water scattering , 2010, 2010 2nd International Conference on Computer Engineering and Technology.

[37]  Runmin Cong,et al.  A hybrid method for underwater image correction , 2017, Pattern Recognit. Lett..

[38]  Anne E. James,et al.  Enhancing the low quality images using Unsupervised Colour Correction Method , 2010, 2010 IEEE International Conference on Systems, Man and Cybernetics.

[39]  Hong Zhu,et al.  Low-Illumination Image Enhancement Algorithm Based on a Physical Lighting Model , 2019, IEEE Transactions on Circuits and Systems for Video Technology.

[40]  Pamela C. Cosman,et al.  Underwater Image Restoration Based on Image Blurriness and Light Absorption , 2017, IEEE Transactions on Image Processing.

[41]  Yoav Y Schechner,et al.  Polarization-based vision through haze. , 2008, Applied optics.

[42]  Honghai Liu,et al.  Navigation Technologies for Autonomous Underwater Vehicles , 2008, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[43]  E H Land,et al.  An alternative technique for the computation of the designator in the retinex theory of color vision. , 1986, Proceedings of the National Academy of Sciences of the United States of America.

[44]  Chia-Chi Sung,et al.  Underwater image restoration by red-dark channel prior and point spread function deconvolution , 2015, 2015 IEEE International Conference on Signal and Image Processing Applications (ICSIPA).

[45]  Hangil Joe,et al.  Time-Delay Controller Design for Position Control of Autonomous Underwater Vehicle Under Disturbances , 2016, IEEE Transactions on Industrial Electronics.

[46]  M. S. Hitam,et al.  Mixture contrast limited adaptive histogram equalization for underwater image enhancement , 2013, 2013 International Conference on Computer Applications Technology (ICCAT).

[47]  Zia-ur Rahman,et al.  Retinex processing for automatic image enhancement , 2004, J. Electronic Imaging.

[48]  R.S. Kamathe,et al.  Quantification of retinex in enhancement of weather degraded images , 2008, 2008 International Conference on Audio, Language and Image Processing.

[49]  王永鑫 Wang Yong-xin,et al.  Underwater Image Enhancement Algorithm Based on Iterative Histogram Equalization with Conventional Light Source , 2018 .

[50]  C. L. Philip Chen,et al.  A Real-Time Vehicle Navigation Algorithm in Sensor Network Environments , 2012, IEEE Transactions on Intelligent Transportation Systems.

[51]  Alexandre Bernardino,et al.  Mosaic-based navigation for autonomous underwater vehicles , 2003 .

[52]  Ming-Hsuan Yang,et al.  Single Image Dehazing via Multi-scale Convolutional Neural Networks with Holistic Edges , 2019, International Journal of Computer Vision.

[53]  Nor Ashidi Mat Isa,et al.  Enhancement of low quality underwater image through integrated global and local contrast correction , 2015, Appl. Soft Comput..

[54]  Son-Cheol Yu,et al.  Robust Sonar-Based Underwater Object Recognition Against Angle-of-View Variation , 2016, IEEE Sensors Journal.

[55]  Yuan Zhou,et al.  Underwater Image Restoration Using Color-Line Model , 2019, IEEE Transactions on Circuits and Systems for Video Technology.

[56]  Yuan Yan Tang,et al.  A Local Contrast Method for Small Infrared Target Detection , 2014, IEEE Transactions on Geoscience and Remote Sensing.

[57]  Jules S. Jaffe,et al.  Computer modeling and the design of optimal underwater imaging systems , 1990 .

[58]  Jie Ma,et al.  A Gravity Gradient Differential Ratio Method for Underwater Object Detection , 2014, IEEE Geoscience and Remote Sensing Letters.

[59]  E. Trucco,et al.  Self-Tuning Underwater Image Restoration , 2006, IEEE Journal of Oceanic Engineering.

[60]  Kui Yang,et al.  Active underwater descattering and image recovery. , 2017, Applied optics.

[61]  Yang Wang,et al.  A deep CNN method for underwater image enhancement , 2017, 2017 IEEE International Conference on Image Processing (ICIP).

[62]  Arcot Sowmya,et al.  New Image Quality Evaluation Metric for Underwater Video , 2014, IEEE Signal Processing Letters.

[63]  Xiao-Ping Zhang,et al.  A retinex-based enhancing approach for single underwater image , 2014, 2014 IEEE International Conference on Image Processing (ICIP).

[64]  Ahmad Shahrizan Abdul Ghani,et al.  Image contrast enhancement using an integration of recursive-overlapped contrast limited adaptive histogram specification and dual-image wavelet fusion for the high visibility of deep underwater image , 2018 .

[65]  A. Bovik,et al.  A universal image quality index , 2002, IEEE Signal Processing Letters.

[66]  Tamaki Ura,et al.  An Optical Image Transmission System for Deep Sea Creature Sampling Missions Using Autonomous Underwater Vehicle , 2020, IEEE Journal of Oceanic Engineering.

[67]  Sun Feifei,et al.  An Approach for Underwater Image Denoising Via Wavelet Decomposition and High-pass Filter , 2011, 2011 Fourth International Conference on Intelligent Computation Technology and Automation.

[68]  Wei Song,et al.  An Experimental-Based Review of Image Enhancement and Image Restoration Methods for Underwater Imaging , 2019, IEEE Access.

[69]  Ming Zhao,et al.  Robust Infrared Maritime Target Detection Based on Visual Attention and Spatiotemporal Filtering , 2017, IEEE Transactions on Geoscience and Remote Sensing.

[70]  Darko Musicki,et al.  Joint Integrated Probabilistic Data Association - JIPDA , 2002, Proceedings of the Fifth International Conference on Information Fusion. FUSION 2002. (IEEE Cat.No.02EX5997).

[71]  Zhou Wang,et al.  Image Quality Assessment: From Error Measurement to Structural Similarity , 2004 .

[72]  Shai Avidan,et al.  Underwater Single Image Color Restoration Using Haze-Lines and a New Quantitative Dataset , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[73]  Yoav Y. Schechner,et al.  Turbid Scene Enhancement Using Multi-Directional Illumination Fusion , 2012, IEEE Transactions on Image Processing.

[74]  Weilin Hou,et al.  Imagery-derived modulation transfer function and its applications for underwater imaging , 2007, SPIE Optical Engineering + Applications.

[75]  Adrian Galdran,et al.  Automatic Red-Channel underwater image restoration , 2015, J. Vis. Commun. Image Represent..

[76]  Hua Lee,et al.  An Efficient Fusion-Based Defogging , 2017, IEEE Transactions on Image Processing.

[77]  R. A. Salam,et al.  Underwater Image Enhancement Using an Integrated Colour Model , 2007 .

[78]  Codruta O. Ancuti,et al.  Fusion-based restoration of the underwater images , 2011, 2011 18th IEEE International Conference on Image Processing.

[79]  Frédéric Galland,et al.  Color image simulation for underwater optics. , 2012, Applied optics.

[80]  Chen Gao,et al.  Human-Visual-System-Inspired Underwater Image Quality Measures , 2016, IEEE Journal of Oceanic Engineering.

[81]  Fatih Murat Porikli,et al.  Underwater scene prior inspired deep underwater image and video enhancement , 2020, Pattern Recognit..

[82]  Julien Bonnel,et al.  Wind Speed Estimation Using Acoustic Underwater Glider in a Near-Shore Marine Environment , 2019, IEEE Transactions on Geoscience and Remote Sensing.

[83]  Zhouhua Peng,et al.  Output-Feedback Path-Following Control of Autonomous Underwater Vehicles Based on an Extended State Observer and Projection Neural Networks , 2018, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[84]  Lin Zhao,et al.  Enhancing Visibility of Polarimetric Underwater Image by Transmittance Correction , 2017, IEEE Photonics Journal.

[85]  Codruta O. Ancuti,et al.  Enhancing underwater images and videos by fusion , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[86]  Ting Wang,et al.  Underwater image enhancement via extended multi-scale Retinex , 2017, Neurocomputing.

[87]  Xuezhi Wang,et al.  Efficient and Enhanced Multi-Target Tracking with Doppler Measurements , 2009, IEEE Transactions on Aerospace and Electronic Systems.

[88]  Graham D. Finlayson,et al.  Shades of Gray and Colour Constancy , 2004, CIC.

[89]  Frédéric D. Petit,et al.  Underwater image enhancement by attenuation inversionwith quaternions , 2009, 2009 IEEE International Conference on Acoustics, Speech and Signal Processing.

[90]  Ying-Ching Chen,et al.  Underwater Image Enhancement by Wavelength Compensation and Dehazing , 2012, IEEE Transactions on Image Processing.

[91]  Jian Sun,et al.  Guided Image Filtering , 2010, ECCV.

[92]  Zhou Wang,et al.  A Patch-Structure Representation Method for Quality Assessment of Contrast Changed Images , 2015, IEEE Signal Processing Letters.

[93]  Md Jahidul Islam,et al.  Understanding human motion and gestures for underwater human–robot collaboration , 2018, J. Field Robotics.

[94]  Yang Li,et al.  Adaptive Neural Network Control of AUVs With Control Input Nonlinearities Using Reinforcement Learning , 2017, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[95]  Qingwu Hu,et al.  Haze and Thin Cloud Removal via Sphere Model Improved Dark Channel Prior , 2019, IEEE Geoscience and Remote Sensing Letters.

[96]  Truong Q. Nguyen,et al.  An Investigation of Dehazing Effects on Image and Video Coding , 2012, IEEE Transactions on Image Processing.

[97]  Joost van de Weijer,et al.  Author Manuscript, Published in "ieee Transactions on Image Processing Edge-based Color Constancy , 2022 .

[98]  Paolo Braca,et al.  Bayesian Tracking in Underwater Wireless Sensor Networks With Port-Starboard Ambiguity , 2014, IEEE Transactions on Signal Processing.

[99]  Y.Y. Schechner,et al.  Recovery of underwater visibility and structure by polarization analysis , 2005, IEEE Journal of Oceanic Engineering.

[100]  Chunle Guo,et al.  Emerging From Water: Underwater Image Color Correction Based on Weakly Supervised Color Transfer , 2017, IEEE Signal Processing Letters.

[101]  Chong-Yi Li,et al.  Underwater Image Enhancement by Dehazing With Minimum Information Loss and Histogram Distribution Prior. , 2016, IEEE transactions on image processing : a publication of the IEEE Signal Processing Society.

[102]  Yoav Y. Schechner,et al.  Active Polarization Descattering , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[103]  Pamela C. Cosman,et al.  Generalization of the Dark Channel Prior for Single Image Restoration , 2018, IEEE Transactions on Image Processing.

[104]  Justin Marshall,et al.  Patterns and properties of polarized light in air and water , 2011, Philosophical Transactions of the Royal Society B: Biological Sciences.

[105]  Alan C. Bovik,et al.  Making a “Completely Blind” Image Quality Analyzer , 2013, IEEE Signal Processing Letters.

[106]  M. Abdullah-Al-Wadud,et al.  A Dynamic Histogram Equalization for Image Contrast Enhancement , 2007, 2007 Digest of Technical Papers International Conference on Consumer Electronics.

[107]  Nan Wang,et al.  Feeble object detection of underwater images through LSR with delay loop. , 2017, Optics express.

[108]  B. L. McGlamery,et al.  A Computer Model For Underwater Camera Systems , 1980, Other Conferences.

[109]  Weilin Hou,et al.  Automated underwater image restoration and retrieval of related optical properties , 2007, 2007 IEEE International Geoscience and Remote Sensing Symposium.

[110]  Huimin Lu,et al.  Underwater Optical Image Processing: a Comprehensive Review , 2017, Mob. Networks Appl..

[111]  Tiegen Liu,et al.  Underwater image recovery considering polarization effects of objects. , 2016, Optics express.

[112]  Jules S Jaffe Enhanced extended range underwater imaging via structured illumination. , 2010, Optics express.

[113]  Zia-ur Rahman,et al.  Multi-scale retinex for color image enhancement , 1996, Proceedings of 3rd IEEE International Conference on Image Processing.

[114]  John P. Oakley,et al.  Improving image quality in poor visibility conditions using a physical model for contrast degradation , 1998, IEEE Trans. Image Process..

[115]  Zhenyu Wu,et al.  All-In-One Underwater Image Enhancement using Domain-Adversarial Learning , 2019, CVPR Workshops.

[116]  Sung-Ho Cho,et al.  Real-Time Underwater Object Detection Based on DC Resistivity Method , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[117]  Huimin Lu,et al.  Underwater image enhancement using guided trigonometric bilateral filter and fast automatic color correction , 2013, 2013 IEEE International Conference on Image Processing.

[118]  Narendra Ahuja,et al.  Single image super-resolution from transformed self-exemplars , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[119]  Diksha Garg,et al.  Underwater image enhancement using blending of CLAHE and percentile methodologies , 2018, Multimedia Tools and Applications.

[120]  Fei Yuan,et al.  UNDERWATER IMAGE DE-SCATTERING AND ENHANCING USING DEHAZENET AND HWD , 2018 .

[121]  N Carlevaris-Bianco,et al.  Initial results in underwater single image dehazing , 2010, OCEANS 2010 MTS/IEEE SEATTLE.

[122]  Joonki Paik,et al.  Adaptive contrast enhancement using gain-controllable clipped histogram equalization , 2008, IEEE Transactions on Consumer Electronics.

[123]  Tian Zhou,et al.  Underwater pipeline leakage detection via multibeam sonar imagery , 2017 .

[124]  Huimin Lu,et al.  Contrast enhancement for images in turbid water. , 2015, Journal of the Optical Society of America. A, Optics, image science, and vision.

[125]  Lin Zhao,et al.  Underwater Image Recovery Under the Nonuniform Optical Field Based on Polarimetric Imaging , 2018, IEEE Photonics Journal.

[126]  Joost van de Weijer,et al.  Improving Color Constancy by Photometric Edge Weighting , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[127]  Christophe Sintes,et al.  CPHD-DOA Tracking of Multiple Extended Sonar Targets in Impulsive Environments , 2016, IEEE Transactions on Signal Processing.

[128]  Tie Qiu,et al.  A Review on Intelligence Dehazing and Color Restoration for Underwater Images , 2020, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[129]  Huimin Lu,et al.  Underwater image dehazing using joint trilateral filter , 2014, Comput. Electr. Eng..

[130]  Wen Gao,et al.  Single underwater image enhancement with a new optical model , 2013, 2013 IEEE International Symposium on Circuits and Systems (ISCAS2013).

[131]  Li Qin,et al.  Single Image Defogging Based on Multi-Channel Convolutional MSRCR , 2019, IEEE Access.

[132]  Yong-Jie Li,et al.  Underwater Image Enhancement Using Adaptive Retinal Mechanisms , 2019, IEEE Transactions on Image Processing.