An efficient nonlocal variational method with application to underwater image restoration

Abstract Light is absorbed and scattered when it travels though water, which causes the captured underwater images often suffering from blurring, low contrast and color degradation. To overcome these problems, we propose a novel variational model based on nonlocal differential operators, in which the underwater image formation model is successfully integrated into the variational framework. The underwater dark channel prior (UDCP) and quad-tree subdivision methods are applied to the construction of underwater image formation model to estimate the transmission map and the global background light. Furthermore, we employ a fast algorithm based on the alternating direction method of multipliers (ADMM) to speed up the solving procedure. To reproduce color saturation, we perform a Gamma correction operation on the recovered image. Both the real underwater image application test and the simulation experiment demonstrate that the proposed underwater nonlocal total variational (UNLTV) approach achieves superb performance on dehazing, denoising, and improving the visibility of underwater images. In addition, six state-of-the-art algorithms are compared under different challenging scenes to evaluate their effectiveness and robustness. Extensive qualitative and quantitative experimental comparisons further validate the superiority of the proposed method.

[1]  Huimin Lu,et al.  Underwater image dehazing using joint trilateral filter , 2014, Comput. Electr. Eng..

[2]  Li Bai,et al.  Novel Methods for Microglia Segmentation, Feature Extraction, and Classification , 2017, IEEE/ACM Transactions on Computational Biology and Bioinformatics.

[3]  Guojia Hou,et al.  Single image dehazing and denoising combining dark channel prior and variational models , 2017, IET Comput. Vis..

[4]  Li Bai,et al.  Introducing diffusion tensor to high order variational model for image reconstruction , 2017, Digit. Signal Process..

[5]  Wen Gao,et al.  Single underwater image enhancement with a new optical model , 2013, 2013 IEEE International Symposium on Circuits and Systems (ISCAS2013).

[6]  Guojia Hou,et al.  Underwater Color Image Enhancement Using Combining Schemes , 2014 .

[7]  Uche Afam Nnolim Improved partial differential equation-based enhancement for underwater images using local-global contrast operators and fuzzy homomorphic processes , 2017, IET Image Process..

[8]  Codruta O. Ancuti,et al.  Color Balance and Fusion for Underwater Image Enhancement , 2018, IEEE Transactions on Image Processing.

[9]  Jichang Guo,et al.  Underwater image enhancement by dehazing and color correction , 2015, J. Electronic Imaging.

[10]  Lan Tang,et al.  Non-Convex Weighted Lp Nuclear Norm based ADMM Framework for Image Restoration , 2017, 1704.07056.

[11]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[12]  Mario Fernando Montenegro Campos,et al.  Underwater Depth Estimation and Image Restoration Based on Single Images , 2016, IEEE Computer Graphics and Applications.

[13]  Zhou Wang,et al.  Image Quality Assessment: From Error Measurement to Structural Similarity , 2004 .

[14]  Tieyong Zeng,et al.  Single Image Dehazing and Denoising: A Fast Variational Approach , 2014, SIAM J. Imaging Sci..

[15]  Chunle Guo,et al.  Emerging From Water: Underwater Image Color Correction Based on Weakly Supervised Color Transfer , 2017, IEEE Signal Processing Letters.

[16]  Guodong Wang,et al.  Nonlocal active contour model for texture segmentation , 2017, Multimedia Tools and Applications.

[17]  Chen Gao,et al.  Human-Visual-System-Inspired Underwater Image Quality Measures , 2016, IEEE Journal of Oceanic Engineering.

[18]  Ting Wang,et al.  Underwater image enhancement via extended multi-scale Retinex , 2017, Neurocomputing.

[19]  Runmin Cong,et al.  A hybrid method for underwater image correction , 2017, Pattern Recognit. Lett..

[20]  B. L. McGlamery,et al.  A Computer Model For Underwater Camera Systems , 1980, Other Conferences.

[21]  Arcot Sowmya,et al.  An Underwater Color Image Quality Evaluation Metric , 2015, IEEE Transactions on Image Processing.

[22]  Nor Ashidi Mat Isa,et al.  Underwater image quality enhancement through integrated color model with Rayleigh distribution , 2015, Appl. Soft Comput..

[23]  Pamela C. Cosman,et al.  Underwater Image Restoration Based on Image Blurriness and Light Absorption , 2017, IEEE Transactions on Image Processing.

[24]  Tony F. Chan,et al.  Total Variation Wavelet Inpainting , 2006, Journal of Mathematical Imaging and Vision.

[25]  Remco C. Veltkamp,et al.  A combined post-filtering method to improve accuracy of variational optical flow estimation , 2014, Pattern Recognit..

[26]  Rui Zhang,et al.  Global sparse gradient guided variational Retinex model for image enhancement , 2017, Signal Process. Image Commun..

[27]  Javier Vazquez-Corral,et al.  Fusion-Based Variational Image Dehazing , 2017, IEEE Signal Processing Letters.

[28]  Adrian Galdran,et al.  Automatic Red-Channel underwater image restoration , 2015, J. Vis. Commun. Image Represent..

[29]  Xiu Li,et al.  Underwater image enhancement via dark channel prior and luminance adjustment , 2016, OCEANS 2016 - Shanghai.

[30]  Jian Sun,et al.  A tensor-based nonlocal total variation model for multi-channel image recovery , 2018, Signal Process..

[31]  Jean-Philippe Tarel,et al.  BLIND CONTRAST ENHANCEMENT ASSESSMENT BY GRADIENT RATIOING AT VISIBLE EDGES , 2011 .

[32]  Wensheng Zhang,et al.  A total variation based nonrigid image registration by combining parametric and non-parametric transformation models , 2014, Neurocomputing.

[33]  Xi Qiao,et al.  Underwater image quality enhancement of sea cucumbers based on improved histogram equalization and wavelet transform , 2017 .

[34]  Wenhao Zhang,et al.  A new underwater image enhancing method via color correction and illumination adjustment , 2017, 2017 IEEE Visual Communications and Image Processing (VCIP).

[35]  Wanquan Liu,et al.  Fast algorithm for color texture image inpainting using the non-local CTV model , 2015, J. Glob. Optim..

[36]  L. Neumann,et al.  a New Color Correction Method for Underwater Imaging , 2015 .

[37]  Michael J. Black,et al.  Secrets of optical flow estimation and their principles , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[38]  Qiguang Miao,et al.  Color texture segmentation based on active contour model with multichannel nonlocal and Tikhonov regularization , 2017, Multimedia Tools and Applications.

[39]  Ying-Ching Chen,et al.  Underwater Image Enhancement by Wavelength Compensation and Dehazing , 2012, IEEE Transactions on Image Processing.

[40]  Guang Jin,et al.  A Variational Model for Sea Image Enhancement , 2018, Remote. Sens..

[41]  N Carlevaris-Bianco,et al.  Initial results in underwater single image dehazing , 2010, OCEANS 2010 MTS/IEEE SEATTLE.

[42]  Guy Gilboa,et al.  Nonlocal Operators with Applications to Image Processing , 2008, Multiscale Model. Simul..

[43]  Runmin Cong,et al.  Underwater Image Enhancement by Dehazing With Minimum Information Loss and Histogram Distribution Prior , 2016, IEEE Transactions on Image Processing.

[44]  Richard G. Baraniuk,et al.  Fast Alternating Direction Optimization Methods , 2014, SIAM J. Imaging Sci..

[45]  Wei Xie,et al.  Variational method for joint optical flow estimation and edge-aware image restoration , 2017, Pattern Recognit..

[46]  Jian-Ru Lin,et al.  Underwater image enhancement method using weighted guided trigonometric filtering and artificial light correction , 2016, J. Vis. Commun. Image Represent..

[47]  Michael Elad,et al.  A Variational Framework for Retinex , 2002, IS&T/SPIE Electronic Imaging.

[48]  Raimondo Schettini,et al.  Underwater Image Processing: State of the Art of Restoration and Image Enhancement Methods , 2010, EURASIP J. Adv. Signal Process..

[49]  Jun Huang,et al.  Gaussian field estimator with manifold regularization for retinal image registration , 2019, Signal Process..

[50]  Zhang Yi,et al.  A Fractional-Order Variational Framework for Retinex: Fractional-Order Partial Differential Equation-Based Formulation for Multi-Scale Nonlocal Contrast Enhancement with Texture Preserving , 2018, IEEE Transactions on Image Processing.

[51]  Silvia Silva da Costa Botelho,et al.  Transmission Estimation in Underwater Single Images , 2013, 2013 IEEE International Conference on Computer Vision Workshops.

[52]  Chuanjiang He,et al.  Indirectly regularized variational level set model for image segmentation , 2016, Neurocomputing.

[53]  Jules S. Jaffe,et al.  Computer modeling and the design of optimal underwater imaging systems , 1990 .

[54]  Jian Sun,et al.  Single image haze removal using dark channel prior , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[55]  Guojia Hou,et al.  Efficient L1-based nonlocal total variational model of Retinex for image restoration , 2018, J. Electronic Imaging.