UCRNet: Underwater color image restoration via a polarization-guided convolutional neural network

Underwater images always suffer from low contrast and color distortion due to the wavelength-dependent scattering and absorption effects caused by particles existing in turbid water, especially in high turbidity conditions. Based on the polarization properties of the backscattering light, polarimetric methods can estimate the intensity level of the backscattering and the transmittance of the media. Accordingly, they can separate the target signal from the undesired ones to achieve high-quality imaging. In addition, learning-based polarimetric methods are effective for gray-model image restoration, but the learning-based polarimetric technique for color image restoration has yet to be considered. In this paper, we propose a 3- dimensional convolutional neural network, which maintains the correlation of polarization information among different polarization channel images as well as embodies polarization constraints, for underwater color image restoration. The experimental results verify that the proposed solution improves the image quality (i.e., the image contrast, details, and color) and outperforms other existing methods, especially when the turbidity of scattering media is high. The proposed solution can be readily applied to practical applications and potentially realize the clear vision in other scattering media, including biomedical imaging and remote sensing.

[1]  S. Kwong,et al.  Underwater Image Enhancement via Minimal Color Loss and Locally Adaptive Contrast Enhancement , 2022, IEEE Transactions on Image Processing.

[2]  Tiegen Liu,et al.  Physics-informed neural network for polarimetric underwater imaging. , 2022, Optics express.

[3]  Haofeng Hu,et al.  Attention-based neural network for polarimetric image denoising. , 2022, Optics letters.

[4]  Xiaobo Li,et al.  Underwater image restoration via Stokes decomposition. , 2022, Optics letters.

[5]  Rongguang Liang,et al.  Color polarization demosaicking by a convolutional neural network. , 2021, Optics letters.

[6]  Liyong Ren,et al.  Low-pass filtering based polarimetric dehazing method for dense haze removal. , 2021, Optics express.

[7]  Xiaopeng Shao,et al.  Enhancement of underwater vision by fully exploiting the polarization information from the Stokes vector. , 2021, Optics express.

[8]  Tiegen Liu,et al.  Underwater imaging enhancement based on a polarization filter and histogram attenuation prior , 2021 .

[9]  Yang Lin,et al.  IPLNet: a neural network for intensity-polarization imaging in low light. , 2020, Optics letters.

[10]  Xiaobo Li,et al.  Polarimetric underwater image recovery via deep learning , 2020 .

[11]  Euclid Seeram,et al.  Digital Image Processing Concepts , 2020, Digital Radiography.

[12]  Yang Lin,et al.  Learning-based denoising for polarimetric images. , 2020, Optics express.

[13]  Kui Yang,et al.  Polarization-based exploration for clear underwater vision in natural illumination. , 2019, Optics express.

[14]  S. Kwong,et al.  An Underwater Image Enhancement Benchmark Dataset and Beyond , 2019, IEEE Transactions on Image Processing.

[15]  Tiegen Liu,et al.  Polarimetric image recovery method combining histogram stretching for underwater imaging , 2018, Scientific Reports.

[16]  Yun Fu,et al.  Residual Dense Network for Image Super-Resolution , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[17]  Lin Zhao,et al.  Underwater Image Recovery Under the Nonuniform Optical Field Based on Polarimetric Imaging , 2018, IEEE Photonics Journal.

[18]  Chunle Guo,et al.  Emerging From Water: Underwater Image Color Correction Based on Weakly Supervised Color Transfer , 2017, IEEE Signal Processing Letters.

[19]  Jizheng Xu,et al.  AOD-Net: All-in-One Dehazing Network , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[20]  Matthew Johnson-Roberson,et al.  Underwater Image Dehazing with a Light Field Camera , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[21]  Andrew Zisserman,et al.  Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[22]  C. Schmid,et al.  Long-Term Temporal Convolutions for Action Recognition , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[23]  Li Fei-Fei,et al.  Perceptual Losses for Real-Time Style Transfer and Super-Resolution , 2016, ECCV.

[24]  Jian Liang,et al.  Polarimetric dehazing method for dense haze removal based on distribution analysis of angle of polarization. , 2015, Optics express.

[25]  Jian Sun,et al.  Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[26]  Raanan Fattal,et al.  Dehazing Using Color-Lines , 2014, ACM Trans. Graph..

[27]  A Dogariu,et al.  Exploring underwater target detection by imaging polarimetry and correlation techniques. , 2013, Applied optics.

[28]  Codruta O. Ancuti,et al.  Enhancing underwater images and videos by fusion , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[29]  A. De Martino,et al.  Contrast evaluation of the polarimetric images of different targets in turbid medium: possible sources of systematic errors , 2010, Defense + Commercial Sensing.

[30]  Tatiana Novikova,et al.  Sources of possible artefacts in the contrast evaluation for the backscattering polarimetric images of different targets in turbid medium. , 2009, Optics express.

[31]  Yonghong He,et al.  Contrast enhancement subsurface optical imaging with different incident polarization states , 2006, International Conference on Photonics and Imaging in Biology and Medicine.

[32]  Gang Yao,et al.  Effects of turbid media optical properties on object visibility in subsurface polarization imaging. , 2006, Applied optics.

[33]  R R Alfano,et al.  Circular polarization memory of light. , 2005, Physical review. E, Statistical, nonlinear, and soft matter physics.

[34]  J Cariou,et al.  Backscattered speckle size as a function of polarization: influence of particle-size and- concentration. , 2005, Optics express.

[35]  Y. Schechner,et al.  Clear underwater vision , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[36]  Shree K. Nayar,et al.  Polarization-based vision through haze , 2003 .

[37]  P. Roberts,et al.  Backscattering target detection in a turbid medium by polarization discrimination. , 1999, Applied optics.

[38]  Yafei Wang,et al.  Multi-polarization fusion generative adversarial networks for clear underwater imaging , 2022, Optics and Lasers in Engineering.

[39]  Tiegen Liu,et al.  U2R-pGAN: Unpaired underwater-image recovery with polarimetric generative adversarial network , 2022, Optics and Lasers in Engineering.

[40]  Boxin Shi,et al.  Learning to dehaze with polarization , 2021, NeurIPS.

[41]  Mohinder Malhotra Single Image Haze Removal Using Dark Channel Prior , 2016 .