Learning Hadamard-Product-Propagation for Image Dehazing and Beyond

Image dehazing has evolved into an attractive research field in the computer vision community in the past few decades. Previous traditional approaches attempt to design energy-based objective functions. However, they cannot accurately express the intrinsic characteristics of the images, posing weak adaptation ability for real-world complex scenarios. More recently, deep learning techniques for image dehazing have matured and become more reliable, showing outstanding performance. Nevertheless, these methods heavily depend on training data, restricting their application ranges. More importantly, both traditional and deep learning approaches all ignore a common issue, noises/artifacts always appear in the recovery process. To this end, a new Hadamard-Product (HP) model is proposed, which consists of a series of data-driven priors. Based on this model, we derive a Learnable Hadamard-Product-Propagation (LHPP) by cascading a series of principle-inspired guidance and recovery modules. In which, the principle-inspired guidance related to transmission is endowed the smoothness property, the other recovery module satisfies the distribution of natural images. The Hadamard-product-based propagations is generated in our developed learnable framework for the task of image dehazing. In this way, we can eliminate noises/artifacts in the recovery procedure to obtain the ideal outputs. Subsequently, since the generality of our HP model, we successfully extend our LHPP to settle low-light image enhancement and underwater image enhancement problems. A series of analytical experiments are performed to verify our effectiveness. Plenty of performance evaluations on three complex tasks fully reveal our superiority against multiple state-of-the-art methods.

[1]  Di Guo,et al.  Wavelet-based edge correlation incorporated iterative reconstruction for undersampled MRI. , 2011, Magnetic resonance imaging.

[2]  Xiao-Ping Zhang,et al.  A Probabilistic Method for Image Enhancement With Simultaneous Illumination and Reflectance Estimation , 2015, IEEE Transactions on Image Processing.

[3]  Hua Huang,et al.  No-reference image quality assessment based on spatial and spectral entropies , 2014, Signal Process. Image Commun..

[4]  Soumik Sarkar,et al.  LLNet: A deep autoencoder approach to natural low-light image enhancement , 2015, Pattern Recognit..

[5]  Kai Zeng,et al.  Perceptual Quality Assessment for Multi-Exposure Image Fusion , 2015, IEEE Transactions on Image Processing.

[6]  H. D. Cheng,et al.  A simple and effective histogram equalization approach to image enhancement , 2004, Digit. Signal Process..

[7]  Donald Geman,et al.  Nonlinear image recovery with half-quadratic regularization , 1995, IEEE Trans. Image Process..

[8]  Jia Xu,et al.  Learning to See in the Dark , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[9]  Xiaochun Cao,et al.  Learning Interleaved Cascade of Shrinkage Fields for Joint Image Dehazing and Denoising , 2020, IEEE Transactions on Image Processing.

[10]  Dacheng Tao,et al.  A Joint Intrinsic-Extrinsic Prior Model for Retinex , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[11]  Madhu S. Nair,et al.  Wavelet Based Sharp Features (WASH): An Image Quality Assessment Metric Based on HVS , 2013, 2013 2nd International Conference on Advanced Computing, Networking and Security.

[12]  Xiaochun Cao,et al.  Single Image Dehazing via Multi-scale Convolutional Neural Networks , 2016, ECCV.

[13]  Ming Wang,et al.  Joint Coding-Transmission Optimization for a Video Surveillance System With Multiple Cameras , 2018, IEEE Transactions on Multimedia.

[14]  Codruta O. Ancuti,et al.  Enhancing underwater images and videos by fusion , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[15]  Qing Zhang,et al.  High-Quality Exposure Correction of Underexposed Photos , 2018, ACM Multimedia.

[16]  Zia-ur Rahman,et al.  Retinex processing for automatic image enhancement , 2002, IS&T/SPIE Electronic Imaging.

[17]  Xiao-Ping Zhang,et al.  A Weighted Variational Model for Simultaneous Reflectance and Illumination Estimation , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[18]  David J. Kriegman,et al.  Learning to See Through Turbulent Water , 2018, 2018 IEEE Winter Conference on Applications of Computer Vision (WACV).

[19]  Minh N. Do,et al.  Robust Image and Video Dehazing with Visual Artifact Suppression via Gradient Residual Minimization , 2016, ECCV.

[20]  N Carlevaris-Bianco,et al.  Initial results in underwater single image dehazing , 2010, OCEANS 2010 MTS/IEEE SEATTLE.

[21]  Jinhui Tang,et al.  Single Image Dehazing via Conditional Generative Adversarial Network , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[22]  Michael Elad,et al.  A Statistical Prediction Model Based on Sparse Representations for Single Image Super-Resolution , 2014, IEEE Transactions on Image Processing.

[23]  Yi He,et al.  On the Convergence of Learning-Based Iterative Methods for Nonconvex Inverse Problems , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[24]  Jie Li,et al.  WaterGAN: Unsupervised Generative Network to Enable Real-Time Color Correction of Monocular Underwater Images , 2017, IEEE Robotics and Automation Letters.

[25]  Ming-Hsuan Yang,et al.  Single Image Dehazing via Multi-scale Convolutional Neural Networks with Holistic Edges , 2019, International Journal of Computer Vision.

[26]  Delu Zeng,et al.  A fusion-based enhancing method for weakly illuminated images , 2016, Signal Process..

[27]  Shai Avidan,et al.  Non-local Image Dehazing , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[28]  Göran Salomonsson,et al.  Image enhancement based on a nonlinear multiscale method , 1997, IEEE Trans. Image Process..

[29]  Jizheng Xu,et al.  AOD-Net: All-in-One Dehazing Network , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[30]  Haojie Li,et al.  Robust Haze Removal Via Joint Deep Transmission and Scene Propagation , 2018, 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[31]  A. Cantor Optics of the atmosphere--Scattering by molecules and particles , 1978, IEEE Journal of Quantum Electronics.

[32]  Ying-Ching Chen,et al.  Underwater Image Enhancement by Wavelength Compensation and Dehazing , 2012, IEEE Transactions on Image Processing.

[33]  Dacheng Tao,et al.  DehazeNet: An End-to-End System for Single Image Haze Removal , 2016, IEEE Transactions on Image Processing.

[34]  Arcot Sowmya,et al.  An Underwater Color Image Quality Evaluation Metric , 2015, IEEE Transactions on Image Processing.

[35]  Seong G. Kong,et al.  An Iterative Image Dehazing Method With Polarization , 2019, IEEE Transactions on Multimedia.

[36]  Dan Feng,et al.  Benchmarking Single-Image Dehazing and Beyond , 2017, IEEE Transactions on Image Processing.

[37]  Xin Yu,et al.  Object Tracking With Multi-View Support Vector Machines , 2015, IEEE Transactions on Multimedia.

[38]  S. Avidan,et al.  Diving into Haze-Lines: Color Restoration of Underwater Images , 2017 .

[39]  Manjunatha Mahadevappa,et al.  Brightness preserving dynamic fuzzy histogram equalization , 2010, IEEE Transactions on Consumer Electronics.

[40]  Mohinder Malhotra Single Image Haze Removal Using Dark Channel Prior , 2016 .

[41]  Michael Werman,et al.  Automatic recovery of the atmospheric light in hazy images , 2014, 2014 IEEE International Conference on Computational Photography (ICCP).

[42]  Derek Hoiem,et al.  Indoor Segmentation and Support Inference from RGBD Images , 2012, ECCV.

[43]  Raanan Fattal,et al.  Dehazing Using Color-Lines , 2014, ACM Trans. Graph..

[44]  Lei Zhang,et al.  Learning Aggregated Transmission Propagation Networks for Haze Removal and Beyond , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[45]  Michael S. Brown,et al.  Globally Optimized Linear Windowed Tone Mapping , 2010, IEEE Transactions on Visualization and Computer Graphics.

[46]  Fei-Fei Li,et al.  ImageNet: A large-scale hierarchical image database , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[47]  Wei Liu,et al.  Gated Fusion Network for Single Image Dehazing , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[48]  Wangmeng Zuo,et al.  Learning Deep CNN Denoiser Prior for Image Restoration , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[49]  Jonathan T. Barron,et al.  Deep bilateral learning for real-time image enhancement , 2017, ACM Trans. Graph..

[50]  Lei Zhang,et al.  FFDNet: Toward a Fast and Flexible Solution for CNN-Based Image Denoising , 2017, IEEE Transactions on Image Processing.

[51]  Gaofeng Meng,et al.  Efficient Image Dehazing with Boundary Constraint and Contextual Regularization , 2013, 2013 IEEE International Conference on Computer Vision.

[52]  Michael S. Brown,et al.  A Contrast Enhancement Framework with JPEG Artifacts Suppression , 2014, ECCV.

[53]  Hai-Miao Hu,et al.  Naturalness Preserved Enhancement Algorithm for Non-Uniform Illumination Images , 2013, IEEE Transactions on Image Processing.

[54]  Xiaoyan Sun,et al.  Structure-Revealing Low-Light Image Enhancement Via Robust Retinex Model , 2018, IEEE Transactions on Image Processing.

[55]  Yunjin Chen,et al.  Trainable Nonlinear Reaction Diffusion: A Flexible Framework for Fast and Effective Image Restoration , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[56]  Chul Lee,et al.  Contrast Enhancement Based on Layered Difference Representation of 2D Histograms , 2013, IEEE Transactions on Image Processing.

[57]  Yu Li,et al.  LIME: Low-Light Image Enhancement via Illumination Map Estimation , 2017, IEEE Transactions on Image Processing.

[58]  Vishal M. Patel,et al.  Densely Connected Pyramid Dehazing Network , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[59]  Adrian Galdran,et al.  Automatic Red-Channel underwater image restoration , 2015, J. Vis. Commun. Image Represent..