Haze concentration adaptive network for image dehazing

Abstract Learning-based methods have attracted considerable interest in image dehazing. However, most existing methods are not well adapted to different hazy conditions, especially when dealing with the heavily hazy scene. There is often a significant amount of haze that remains in the images recovered by most methods. To address this issue, we propose an end-to-end Haze Concentration Adaptive Network (HCAN), including a pyramid feature extractor (PFE), a feature enhancement module (FEM), and a multi-scale feature attention module (MSFAM) for image dehazing. Specifically, PFE based on the feature pyramid structure leverages complementary features from different CNN layers to help the clear image prediction. Then, FEM fuses four kinds of images with different haze density (i.e., three recovered images in the FEM with light haze density, and the input hazy image with strong haze condition) to guide the network to adaptively perceive images under different haze conditions. Finally, MSFAM is designed under two principles, multi-scale structure and attention mechanism. It is used to help the network produce a clear image with more details, and ease the network training. Comprehensive experiments demonstrate that the proposed HCAN performs favorably against the state-of-the-art methods in terms of PSNR, SSIM, and visual effect. The results, per-trained models and code are available at https://github.com/TaoWangzj/HCAN .

[1]  Kaiming He,et al.  Feature Pyramid Networks for Object Detection , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[2]  Shai Avidan,et al.  Non-local Image Dehazing , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[3]  Radu Timofte,et al.  O-HAZE: A Dehazing Benchmark with Real Hazy and Haze-Free Outdoor Images , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[4]  Li Fei-Fei,et al.  Perceptual Losses for Real-Time Style Transfer and Super-Resolution , 2016, ECCV.

[5]  Minh N. Do,et al.  Robust Image and Video Dehazing with Visual Artifact Suppression via Gradient Residual Minimization , 2016, ECCV.

[6]  Gang Hua,et al.  Gated Context Aggregation Network for Image Dehazing and Deraining , 2018, 2019 IEEE Winter Conference on Applications of Computer Vision (WACV).

[7]  Rob Fergus,et al.  Visualizing and Understanding Convolutional Networks , 2013, ECCV.

[8]  Dani Lischinski,et al.  Deep photo: model-based photograph enhancement and viewing , 2008, SIGGRAPH 2008.

[9]  Wei Liu,et al.  Gated Fusion Network for Single Image Dehazing , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[10]  Robby T. Tan,et al.  Visibility in bad weather from a single image , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[11]  Radu Timofte,et al.  I-HAZE: a dehazing benchmark with real hazy and haze-free indoor images , 2018, ACIVS.

[12]  Zhe Liu,et al.  Single image dehazing using kernel regression model and dark channel prior , 2017, Signal Image Video Process..

[13]  Shree K. Nayar,et al.  Contrast Restoration of Weather Degraded Images , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[14]  Richard Szeliski,et al.  High-accuracy stereo depth maps using structured light , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[15]  Wenhan Yang,et al.  Dmcnn: Dual-Domain Multi-Scale Convolutional Neural Network for Compression Artifacts Removal , 2018, 2018 25th IEEE International Conference on Image Processing (ICIP).

[16]  Zhengguo Li,et al.  Edge-Preserving Decomposition-Based Single Image Haze Removal , 2015, IEEE Transactions on Image Processing.

[17]  Derek Hoiem,et al.  Indoor Segmentation and Support Inference from RGBD Images , 2012, ECCV.

[18]  Bobby Bodenheimer,et al.  Synthesis and evaluation of linear motion transitions , 2008, TOGS.

[19]  Ko Nishino,et al.  Bayesian Defogging , 2012, International Journal of Computer Vision.

[20]  Shree K. Nayar,et al.  Instant dehazing of images using polarization , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[21]  Christophe De Vleeschouwer,et al.  D-HAZY: A dataset to evaluate quantitatively dehazing algorithms , 2016, 2016 IEEE International Conference on Image Processing (ICIP).

[22]  Raanan Fattal Single image dehazing , 2008, SIGGRAPH 2008.

[23]  Mohinder Malhotra Single Image Haze Removal Using Dark Channel Prior , 2016 .

[24]  Ling Shao,et al.  A Fast Single Image Haze Removal Algorithm Using Color Attenuation Prior , 2015, IEEE Transactions on Image Processing.

[25]  Xiaoqin Zhang,et al.  Pyramid Channel-based Feature Attention Network for image dehazing , 2020, Comput. Vis. Image Underst..

[26]  Wangmeng Zuo,et al.  Attention-guided CNN for image denoising , 2020, Neural Networks.

[27]  Raanan Fattal,et al.  Dehazing Using Color-Lines , 2014, ACM Trans. Graph..

[28]  Radu Timofte,et al.  Dense-Haze: A Benchmark for Image Dehazing with Dense-Haze and Haze-Free Images , 2019, 2019 IEEE International Conference on Image Processing (ICIP).

[29]  Gaofeng Meng,et al.  Efficient Image Dehazing with Boundary Constraint and Contextual Regularization , 2013, 2013 IEEE International Conference on Computer Vision.

[30]  Vishal M. Patel,et al.  Multi-scale Single Image Dehazing Using Perceptual Pyramid Deep Network , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[31]  Xiaochun Cao,et al.  Single Image Dehazing via Multi-scale Convolutional Neural Networks , 2016, ECCV.

[32]  Jun Chen,et al.  GridDehazeNet: Attention-Based Multi-Scale Network for Image Dehazing , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[33]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[34]  Dan Feng,et al.  Benchmarking Single-Image Dehazing and Beyond , 2017, IEEE Transactions on Image Processing.

[35]  Jean-Philippe Tarel,et al.  Fast visibility restoration from a single color or gray level image , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[36]  Dacheng Tao,et al.  DehazeNet: An End-to-End System for Single Image Haze Removal. , 2016, IEEE transactions on image processing : a publication of the IEEE Signal Processing Society.

[37]  Yanyun Qu,et al.  Enhanced Pix2pix Dehazing Network , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[38]  Xin Zhao,et al.  Single image dehazing based on fusion strategy , 2020, Neurocomputing.

[39]  Jizheng Xu,et al.  End-to-End United Video Dehazing and Detection , 2017, AAAI.

[40]  Ephraim Speech enhancement using a minimum mean square error short-time spectral amplitude estimator , 1984 .

[41]  Jizheng Xu,et al.  AOD-Net: All-in-One Dehazing Network , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[42]  Pheng-Ann Heng,et al.  Deep Multi-Model Fusion for Single-Image Dehazing , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[43]  Zhengyuan Zhou,et al.  Robust Low-Rank Tensor Recovery with Rectification and Alignment , 2021, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[44]  Jinhui Tang,et al.  Single Image Dehazing via Conditional Generative Adversarial Network , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[45]  Sehoon Ha,et al.  Iterative Training of Dynamic Skills Inspired by Human Coaching Techniques , 2014, ACM Trans. Graph..

[46]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[47]  Shree K. Nayar,et al.  Chromatic framework for vision in bad weather , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[48]  Hujun Bao,et al.  A Robust Tracking System for Low Frame Rate Video , 2015, International Journal of Computer Vision.

[49]  Danping Zou,et al.  Simultaneous video defogging and stereo reconstruction , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).