Pyramid Global Context Network for Image Dehazing

Haze caused by atmospheric scattering and absorption would severely affect scene visibility of an image. Thus, image dehazing for haze removal has been widely studied in the literature. Within a hazy image, haze is not confined in a small local patch/position, while widely diffusing in a whole image. Under this circumstance, global context is a crucial factor in the success of dehazing, which was seldom investigated in existing dehazing algorithms. In the literature, the global context (GC) block has been designed to learn point-wise long-range dependencies of an image for global context modeling; however, patch-wise long-range dependencies were ignored. To image dehazing, patch-wise long-range dependencies should be highlighted to cooperate with patch-wise operations of image dehazing. In this paper, we first extend the point-wise GC into a Pyramid Global Context (PGC), which is a multi-scale GC, after undergoing the pyramid pooling. Thus, patch-wise long-range dependencies can be explored by the PGC. Then, the proposed PGC is plugged into a U-Net, getting an attentive U-Net. Further, the attentive U-Net is optimized by importing ResNet’s shortcut connection and dilated convolution. Thus, the finalized dehazing model can explore both long-range and patch-wise context dependencies for global context modeling, which is crucial for image dehazing. The extensive experiments on synthetic databases and real-world hazy images demonstrate the superiority of our model over other representative state-of-the-art models from both quantitative and qualitative comparisons.

[1]  Shih-Chia Huang,et al.  DesnowGAN: An Efficient Single Image Snow Removal Framework Using Cross-Resolution Lateral Connection and GANs , 2021, IEEE Transactions on Circuits and Systems for Video Technology.

[2]  Shuicheng Yan,et al.  Joint Rain Detection and Removal from a Single Image with Contextualized Deep Networks , 2020, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[3]  Robby T. Tan,et al.  All in One Bad Weather Removal Using Architectural Search , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[4]  Zhixun Su,et al.  Learning Deep Priors for Image Dehazing , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[5]  Xiang Bai,et al.  Asymmetric Non-Local Neural Networks for Semantic Segmentation , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[6]  Jun Chen,et al.  GridDehazeNet: Attention-Based Multi-Scale Network for Image Dehazing , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[7]  Ling-Yu Duan,et al.  IDeRs: Iterative dehazing method for single remote sensing image , 2019, Inf. Sci..

[8]  Dacheng Tao,et al.  FAMED-Net: A Fast and Accurate Multi-Scale End-to-End Dehazing Network , 2019, IEEE Transactions on Image Processing.

[9]  Yanyun Qu,et al.  Enhanced Pix2pix Dehazing Network , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[10]  Shu-Tao Xia,et al.  Second-Order Attention Network for Single Image Super-Resolution , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[11]  Ling-Yu Duan,et al.  Multi-scale Optimal Fusion model for single image dehazing , 2019, Signal Process. Image Commun..

[12]  Stephen Lin,et al.  GCNet: Non-Local Networks Meet Squeeze-Excitation Networks and Beyond , 2019, 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW).

[13]  Loong Fah Cheong,et al.  Heavy Rain Image Restoration: Integrating Physics Model and Conditional Adversarial Learning , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[14]  Xiaochun Cao,et al.  Deep Video Dehazing With Semantic Segmentation , 2019, IEEE Transactions on Image Processing.

[15]  Anna Wang,et al.  AIPNet: Image-to-Image Single Image Dehazing With Atmospheric Illumination Prior , 2019, IEEE Transactions on Image Processing.

[16]  Shuicheng Yan,et al.  Graph-Based Global Reasoning Networks , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[17]  Xinghao Ding,et al.  A2Net: Adjacent Aggregation Networks for Image Raindrop Removal , 2018, IEEE Access.

[18]  Errui Ding,et al.  Compact Generalized Non-local Network , 2018, NeurIPS.

[19]  Fan Guo,et al.  Parameter Selection of Image Fog Removal Using Artificial Fish Swarm Algorithm , 2018, ICIC.

[20]  Liang Lin,et al.  Non-locally Enhanced Encoder-Decoder Network for Single Image De-raining , 2018, ACM Multimedia.

[21]  Hongdong Li,et al.  Semantic Single-Image Dehazing , 2018, ArXiv.

[22]  Radu Timofte,et al.  I-HAZE: a dehazing benchmark with real hazy and haze-free indoor images , 2018, ACIVS.

[23]  Radu Timofte,et al.  O-HAZE: A Dehazing Benchmark with Real Hazy and Haze-Free Outdoor Images , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[24]  Wei Liu,et al.  Gated Fusion Network for Single Image Dehazing , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[25]  Vishal M. Patel,et al.  Densely Connected Pyramid Dehazing Network , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[26]  Wonha Kim,et al.  Single Image Dehazing Using Color Ellipsoid Prior , 2018, IEEE Transactions on Image Processing.

[27]  Dan Feng,et al.  Benchmarking Single-Image Dehazing and Beyond , 2017, IEEE Transactions on Image Processing.

[28]  Jan Kautz,et al.  High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANs , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[29]  Abhinav Gupta,et al.  Non-local Neural Networks , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[30]  Jiri Matas,et al.  DeblurGAN: Blind Motion Deblurring Using Conditional Adversarial Networks , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[31]  Jizheng Xu,et al.  AOD-Net: All-in-One Dehazing Network , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[32]  Enhua Wu,et al.  Squeeze-and-Excitation Networks , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[33]  Gaurav Sharma,et al.  HazeRD: An outdoor scene dataset and benchmark for single image dehazing , 2017, 2017 IEEE International Conference on Image Processing (ICIP).

[34]  Alain Trémeau,et al.  Residual Conv-Deconv Grid Network for Semantic Segmentation , 2017, BMVC.

[35]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[36]  Thomas A. Funkhouser,et al.  Dilated Residual Networks , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[37]  Serge J. Belongie,et al.  Arbitrary Style Transfer in Real-Time with Adaptive Instance Normalization , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[38]  Nanning Zheng,et al.  Haze Removal Using the Difference- Structure-Preservation Prior , 2017, IEEE Transactions on Image Processing.

[39]  Xiaogang Wang,et al.  Pyramid Scene Parsing Network , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[40]  Alexei A. Efros,et al.  Image-to-Image Translation with Conditional Adversarial Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[41]  Xiaochun Cao,et al.  Single Image Dehazing via Multi-scale Convolutional Neural Networks , 2016, ECCV.

[42]  Andrea Vedaldi,et al.  Instance Normalization: The Missing Ingredient for Fast Stylization , 2016, ArXiv.

[43]  Shai Avidan,et al.  Non-local Image Dehazing , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[44]  Dacheng Tao,et al.  DehazeNet: An End-to-End System for Single Image Haze Removal , 2016, IEEE Transactions on Image Processing.

[45]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[46]  Vladlen Koltun,et al.  Multi-Scale Context Aggregation by Dilated Convolutions , 2015, ICLR.

[47]  Ling Shao,et al.  A Fast Single Image Haze Removal Algorithm Using Color Attenuation Prior , 2015, IEEE Transactions on Image Processing.

[48]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[49]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[50]  Raanan Fattal,et al.  Dehazing Using Color-Lines , 2014, ACM Trans. Graph..

[51]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[52]  Xi Wang,et al.  High-Resolution Stereo Datasets with Subpixel-Accurate Ground Truth , 2014, GCPR.

[53]  Jian Sun,et al.  Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[54]  Codruta O. Ancuti,et al.  Single Image Dehazing by Multi-Scale Fusion , 2013, IEEE Transactions on Image Processing.

[55]  Derek Hoiem,et al.  Indoor Segmentation and Support Inference from RGBD Images , 2012, ECCV.

[56]  David Zhang,et al.  FSIM: A Feature Similarity Index for Image Quality Assessment , 2011, IEEE Transactions on Image Processing.

[57]  Michael F. Cohen,et al.  Deep photo: model-based photograph enhancement and viewing , 2008, SIGGRAPH Asia '08.

[58]  Raanan Fattal,et al.  Single image dehazing , 2008, ACM Trans. Graph..

[59]  Zhenyang Wu,et al.  Natural color image enhancement and evaluation algorithm based on human visual system , 2006, Comput. Vis. Image Underst..

[60]  Cordelia Schmid,et al.  Beyond Bags of Features: Spatial Pyramid Matching for Recognizing Natural Scene Categories , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[61]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[62]  Shree K. Nayar,et al.  Contrast Restoration of Weather Degraded Images , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[63]  Shree K. Nayar,et al.  Instant dehazing of images using polarization , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[64]  Shree K. Nayar,et al.  Chromatic framework for vision in bad weather , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[65]  Shree K. Nayar,et al.  Vision in bad weather , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[66]  Huib de Ridder,et al.  Perceptually optimal color reproduction , 1998, Electronic Imaging.

[67]  Xiaoshuai Sun,et al.  Strong Baseline for Single Image Dehazing with Deep Features and Instance Normalization , 2018, BMVC.

[68]  Mohinder Malhotra Single Image Haze Removal Using Dark Channel Prior , 2016 .

[69]  唐琎,et al.  Objective measurement for image defogging algorithms , 2014 .

[70]  Ric,et al.  BLIND CONTRAST ENHANCEMENT ASSESSMENT BY GRADIENT RATIOING AT VISIBLE EDGES , 2008 .