RefineDNet: A Weakly Supervised Refinement Framework for Single Image Dehazing

Haze-free images are the prerequisites of many vision systems and algorithms, and thus single image dehazing is of paramount importance in computer vision. In this field, prior-based methods have achieved initial success. However, they often introduce annoying artifacts to outputs because their priors can hardly fit all situations. By contrast, learning-based methods can generate more natural results. Nonetheless, due to the lack of paired foggy and clear outdoor images of the same scenes as training samples, their haze removal abilities are limited. In this work, we attempt to merge the merits of prior-based and learning-based approaches by dividing the dehazing task into two sub-tasks, i.e., visibility restoration and realness improvement. Specifically, we propose a two-stage weakly supervised dehazing framework, RefineDNet. In the first stage, RefineDNet adopts the dark channel prior to restore visibility. Then, in the second stage, it refines preliminary dehazing results of the first stage to improve realness via adversarial learning with unpaired foggy and clear images. To get more qualified results, we also propose an effective perceptual fusion strategy to blend different dehazing outputs. Extensive experiments corroborate that RefineDNet with the perceptual fusion has an outstanding haze removal capability and can also produce visually pleasing results. Even implemented with basic backbone networks, RefineDNet can outperform supervised dehazing approaches as well as other state-of-the-art methods on indoor and outdoor datasets. To make our results reproducible, relevant code and data are available at https://github.com/xiaofeng94/RefineDNet-for-dehazing.

[1]  Robby T. Tan,et al.  Visibility in bad weather from a single image , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[2]  Xiaoqin Zhang,et al.  Single Image Dehazing via Lightweight Multi-scale Networks , 2019, 2019 IEEE International Conference on Big Data (Big Data).

[3]  Raanan Fattal,et al.  Dehazing Using Color-Lines , 2014, ACM Trans. Graph..

[4]  Jinhui Tang,et al.  Single Image Dehazing via Conditional Generative Adversarial Network , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[5]  Gaofeng Meng,et al.  Efficient Image Dehazing with Boundary Constraint and Contextual Regularization , 2013, 2013 IEEE International Conference on Computer Vision.

[6]  Anna Wang,et al.  AIPNet: Image-to-Image Single Image Dehazing With Atmospheric Illumination Prior , 2019, IEEE Transactions on Image Processing.

[7]  Xinbo Gao,et al.  Single Image Dehazing With Depth-Aware Non-Local Total Variation Regularization , 2018, IEEE Transactions on Image Processing.

[8]  David Zhang,et al.  FSIM: A Feature Similarity Index for Image Quality Assessment , 2011, IEEE Transactions on Image Processing.

[9]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[10]  Mohinder Malhotra Single Image Haze Removal Using Dark Channel Prior , 2016 .

[11]  Ying Shen,et al.  Dehazing Evaluation: Real-World Benchmark Datasets, Criteria, and Baselines , 2020, IEEE Transactions on Image Processing.

[12]  Xiaochun Cao,et al.  Single Image Dehazing via Multi-scale Convolutional Neural Networks , 2016, ECCV.

[13]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[14]  Hongyu Li,et al.  VSI: A Visual Saliency-Induced Index for Perceptual Image Quality Assessment , 2014, IEEE Transactions on Image Processing.

[15]  Jun Chen,et al.  GridDehazeNet: Attention-Based Multi-Scale Network for Image Dehazing , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[16]  Lei Zhang,et al.  Gradient Magnitude Similarity Deviation: A Highly Efficient Perceptual Image Quality Index , 2013, IEEE Transactions on Image Processing.

[17]  Dacheng Tao,et al.  FAMED-Net: A Fast and Accurate Multi-Scale End-to-End Dehazing Network , 2019, IEEE Transactions on Image Processing.

[18]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[19]  Christian Ledig,et al.  Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[20]  Arnold W. M. Smeulders,et al.  Color and Scale: The Spatial Structure of Color Images , 2000, ECCV.

[21]  Andrea Vedaldi,et al.  Deep Image Prior , 2017, International Journal of Computer Vision.

[22]  Christophe De Vleeschouwer,et al.  D-HAZY: A dataset to evaluate quantitatively dehazing algorithms , 2016, 2016 IEEE International Conference on Image Processing (ICIP).

[23]  Ying Shen,et al.  Evaluation of Defogging: A Real-World Benchmark Dataset, A New Criterion and Baselines , 2019, 2019 IEEE International Conference on Multimedia and Expo (ICME).

[24]  Joo-Hwee Lim,et al.  DehazeGAN: When Image Dehazing Meets Differential Programming , 2018, IJCAI.

[25]  Jizheng Xu,et al.  AOD-Net: All-in-One Dehazing Network , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[26]  Dan Feng,et al.  Benchmarking Single-Image Dehazing and Beyond , 2017, IEEE Transactions on Image Processing.

[27]  Xi Wang,et al.  High-Resolution Stereo Datasets with Subpixel-Accurate Ground Truth , 2014, GCPR.

[28]  Ling Shao,et al.  A Fast Single Image Haze Removal Algorithm Using Color Attenuation Prior , 2015, IEEE Transactions on Image Processing.

[29]  Alexei A. Efros,et al.  Image-to-Image Translation with Conditional Adversarial Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[30]  Wei Liu,et al.  Gated Fusion Network for Single Image Dehazing , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[31]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[32]  W. Middleton,et al.  Vision Through the Atmosphere , 1952 .

[33]  Jiebo Luo,et al.  Towards Perceptual Image Dehazing by Physics-Based Disentanglement and Adversarial Training , 2018, AAAI.

[34]  Bhabatosh Chanda,et al.  Learning a Patch Quality Comparator for Single Image Dehazing , 2018, IEEE Transactions on Image Processing.

[35]  拓海 杉山,et al.  “Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks”の学習報告 , 2017 .

[36]  Derek Hoiem,et al.  Indoor Segmentation and Support Inference from RGBD Images , 2012, ECCV.

[37]  Eduardo Cabal-Yepez,et al.  A Fast Image Dehazing Algorithm Using Morphological Reconstruction , 2019, IEEE Transactions on Image Processing.

[38]  Yun Fu,et al.  Residual Dense Network for Image Super-Resolution , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[39]  Raanan Fattal,et al.  Single image dehazing , 2008, ACM Trans. Graph..

[40]  Zhixun Su,et al.  Learning Deep Priors for Image Dehazing , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[41]  Dacheng Tao,et al.  DehazeNet: An End-to-End System for Single Image Haze Removal , 2016, IEEE Transactions on Image Processing.

[42]  Wonha Kim,et al.  Single Image Dehazing Using Color Ellipsoid Prior , 2018, IEEE Transactions on Image Processing.

[43]  Arnold W. M. Smeulders,et al.  Color Invariance , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[44]  Shai Avidan,et al.  Non-local Image Dehazing , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[45]  Ramin Zabih,et al.  Factorial Markov Random Fields , 2002, ECCV.

[46]  Vishal M. Patel,et al.  Densely Connected Pyramid Dehazing Network , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[47]  Ko Nishino,et al.  Bayesian Defogging , 2012, International Journal of Computer Vision.

[48]  Jean-Philippe Tarel,et al.  Fast visibility restoration from a single color or gray level image , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[49]  Yanyun Qu,et al.  Enhanced Pix2pix Dehazing Network , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[50]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.