From Synthetic to Real: Image Dehazing Collaborating with Unlabeled Real Data

Single image dehazing is a challenging task, for which the domain shift between synthetic training data and real-world testing images usually leads to degradation of existing methods. To address this issue, we propose a novel image dehazing framework collaborating with unlabeled real data. First, we develop a disentangled image dehazing network (DID-Net), which disentangles the feature representations into three component maps, i.e. the latent haze-free image, the transmission map, and the global atmospheric light estimate, respecting the physical model of a haze process. Our DID-Net predicts the three component maps by progressively integrating features across scales, and refines each map by passing an independent refinement network. Then a disentangled-consistency mean-teacher network (DMT-Net) is employed to collaborate unlabeled real data for boosting single image dehazing. Specifically, we encourage the coarse predictions and refinements of each disentangled component to be consistent between the student and teacher networks by using a consistency loss on unlabeled real data. We make comparison with 13 state-of-the-art dehazing methods on a new collected dataset (Haze4K) and two widely-used dehazing datasets (i.e., SOTS and HazeRD), as well as on real-world hazy images. Experimental results demonstrate that our method has obvious quantitative and qualitative improvements over the existing methods.

[1]  Gaurav Sharma,et al.  HazeRD: An outdoor scene dataset and benchmark for single image dehazing , 2017, 2017 IEEE International Conference on Image Processing (ICIP).

[2]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[3]  Shai Avidan,et al.  Air-light estimation using haze-lines , 2017, 2017 IEEE International Conference on Computational Photography (ICCP).

[4]  Mohinder Malhotra Single Image Haze Removal Using Dark Channel Prior , 2016 .

[5]  Haoran Xie,et al.  Learning Gated Non-Local Residual for Single-Image Rain Streak Removal , 2020, IEEE Transactions on Circuits and Systems for Video Technology.

[6]  Huazhu Fu,et al.  PDR-Net: Perception-Inspired Single Image Dehazing Network With Refinement , 2020, IEEE Transactions on Multimedia.

[7]  Ziling Huang,et al.  HardGAN: A Haze-Aware Representation Distillation GAN for Single Image Dehazing , 2020, ECCV.

[8]  Xi Peng,et al.  Zero-Shot Image Dehazing , 2020, IEEE Transactions on Image Processing.

[9]  Dan Feng,et al.  Benchmarking Single-Image Dehazing and Beyond , 2017, IEEE Transactions on Image Processing.

[10]  Xiaodong Xie,et al.  FFA-Net: Feature Fusion Attention Network for Single Image Dehazing , 2019, AAAI.

[11]  Harri Valpola,et al.  Weight-averaged consistency targets improve semi-supervised deep learning results , 2017, ArXiv.

[12]  Shai Avidan,et al.  Non-local Image Dehazing , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[13]  Yunlong Liu,et al.  Fast Image Dehazing Method Based on Linear Transformation , 2017, IEEE Transactions on Multimedia.

[14]  Yun Fu,et al.  Image Super-Resolution Using Very Deep Residual Channel Attention Networks , 2018, ECCV.

[15]  Ling Shao,et al.  A Fast Single Image Haze Removal Algorithm Using Color Attenuation Prior , 2015, IEEE Transactions on Image Processing.

[16]  Lei Xiang,et al.  Multi-Scale Boosted Dehazing Network With Dense Feature Fusion , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[17]  Jinhui Tang,et al.  Single Image Dehazing via Conditional Generative Adversarial Network , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[18]  Dani Lischinski,et al.  Joint Bi-layer Optimization for Single-Image Rain Streak Removal , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[19]  Zhuowen Tu,et al.  Aggregated Residual Transformations for Deep Neural Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[20]  Vishal M. Patel,et al.  Densely Connected Pyramid Dehazing Network , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[21]  Jizheng Xu,et al.  An All-in-One Network for Dehazing and Beyond , 2017, ArXiv.

[22]  Alexei A. Efros,et al.  Image-to-Image Translation with Conditional Adversarial Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[23]  Wei Liu,et al.  Gated Fusion Network for Single Image Dehazing , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[24]  Raanan Fattal,et al.  Single image dehazing , 2008, ACM Trans. Graph..

[25]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[26]  Wei Liu,et al.  ParseNet: Looking Wider to See Better , 2015, ArXiv.

[27]  Michael S. Brown,et al.  A Non-local Low-Rank Framework for Ultrasound Speckle Reduction , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[28]  Dacheng Tao,et al.  DehazeNet: An End-to-End System for Single Image Haze Removal , 2016, IEEE Transactions on Image Processing.

[29]  Song Wang,et al.  A Multi-Task Mean Teacher for Semi-Supervised Shadow Detection , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[30]  Xiaowu Chen,et al.  Single Image Dehazing Using Ranking Convolutional Neural Network , 2018, IEEE Transactions on Multimedia.

[31]  Jiangxin Dong,et al.  Physics-Based Feature Dehazing Networks , 2020, ECCV.

[32]  Dong Yang,et al.  Proximal Dehaze-Net: A Prior Learning-Based Deep Network for Single Image Dehazing , 2018, ECCV.

[33]  Derek Hoiem,et al.  Indoor Segmentation and Support Inference from RGBD Images , 2012, ECCV.

[34]  Miss A.O. Penney (b) , 1974, The New Yale Book of Quotations.

[35]  Codruta O. Ancuti,et al.  Single Image Dehazing by Multi-Scale Fusion , 2013, IEEE Transactions on Image Processing.

[36]  Alessandro Bria,et al.  On the Duality Between Retinex and Image Dehazing , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[37]  Michael Werman,et al.  Automatic recovery of the atmospheric light in hazy images , 2014, 2014 IEEE International Conference on Computational Photography (ICCP).

[38]  Yu Liu,et al.  Exploring Disentangled Feature Representation Beyond Face Identification , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[39]  Huazhu Fu,et al.  A Cascaded Convolutional Neural Network for Single Image Dehazing , 2018, IEEE Access.

[40]  Raanan Fattal,et al.  Dehazing Using Color-Lines , 2014, ACM Trans. Graph..

[41]  Pheng-Ann Heng,et al.  Deep Multi-Model Fusion for Single-Image Dehazing , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[42]  Shree K. Nayar,et al.  Vision in bad weather , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[43]  Timo Aila,et al.  Temporal Ensembling for Semi-Supervised Learning , 2016, ICLR.

[44]  Xiaochun Cao,et al.  Ultra-High-Definition Image Dehazing via Multi-Guided Bilateral Learning , 2021, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[45]  Changxin Gao,et al.  Domain Adaptation for Image Dehazing , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[46]  Xiaochun Cao,et al.  Single Image Dehazing via Multi-scale Convolutional Neural Networks , 2016, ECCV.

[47]  Jun Chen,et al.  GridDehazeNet: Attention-Based Multi-Scale Network for Image Dehazing , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[48]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.