RYF-Net: Deep Fusion Network for Single Image Haze Removal

Haze removal from a single image is a challenging task. Estimation of accurate scene transmission map (TrMap) is the key to reconstruct the haze-free scene. In this paper, we propose a convolutional neural network based architecture to estimate the TrMap of the hazy scene. The proposed network takes the hazy image as an input and extracts the haze relevant features using proposed RNet and YNet through RGB and YCbCr color spaces respectively and generates two TrMaps. Further, we propose a novel TrMap fusion network (FNet) to integrate two TrMaPs and estimate robust TrMap for the hazy scene. To analyze the robustness of FNet, we tested it on combinations of TrMaps obtained from existing state-of-the-art methods. Performance evaluation of the proposed approach has been carried out using the structural similarity index, mean square error and peak signal to noise ratio. We conduct experiments on five datasets namely: D-HAZY, Imagenet, Indoor SOTS, HazeRD and set of real-world hazy images. Performance analysis shows that the proposed approach outperforms the existing state-of-the-art methods for single image dehazing. Further, we extended our work to address high-level vision task such as object detection in hazy scenes. It is observed that there is a significant improvement in accurate object detection in hazy scenes using proposed approach.

[1]  Jiwen Lu,et al.  PCANet: A Simple Deep Learning Baseline for Image Classification? , 2014, IEEE Transactions on Image Processing.

[2]  Xiaochun Cao,et al.  Single Image Dehazing via Multi-scale Convolutional Neural Networks , 2016, ECCV.

[3]  Trevor Darrell,et al.  Fully Convolutional Networks for Semantic Segmentation , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[4]  Shree K. Nayar,et al.  Vision in bad weather , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[5]  Robby T. Tan,et al.  Visibility in bad weather from a single image , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[6]  Jason Cong,et al.  Minimizing Computation in Convolutional Neural Networks , 2014, ICANN.

[7]  Jian Sun,et al.  Guided Image Filtering , 2010, ECCV.

[8]  Xiaoou Tang,et al.  Single Image Haze Removal Using Dark Channel Prior , 2011 .

[9]  Ling Shao,et al.  A Fast Single Image Haze Removal Algorithm Using Color Attenuation Prior , 2015, IEEE Transactions on Image Processing.

[10]  Yu Jing Physics-based Fast Single Image Fog Removal , 2011 .

[11]  Victor J. D. Tsai,et al.  A comparative study on shadow compensation of color aerial images in invariant color models , 2006, IEEE Transactions on Geoscience and Remote Sensing.

[12]  Ketan Tang,et al.  Investigating Haze-Relevant Features in a Learning Framework for Image Dehazing , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[13]  Roberto Manduchi,et al.  Bilateral filtering for gray and color images , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[14]  Jenq-Neng Hwang,et al.  DesnowNet: Context-Aware Deep Network for Snow Removal , 2017, IEEE Transactions on Image Processing.

[15]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[16]  Trevor Darrell,et al.  Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation , 2013, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[17]  Gaurav Sharma,et al.  HazeRD: An outdoor scene dataset and benchmark for single image dehazing , 2017, 2017 IEEE International Conference on Image Processing (ICIP).

[18]  S. L. Phung,et al.  A novel skin color model in YCbCr color space and its application to human face detection , 2002, Proceedings. International Conference on Image Processing.

[19]  Truong Q. Nguyen,et al.  An Investigation of Dehazing Effects on Image and Video Coding , 2012, IEEE Transactions on Image Processing.

[20]  Raanan Fattal,et al.  Single image dehazing , 2008, ACM Trans. Graph..

[21]  Fabio Gagliardi Cozman,et al.  Depth from scattering , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[22]  Shree K. Nayar,et al.  Instant dehazing of images using polarization , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[23]  T. Aaron Gulliver,et al.  A Faster RCNN-Based Pedestrian Detection System , 2016, 2016 IEEE 84th Vehicular Technology Conference (VTC-Fall).

[24]  Bernhard Schölkopf,et al.  A Machine Learning Approach for Non-blind Image Deconvolution , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[25]  Yoshua Bengio,et al.  Maxout Networks , 2013, ICML.

[26]  Subrahmanyam Murala,et al.  C^2MSNet: A Novel Approach for Single Image Haze Removal , 2018, 2018 IEEE Winter Conference on Applications of Computer Vision (WACV).

[27]  Frédo Durand,et al.  A Fast Approximation of the Bilateral Filter Using a Signal Processing Approach , 2006, ECCV.

[28]  Derek Hoiem,et al.  Indoor Segmentation and Support Inference from RGBD Images , 2012, ECCV.

[29]  Andrew Zisserman,et al.  Deep Face Recognition , 2015, BMVC.

[30]  Dahua Lin,et al.  Adjustable Bounded Rectifiers: Towards Deep Binary Representations , 2015, ArXiv.

[31]  Dan Feng,et al.  Benchmarking Single-Image Dehazing and Beyond , 2017, IEEE Transactions on Image Processing.

[32]  P. Ganesan,et al.  International Conference on Recent Trends in Computing 2015 ( ICRTC-2015 ) Comparative Study of Skin Color Detection and Segmentation in HSV and YCbCr Color Space , 2015 .

[33]  Cosmin Ancuti,et al.  A Fast Semi-inverse Approach to Detect and Remove the Haze from a Single Image , 2010, ACCV.

[34]  Wei Liu,et al.  Gated Fusion Network for Single Image Dehazing , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[35]  Dacheng Tao,et al.  DehazeNet: An End-to-End System for Single Image Haze Removal , 2016, IEEE Transactions on Image Processing.

[36]  Fei-Fei Li,et al.  ImageNet: A large-scale hierarchical image database , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[37]  Xiaoou Tang,et al.  Image Super-Resolution Using Deep Convolutional Networks , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[38]  Shih-Chia Huang,et al.  Visibility Restoration of Single Hazy Images Captured in Real-World Weather Conditions , 2014, IEEE Transactions on Circuits and Systems for Video Technology.

[39]  Rob Fergus,et al.  Restoring an Image Taken through a Window Covered with Dirt or Rain , 2013, 2013 IEEE International Conference on Computer Vision.

[40]  Lawrence D. Jackel,et al.  Backpropagation Applied to Handwritten Zip Code Recognition , 1989, Neural Computation.

[41]  Rob Fergus,et al.  Visualizing and Understanding Convolutional Networks , 2013, ECCV.

[42]  Jiansheng Qian,et al.  Research on Moving Object Detecting and Shadow Removal , 2009, 2009 First International Conference on Information Science and Engineering.

[43]  Christophe De Vleeschouwer,et al.  D-HAZY: A dataset to evaluate quantitatively dehazing algorithms , 2016, 2016 IEEE International Conference on Image Processing (ICIP).

[44]  Yoav Y. Schechner,et al.  Blind Haze Separation , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).