Multi-scale Single Image Dehazing Using Perceptual Pyramid Deep Network

Haze adversely degrades quality of an image thereby affecting its aesthetic appeal and visibility in outdoor scenes. Single image dehazing is particularly challenging due to its ill-posed nature. Most existing work, including the recent convolutional neural network (CNN) based methods, rely on the classical mathematical formulation where the hazy image is modeled as the superposition of attenuated scene radiance and the atmospheric light. In this work, we explore CNNs to directly learn a non-linear function between hazy images and the corresponding clear images. We present a multi-scale image dehazing method using Perceptual Pyramid Deep Network based on the recently popular dense blocks and residual blocks. The proposed method involves an encoder-decoder structure with a pyramid pooling module in the decoder to incorporate contextual information of the scene while decoding. The network is learned by minimizing the mean squared error and perceptual losses. Multi-scale patches are used during training and inference process to further improve the performance. Experiments on the recently released NTIRE2018-Dehazing dataset demonstrates the superior performance of the proposed method over recent state-of-the-art approaches. Additionally, the proposed method is ranked among top-3 methods in terms of quantitative performance in the recently conducted NTIRE2018-Dehazing challenge. Code can be found at https://github.com/hezhangsprinter/NTIRE-2018-Dehazing-Challenge

[1]  Jean-Philippe Tarel,et al.  Fast visibility restoration from a single color or gray level image , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[2]  Lei Zhang,et al.  Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising , 2016, IEEE Transactions on Image Processing.

[3]  Ko Nishino,et al.  Factorizing Scene Albedo and Depth from a Single Foggy Image , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[4]  Xinghao Ding,et al.  Clearing the Skies: A Deep Network Architecture for Single-Image Rain Removal , 2016, IEEE Transactions on Image Processing.

[5]  Codruta O. Ancuti,et al.  Single Image Dehazing by Multi-Scale Fusion , 2013, IEEE Transactions on Image Processing.

[6]  Dani Lischinski,et al.  Deep photo: model-based photograph enhancement and viewing , 2008, SIGGRAPH 2008.

[7]  Jizheng Xu,et al.  An All-in-One Network for Dehazing and Beyond , 2017, ArXiv.

[8]  Gaurav Sharma,et al.  HazeRD: An outdoor scene dataset and benchmark for single image dehazing , 2017, 2017 IEEE International Conference on Image Processing (ICIP).

[9]  Jan Kautz,et al.  High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANs , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[10]  Kilian Q. Weinberger,et al.  Densely Connected Convolutional Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[11]  Xiaochun Cao,et al.  Single Image Dehazing via Multi-scale Convolutional Neural Networks , 2016, ECCV.

[12]  Vishal M. Patel,et al.  Densely Connected Pyramid Dehazing Network , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[13]  Vishal M. Patel,et al.  GP-GAN: Gender Preserving GAN for Synthesizing Faces from Landmarks , 2017, 2018 24th International Conference on Pattern Recognition (ICPR).

[14]  Cosmin Ancuti,et al.  A Fast Semi-inverse Approach to Detect and Remove the Haze from a Single Image , 2010, ACCV.

[15]  Wei Liu,et al.  Gated Fusion Network for Single Image Dehazing , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[16]  Vishal M. Patel,et al.  High-Quality Facial Photo-Sketch Synthesis Using Multi-Adversarial Networks , 2017, 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018).

[17]  Vishal M. Patel,et al.  Generating High-Quality Crowd Density Maps Using Contextual Pyramid CNNs , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[18]  Alexei A. Efros,et al.  Image-to-Image Translation with Conditional Adversarial Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[19]  Dacheng Tao,et al.  DehazeNet: An End-to-End System for Single Image Haze Removal , 2016, IEEE Transactions on Image Processing.

[20]  Xiaogang Wang,et al.  Pyramid Scene Parsing Network , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[21]  Shai Avidan,et al.  Non-local Image Dehazing , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[22]  Radu Timofte,et al.  I-HAZE: a dehazing benchmark with real hazy and haze-free indoor images , 2018, ACIVS.

[23]  Truong Q. Nguyen,et al.  An Investigation of Dehazing Effects on Image and Video Coding , 2012, IEEE Transactions on Image Processing.

[24]  Jian Sun,et al.  Guided Image Filtering , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[25]  Dani Lischinski,et al.  Deep photo: model-based photograph enhancement and viewing , 2008, SIGGRAPH Asia '08.

[26]  Vishal M. Patel,et al.  Joint Transmission Map Estimation and Dehazing Using Deep Networks , 2017, IEEE Transactions on Circuits and Systems for Video Technology.

[27]  Shree K. Nayar,et al.  Instant dehazing of images using polarization , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[28]  Shai Avidan,et al.  Air-light estimation using haze-lines , 2017, 2017 IEEE International Conference on Computational Photography (ICCP).

[29]  Jizheng Xu,et al.  AOD-Net: All-in-One Dehazing Network , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[30]  Vishal M. Patel,et al.  CNN-Based cascaded multi-task learning of high-level prior and density estimation for crowd counting , 2017, 2017 14th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS).

[31]  Ling Shao,et al.  A Fast Single Image Haze Removal Algorithm Using Color Attenuation Prior , 2015, IEEE Transactions on Image Processing.

[32]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[33]  Thomas Brox,et al.  Generating Images with Perceptual Similarity Metrics based on Deep Networks , 2016, NIPS.

[34]  Vishal M. Patel,et al.  SAR Image Despeckling Using a Convolutional Neural Network , 2017, IEEE Signal Processing Letters.

[35]  Vishal M. Patel,et al.  Image De-Raining Using a Conditional Generative Adversarial Network , 2017, IEEE Transactions on Circuits and Systems for Video Technology.

[36]  Robby T. Tan,et al.  Visibility in bad weather from a single image , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[37]  Trevor Darrell,et al.  Fully Convolutional Networks for Semantic Segmentation , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[38]  Larry S. Davis,et al.  SSH: Single Stage Headless Face Detector , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[39]  Radu Timofte,et al.  O-HAZE: A Dehazing Benchmark with Real Hazy and Haze-Free Outdoor Images , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[40]  Christophe De Vleeschouwer,et al.  D-HAZY: A dataset to evaluate quantitatively dehazing algorithms , 2016, 2016 IEEE International Conference on Image Processing (ICIP).

[41]  Raanan Fattal,et al.  Single image dehazing , 2008, ACM Trans. Graph..

[42]  Fabio Gagliardi Cozman,et al.  Depth from scattering , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[43]  Li Fei-Fei,et al.  Perceptual Losses for Real-Time Style Transfer and Super-Resolution , 2016, ECCV.

[44]  Subrahmanyam Murala,et al.  C^2MSNet: A Novel Approach for Single Image Haze Removal , 2018, 2018 IEEE Winter Conference on Applications of Computer Vision (WACV).

[45]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[46]  Wenjun Zeng,et al.  RESIDE: A Benchmark for Single Image Dehazing , 2017, ArXiv.

[47]  Joonki Paik,et al.  Contrast enhancement system using spatially adaptive histogram equalization with temporal filtering , 1998 .

[48]  Chuan Li,et al.  Precomputed Real-Time Texture Synthesis with Markovian Generative Adversarial Networks , 2016, ECCV.

[49]  Zhe L. Lin,et al.  Top-Down Neural Attention by Excitation Backprop , 2016, International Journal of Computer Vision.

[50]  Jean-Philippe Tarel,et al.  Vision Enhancement in Homogeneous and Heterogeneous Fog , 2012, IEEE Intelligent Transportation Systems Magazine.

[51]  J. Alex Stark,et al.  Adaptive image contrast enhancement using generalizations of histogram equalization , 2000, IEEE Trans. Image Process..

[52]  Mohinder Malhotra Single Image Haze Removal Using Dark Channel Prior , 2016 .

[53]  Ketan Tang,et al.  Investigating Haze-Relevant Features in a Learning Framework for Image Dehazing , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[54]  Raanan Fattal,et al.  Dehazing Using Color-Lines , 2014, ACM Trans. Graph..

[55]  Vishal M. Patel,et al.  Density-Aware Single Image De-raining Using a Multi-stream Dense Network , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[56]  Shree K. Nayar,et al.  Contrast Restoration of Weather Degraded Images , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[57]  Luc Van Gool,et al.  Semantic Foggy Scene Understanding with Synthetic Data , 2017, International Journal of Computer Vision.