Multi-Purpose Oriented Single Nighttime Image Haze Removal Based on Unified Variational Retinex Model

Under the nighttime haze environment, the quality of acquired images will be deteriorated significantly owing to the influences of multiple adverse degradation factors. In this paper, we develop a multi-purpose oriented haze removal framework focusing on nighttime hazy images. First, we construct a nonlinear model based on the classic Retinex theory to formulate multiple adverse degradations of a nighttime hazy image. Then, a novel variational Retinex model is presented to simultaneously estimate a smoothed illumination component and a detail-revealed reflectance component and predict the noise map from a pre-processed nighttime hazy image in a unified manner. Specifically, an <inline-formula> <tex-math notation="LaTeX">${\ell _{0}}$ </tex-math></inline-formula> norm is imposed on the reflectance to reveal the structural details and we make use of <inline-formula> <tex-math notation="LaTeX">$\ell _{1}$ </tex-math></inline-formula> norm to constrain the piece-wise smoothness of the illumination and apply <inline-formula> <tex-math notation="LaTeX">${\ell _{2}}$ </tex-math></inline-formula> norm to enforce the total intensity of the noise map. Afterwards, the haze in the illumination component is removed based on prior-based dehazing method and the contrast of the reflectance component is improved in the gradient domain. Finally, we combine the dehazed illumination and the improved reflectance to generate the haze-free image. Experiments show that our proposed framework performs better than famous nighttime image dehazing methods both in visual effects and objective comparisons. In addition, the proposed framework can also be applicable to other types of degraded images.

[1]  Tian Ye,et al.  Nighttime Image Dehazing Based on Variational Decomposition Model , 2022, 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[2]  Hongqing Zhu,et al.  TMS-GAN: A Twofold Multi-Scale Generative Adversarial Network for Single Image Dehazing , 2022, IEEE Transactions on Circuits and Systems for Video Technology.

[3]  Wenhui Wang,et al.  Variational Single Nighttime Image Haze Removal With a Gray Haze-Line Prior , 2022, IEEE Transactions on Image Processing.

[4]  Wenqi Ren,et al.  IDBP: Image Dehazing Using Blended Priors Including Non-Local, Local, and Global Priors , 2021, IEEE Transactions on Circuits and Systems for Video Technology.

[5]  Dwarikanath Mahapatra,et al.  Multi-path dilated convolution network for haze and glow removal in nighttime images , 2021, The Visual Computer.

[6]  Chih-Hsiang Yang,et al.  A Variation-based Nighttime Image Dehazing Flow with a Physically Valid Illumination Estimator and a Luminance-Guided Coloring Model , 2022, IEEE Access.

[7]  Chongyi Li,et al.  Nighttime image dehazing using color cast removal and dual path multi-scale fusion strategy , 2021, Frontiers of Computer Science.

[8]  Dacheng Tao,et al.  IDRLP: Image Dehazing Using Region Line Prior , 2021, IEEE Transactions on Image Processing.

[9]  Zeyuan Chen,et al.  PSD: Principled Synthetic-to-Real Dehazing Guided by Physical Priors , 2021, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[10]  Wenhan Luo,et al.  Multi-Level Fusion and Attention-Guided CNN for Image Dehazing , 2020, IEEE Transactions on Circuits and Systems for Video Technology.

[11]  Yun Liu,et al.  Single nighttime image dehazing based on image decomposition , 2021, Signal Process..

[12]  Tieyong Zeng,et al.  Variational Single Image Dehazing for Enhanced Visualization , 2020, IEEE Transactions on Multimedia.

[13]  Dengxin Dai,et al.  Nighttime Defogging Using High-Low Frequency Decomposition and Grayscale-Color Networks , 2020, ECCV.

[14]  Zheng-Jun Zha,et al.  Nighttime Dehazing with a Synthetic Benchmark , 2020, ACM Multimedia.

[15]  Christophe De Vleeschouwer,et al.  Day and Night-Time Dehazing by Local Airlight Estimation , 2020, IEEE Transactions on Image Processing.

[16]  Wen-Huang Cheng,et al.  LR3M: Robust Low-Light Enhancement via Low-Rank Regularized Retinex Model , 2020, IEEE Transactions on Image Processing.

[17]  Shai Avidan,et al.  Single Image Dehazing Using Haze-Lines , 2020, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[18]  Ling Shao,et al.  STAR: A Structure and Texture Aware Retinex Model , 2019, IEEE Transactions on Image Processing.

[19]  Beomhyuk Koo,et al.  Nighttime Haze Removal with Glow Decomposition Using GAN , 2019, ACPR.

[20]  Guowei Yang,et al.  Nighttime Single Image Dehazing via Pixel-Wise Alpha Blending , 2019, IEEE Access.

[21]  Y. Jay Guo,et al.  BDPK: Bayesian Dehazing Using Prior Knowledge , 2019, IEEE Transactions on Circuits and Systems for Video Technology.

[22]  Jie Yang,et al.  A Visibility-Guided Fusion Framework for Fast Nighttime Image Dehazing , 2018, PRCV.

[23]  Dong Yang,et al.  Proximal Dehaze-Net: A Prior Learning-Based Deep Network for Single Image Dehazing , 2018, ECCV.

[24]  Xinbo Gao,et al.  Single Image Dehazing With Depth-Aware Non-Local Total Variation Regularization , 2018, IEEE Transactions on Image Processing.

[25]  David Zhang,et al.  A Hybrid l1-l0 Layer Decomposition Model for Tone Mapping , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[26]  Xiaoyan Sun,et al.  Structure-Revealing Low-Light Image Enhancement Via Robust Retinex Model , 2018, IEEE Transactions on Image Processing.

[27]  Wonha Kim,et al.  Single Image Dehazing Using Color Ellipsoid Prior , 2018, IEEE Transactions on Image Processing.

[28]  Dengyin Zhang,et al.  Single image haze removal based on the improved atmospheric scattering model , 2017, Neurocomputing.

[29]  Dacheng Tao,et al.  A Joint Intrinsic-Extrinsic Prior Model for Retinex , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[30]  Jizheng Xu,et al.  AOD-Net: All-in-One Dehazing Network , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[31]  Jing Zhang,et al.  Fast Haze Removal for Nighttime Image Using Maximum Reflectance Prior , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[32]  Javier Vazquez-Corral,et al.  Fusion-Based Variational Image Dehazing , 2017, IEEE Signal Processing Letters.

[33]  Yu Li,et al.  LIME: Low-Light Image Enhancement via Illumination Map Estimation , 2017, IEEE Transactions on Image Processing.

[34]  Minh N. Do,et al.  Robust Image and Video Dehazing with Visual Artifact Suppression via Gradient Residual Minimization , 2016, ECCV.

[35]  Xiaochun Cao,et al.  Single Image Dehazing via Multi-scale Convolutional Neural Networks , 2016, ECCV.

[36]  Xiao-Ping Zhang,et al.  A Weighted Variational Model for Simultaneous Reflectance and Illumination Estimation , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[37]  Minghui Wang,et al.  Single image dehazing via multi-scale gradient domain contrast enhancement , 2016, IET Image Process..

[38]  Dacheng Tao,et al.  DehazeNet: An End-to-End System for Single Image Haze Removal , 2016, IEEE Transactions on Image Processing.

[39]  Mohinder Malhotra Single Image Haze Removal Using Dark Channel Prior , 2016 .

[40]  Michael S. Brown,et al.  Nighttime Haze Removal with Glow and Multiple Light Colors , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[41]  Xiao-Ping Zhang,et al.  A Probabilistic Method for Image Enhancement With Simultaneous Illumination and Reflectance Estimation , 2015, IEEE Transactions on Image Processing.

[42]  Ling Shao,et al.  A Fast Single Image Haze Removal Algorithm Using Color Attenuation Prior , 2015, IEEE Transactions on Image Processing.

[43]  Javier Vazquez-Corral,et al.  Enhanced Variational Image Dehazing , 2015, SIAM J. Imaging Sci..

[44]  Raanan Fattal,et al.  Dehazing Using Color-Lines , 2014, ACM Trans. Graph..

[45]  Jing Zhang,et al.  Nighttime haze removal based on a new imaging model , 2014, 2014 IEEE International Conference on Image Processing (ICIP).

[46]  Gaofeng Meng,et al.  Efficient Image Dehazing with Boundary Constraint and Contextual Regularization , 2013, 2013 IEEE International Conference on Computer Vision.

[47]  Jian Sun,et al.  Guided Image Filtering , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[48]  Pascal Fua,et al.  SLIC Superpixels Compared to State-of-the-Art Superpixel Methods , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[49]  Soo-Chang Pei,et al.  Nighttime haze removal using color transfer pre-processing and Dark Channel Prior , 2012, 2012 19th IEEE International Conference on Image Processing.

[50]  Cewu Lu,et al.  Image smoothing via L0 gradient minimization , 2011, ACM Trans. Graph..

[51]  Stephen P. Boyd,et al.  Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers , 2011, Found. Trends Mach. Learn..

[52]  Michael F. Cohen,et al.  Deep photo: model-based photograph enhancement and viewing , 2008, SIGGRAPH Asia '08.

[53]  Robby T. Tan,et al.  Visibility in bad weather from a single image , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[54]  Karen O. Egiazarian,et al.  Color Image Denoising via Sparse 3D Collaborative Filtering with Grouping Constraint in Luminance-Chrominance Space , 2007, 2007 IEEE International Conference on Image Processing.

[55]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[56]  Shree K. Nayar,et al.  Contrast Restoration of Weather Degraded Images , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[57]  Shree K. Nayar,et al.  Polarization-based vision through haze , 2003 .

[58]  Ronen Basri,et al.  Lambertian reflectance and linear subspaces , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[59]  E. Land The retinex theory of color vision. , 1977, Scientific American.

[60]  E. Land,et al.  Lightness and retinex theory. , 1971, Journal of the Optical Society of America.