Decoupled Low-Light Image Enhancement

The visual quality of photographs taken under imperfect lightness conditions can be degenerated by multiple factors, e.g., low lightness, imaging noise, color distortion and so on. Current low-light image enhancement models focus on the improvement of low lightness only, or simply deal with all the degeneration factors as a whole, therefore leading to a sub-optimal performance. In this paper, we propose to decouple the enhancement model into two sequential stages. The first stage focuses on improving the scene visibility based on a pixel-wise non-linear mapping. The second stage focuses on improving the appearance fidelity by suppressing the rest degeneration factors. The decoupled model facilitates the enhancement in two aspects. On the one hand, the whole low-light enhancement can be divided into two easier subtasks. The first one only aims to enhance the visibility. It also helps to bridge the large intensity gap between the low-light and normal-light images. In this way, the second subtask can be shaped as the local appearance adjustment. On the other hand, since the parameter matrix learned from the first stage is aware of the lightness distribution and the scene structure, it can be incorporated into the second stage as the complementary information. In the experiments, our model demonstrates the state-of-the-art performance in both qualitative and quantitative comparisons, compared with other low-light image enhancement models. In addition, the ablation studies also validate the effectiveness of our model in multiple aspects, such as model structure and loss function. The trained model is available at https://github.com/hanxuhfut/Decoupled-Low-light-Image-Enhancement.

[1]  Soumik Sarkar,et al.  LLNet: A deep autoencoder approach to natural low-light image enhancement , 2015, Pattern Recognit..

[2]  Wenjun Zhang,et al.  Using Free Energy Principle For Blind Image Quality Assessment , 2015, IEEE Transactions on Multimedia.

[3]  Michael S. Brown,et al.  Single Image Layer Separation Using Relative Smoothness , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[4]  Xiaojie Guo,et al.  Kindling the Darkness: A Practical Low-light Image Enhancer , 2019, ACM Multimedia.

[5]  Ali M. Reza,et al.  Realization of the Contrast Limited Adaptive Histogram Equalization (CLAHE) for Real-Time Image Enhancement , 2004, J. VLSI Signal Process..

[6]  Lei Zhang,et al.  Robust Multi-Exposure Image Fusion: A Structural Patch Decomposition Approach , 2017, IEEE Transactions on Image Processing.

[7]  Wenhan Yang,et al.  Band Representation-Based Semi-Supervised Low-Light Image Enhancement: Bridging the Gap Between Signal Fidelity and Perceptual Quality , 2021, IEEE Transactions on Image Processing.

[8]  Antonios Gasteratos,et al.  Fast centre- surround contrast modification , 2008 .

[9]  Sumohana S. Channappayya,et al.  Blind image quality evaluation using perception based features , 2015, 2015 Twenty First National Conference on Communications (NCC).

[10]  Dacheng Tao,et al.  A Joint Intrinsic-Extrinsic Prior Model for Retinex , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[11]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[12]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[13]  Alan C. Bovik,et al.  Making a “Completely Blind” Image Quality Analyzer , 2013, IEEE Signal Processing Letters.

[14]  Chul Lee,et al.  Contrast Enhancement Based on Layered Difference Representation of 2D Histograms , 2013, IEEE Transactions on Image Processing.

[15]  Xiaoyan Sun,et al.  Structure-Revealing Low-Light Image Enhancement Via Robust Retinex Model , 2018, IEEE Transactions on Image Processing.

[16]  Jiangtao Wen,et al.  Fast efficient algorithm for enhancement of low lighting video , 2010, 2011 IEEE International Conference on Multimedia and Expo.

[17]  Alan C. Bovik,et al.  Blind/Referenceless Image Spatial Quality Evaluator , 2011, 2011 Conference Record of the Forty Fifth Asilomar Conference on Signals, Systems and Computers (ASILOMAR).

[18]  Fei Li,et al.  Generative adversarial network for low-light image enhancement , 2021, IET Image Process..

[19]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[20]  Lei Zhang,et al.  Learning a Deep Single Image Contrast Enhancer from Multi-Exposure Images , 2018, IEEE Transactions on Image Processing.

[21]  Delu Zeng,et al.  A fusion-based enhancing method for weakly illuminated images , 2016, Signal Process..

[22]  Ding Liu,et al.  EnlightenGAN: Deep Light Enhancement Without Paired Supervision , 2019, IEEE Transactions on Image Processing.

[23]  Sam Kwong,et al.  Zero-Reference Deep Curve Estimation for Low-Light Image Enhancement , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[24]  Randa Atta,et al.  A low-light image enhancement method based on bright channel prior and maximum colour channel , 2021, IET Image Process..

[25]  Chen Wei,et al.  Deep Retinex Decomposition for Low-Light Enhancement , 2018, BMVC.

[26]  Dongming Zhou,et al.  AMBCR: Low-light image enhancement via attention guided multi-branch construction and Retinex theory , 2021, IET Image Process..

[27]  Meng Wang,et al.  Low-Light Image Enhancement With Semi-Decoupled Decomposition , 2020, IEEE Transactions on Multimedia.

[28]  Jian Sun,et al.  Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[29]  Chul Lee,et al.  Contrast enhancement based on layered difference representation , 2012, 2012 19th IEEE International Conference on Image Processing.

[30]  Wen-Huang Cheng,et al.  LR3M: Robust Low-Light Enhancement via Low-Rank Regularized Retinex Model , 2020, IEEE Transactions on Image Processing.

[31]  Qing Zhang,et al.  High-Quality Exposure Correction of Underexposed Photos , 2018, ACM Multimedia.

[32]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[33]  Xiao-Ping Zhang,et al.  A Weighted Variational Model for Simultaneous Reflectance and Illumination Estimation , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[34]  Wen-Huang Cheng,et al.  Joint Enhancement and Denoising Method via Sequential Decomposition , 2018, 2018 IEEE International Symposium on Circuits and Systems (ISCAS).

[35]  Yu Li,et al.  LIME: Low-Light Image Enhancement via Illumination Map Estimation , 2017, IEEE Transactions on Image Processing.

[36]  Chen Wei,et al.  GLADNet: Low-Light Enhancement Network with Global Awareness , 2018, 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018).

[37]  Peter V. Gehler,et al.  Recovering Intrinsic Images with a Global Sparsity Prior on Reflectance , 2011, NIPS.

[38]  Chi-Wing Fu,et al.  Underexposed Photo Enhancement Using Deep Illumination Estimation , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[39]  Xiao-Ping Zhang,et al.  A fusion-based method for single backlit image enhancement , 2016, 2016 IEEE International Conference on Image Processing (ICIP).

[40]  Jinwei Gu,et al.  Low-Light Image and Video Enhancement Using Deep Learning: A Survey. , 2021, IEEE transactions on pattern analysis and machine intelligence.

[41]  Christian Ledig,et al.  Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[42]  Yanrong Guo,et al.  Lightness-aware contrast enhancement for images with different illumination conditions , 2018, Multimedia Tools and Applications.

[43]  Yue Wang,et al.  From Fidelity to Perceptual Quality: A Semi-Supervised Approach for Low-Light Image Enhancement , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).