Low Light Image Enhancement via Global and Local Context Modeling

Images captured under low-light conditions manifest poor visibility, lack contrast and color vividness. Compared to conventional approaches, deep convolutional neural networks (CNNs) perform well in enhancing images. However, being solely reliant on confined fixed primitives to model dependencies, existing data-driven deep models do not exploit the contexts at various spatial scales to address low-light image enhancement. These contexts can be crucial towards inferring several image enhancement tasks, e.g., local and global contrast, brightness and color corrections; which requires cues from both local and global spatial extent. To this end, we introduce a context-aware deep network for lowlight image enhancement. First, it features a global context module that models spatial correlations to find complementary cues over full spatial domain. Second, it introduces a dense residual block that captures local context with a relatively large receptive field. We evaluate the proposed approach using three challenging datasets: MIT-Adobe FiveK, LoL, and SID. On all these datasets, our method performs favorably against the state-of-the-arts in terms of standard image fidelity metrics. In particular, compared to the best performing method on the MIT-Adobe FiveK dataset, our algorithm improves PSNR from 23.04 dB to 24.45 dB.

[1]  Shuicheng Yan,et al.  A2-Nets: Double Attention Networks , 2018, NeurIPS.

[2]  Soumik Sarkar,et al.  LLNet: A deep autoencoder approach to natural low-light image enhancement , 2015, Pattern Recognit..

[3]  Jiangtao Wen,et al.  Fast efficient algorithm for enhancement of low lighting video , 2010, 2011 IEEE International Conference on Multimedia and Expo.

[4]  Xiao-Ping Zhang,et al.  A Weighted Variational Model for Simultaneous Reflectance and Illumination Estimation , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[5]  Jie Ma,et al.  MSR-net: Low-light Image Enhancement Using Deep Convolutional Network , 2017, ArXiv.

[6]  Zia-ur Rahman,et al.  A multiscale retinex for bridging the gap between color images and the human observation of scenes , 1997, IEEE Trans. Image Process..

[7]  Karel J. Zuiderveld,et al.  Contrast Limited Adaptive Histogram Equalization , 1994, Graphics Gems.

[8]  Stamatios Lefkimmiatis,et al.  Deep Image Demosaicking using a Cascade of Convolutional Residual Denoising Networks , 2018, ECCV.

[9]  Xiaoou Tang,et al.  Aesthetic-Driven Image Enhancement by Adversarial Learning , 2017, ACM Multimedia.

[10]  Fei-Fei Li,et al.  Modeling mutual context of object and human pose in human-object interaction activities , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[11]  Yi Zhang,et al.  PSANet: Point-wise Spatial Attention Network for Scene Parsing , 2018, ECCV.

[12]  Xinlei Chen,et al.  Spatial Memory for Context Reasoning in Object Detection , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[13]  Hai-Miao Hu,et al.  Naturalness Preserved Enhancement Algorithm for Non-Uniform Illumination Images , 2013, IEEE Transactions on Image Processing.

[14]  Dacheng Tao,et al.  A Joint Intrinsic-Extrinsic Prior Model for Retinex , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[15]  Ronggang Wang,et al.  A New Image Contrast Enhancement Algorithm Using Exposure Fusion Framework , 2017, CAIP.

[16]  Jn Morovi,et al.  Color Gamut Mapping , 2008 .

[17]  Ling Shao,et al.  Learning Digital Camera Pipeline for Extreme Low-Light Imaging , 2019, Neurocomputing.

[18]  Thomas S. Huang,et al.  Generative Image Inpainting with Contextual Attention , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[19]  Yung-Yu Chuang,et al.  Deep Photo Enhancer: Unpaired Learning for Image Enhancement from Photographs with GANs , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[20]  Chen Wei,et al.  Deep Retinex Decomposition for Low-Light Enhancement , 2018, BMVC.

[21]  Chi-Wing Fu,et al.  Underexposed Photo Enhancement Using Deep Illumination Estimation , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[22]  Trevor Darrell,et al.  Fully Convolutional Networks for Semantic Segmentation , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[23]  Joonki Paik,et al.  Contrast-dependent saturation adjustment for outdoor image enhancement. , 2017, Journal of the Optical Society of America. A, Optics, image science, and vision.

[24]  Enhua Wu,et al.  Squeeze-and-Excitation Networks , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[25]  A. Culyer Thurstone’s Law of Comparative Judgment , 2014 .

[26]  Dacheng Tao,et al.  DehazeNet: An End-to-End System for Single Image Haze Removal , 2016, IEEE Transactions on Image Processing.

[27]  Hao He,et al.  Exposure , 2017, ACM Trans. Graph..

[28]  Shiguang Shan,et al.  Structure Inference Net: Object Detection Using Scene-Level Context and Instance-Level Relationships , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[29]  Xiaoyan Sun,et al.  Structure-Revealing Low-Light Image Enhancement Via Robust Retinex Model , 2018, IEEE Transactions on Image Processing.

[30]  Gang Sun,et al.  Gather-Excite: Exploiting Feature Context in Convolutional Neural Networks , 2018, NeurIPS.

[31]  Xiaojie Guo,et al.  Kindling the Darkness: A Practical Low-light Image Enhancer , 2019, ACM Multimedia.

[32]  In-So Kweon,et al.  Distort-and-Recover: Color Enhancement Using Deep Reinforcement Learning , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[33]  Yue Li,et al.  Improving Extreme Low-Light Image Denoising via Residual Learning , 2019, 2019 IEEE International Conference on Multimedia and Expo (ICME).

[34]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[35]  Abhinav Gupta,et al.  Non-local Neural Networks , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[36]  Jia Deng,et al.  Stacked Hourglass Networks for Human Pose Estimation , 2016, ECCV.

[37]  Junping Du,et al.  Low-Light Image Enhancement via a Deep Hybrid Network , 2019, IEEE Transactions on Image Processing.

[38]  Deva Ramanan,et al.  Attentional Pooling for Action Recognition , 2017, NIPS.

[39]  Yun Fu,et al.  Residual Non-local Attention Networks for Image Restoration , 2019, ICLR.

[40]  Jia Xu,et al.  Learning to See in the Dark , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[41]  Luc Van Gool,et al.  WESPE: Weakly Supervised Photo Enhancer for Digital Cameras , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[42]  Wen Gao,et al.  A Bio-Inspired Multi-Exposure Fusion Framework for Low-light Image Enhancement , 2017, ArXiv.

[43]  Alessandro Rizzi,et al.  Perceptual Color Correction Through Variational Techniques , 2007, IEEE Transactions on Image Processing.

[44]  Xiangyu Zhang,et al.  Large Kernel Matters — Improve Semantic Segmentation by Global Convolutional Network , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[45]  Chen Wei,et al.  GLADNet: Low-Light Enhancement Network with Global Awareness , 2018, 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018).

[46]  Gang Wang,et al.  Context Contrasted Feature and Gated Multi-scale Aggregation for Scene Segmentation , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[47]  Zia-ur Rahman,et al.  Properties and performance of a center/surround retinex , 1997, IEEE Trans. Image Process..

[48]  Stamatios Lefkimmiatis,et al.  Universal Denoising Networks : A Novel CNN Architecture for Image Denoising , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[49]  Erkut Erdem,et al.  Burst Photography for Learning to Enhance Extremely Dark Images , 2020, IEEE Transactions on Image Processing.

[50]  Stefan Roth,et al.  Benchmarking Denoising Algorithms with Real Photographs , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[51]  W.E. Snyder,et al.  Color image processing pipeline , 2005, IEEE Signal Processing Magazine.

[52]  Jonathan T. Barron,et al.  Deep bilateral learning for real-time image enhancement , 2017, ACM Trans. Graph..

[53]  Yu Li,et al.  LIME: Low-Light Image Enhancement via Illumination Map Estimation , 2017, IEEE Transactions on Image Processing.

[54]  E. Land The retinex theory of color vision. , 1977, Scientific American.

[55]  Jian Sun,et al.  Automatic Exposure Correction of Consumer Photographs , 2012, ECCV.