Deep Low-Contrast Image Enhancement using Structure Tensor Representation

We present a new deep learning framework for low-contrast image enhancement, which trains the network using the multi-exposure sequences rather than explicit ground-truth images. The purpose of our method is to enhance a lowcontrast image so as to contain abundant details in various exposure levels. To realize this, we propose to design the loss function using the structure tensor representation, which has been widely used as high-dimensional image contrast. Our loss function penalizes the difference of the structure tensors between the network output and the multi-exposure images in a multi-scale manner. Eventually, the network trained by the loss function produces a high-quality image approximating the overall contrast of the sequence. We provide indepth analysis on our method and comparison with conventional loss functions. Quantitative and qualitative evaluations demonstrate that the proposed method outperforms the existing state-of-the art approaches in various benchmarks.

[1]  Yu Li,et al.  LIME: Low-Light Image Enhancement via Illumination Map Estimation , 2017, IEEE Transactions on Image Processing.

[2]  Lawrence B. Wolff,et al.  Multispectral image visualization through first-order fusion , 2002, IEEE Trans. Image Process..

[3]  Bo Gu,et al.  Gradient field multi-exposure images fusion for high dynamic range image visualization , 2012, J. Vis. Commun. Image Represent..

[4]  Jian Sun,et al.  Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[5]  Chi-Wing Fu,et al.  Underexposed Photo Enhancement Using Deep Illumination Estimation , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[6]  E. Land The retinex theory of color vision. , 1977, Scientific American.

[7]  Jiri Matas,et al.  DeblurGAN: Blind Motion Deblurring Using Conditional Adversarial Networks , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[8]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[9]  Michael C. Mozer,et al.  A Focused Backpropagation Algorithm for Temporal Pattern Recognition , 1989, Complex Syst..

[10]  Thomas Brox,et al.  Generating Images with Perceptual Similarity Metrics based on Deep Networks , 2016, NIPS.

[11]  Ge Li,et al.  LECARM: Low-Light Image Enhancement Using the Camera Response Model , 2019, IEEE Transactions on Circuits and Systems for Video Technology.

[12]  Alan C. Bovik,et al.  Making a “Completely Blind” Image Quality Analyzer , 2013, IEEE Signal Processing Letters.

[13]  Vladlen Koltun,et al.  Multi-Scale Context Aggregation by Dilated Convolutions , 2015, ICLR.

[14]  Silvano Di Zenzo,et al.  A note on the gradient of a multi-image , 1986, Comput. Vis. Graph. Image Process..

[15]  Trevor Darrell,et al.  Learning Features by Watching Objects Move , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[16]  Xiao-Ping Zhang,et al.  A Weighted Variational Model for Simultaneous Reflectance and Illumination Estimation , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[17]  Lei Zhang,et al.  Learning a Deep Single Image Contrast Enhancer from Multi-Exposure Images , 2018, IEEE Transactions on Image Processing.

[18]  David Connah,et al.  Spectral Edge Image Fusion: Theory and Applications , 2014, ECCV.

[19]  Wen Gao,et al.  A Bio-Inspired Multi-Exposure Fusion Framework for Low-light Image Enhancement , 2017, ArXiv.

[20]  Lei Zhang,et al.  Robust Multi-Exposure Image Fusion: A Structural Patch Decomposition Approach , 2017, IEEE Transactions on Image Processing.

[21]  Seunghoon Hong,et al.  Learning Deconvolution Network for Semantic Segmentation , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[22]  Luc Van Gool,et al.  DSLR-Quality Photos on Mobile Devices with Deep Convolutional Networks , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[23]  Zhou Wang,et al.  No-Reference Quality Assessment of Contrast-Distorted Images Based on Natural Scene Statistics , 2015, IEEE Signal Processing Letters.

[24]  Yongwei Nie,et al.  Dual Illumination Estimation for Robust Exposure Correction , 2019, Comput. Graph. Forum.

[25]  Jan Kautz,et al.  Loss Functions for Image Restoration With Neural Networks , 2017, IEEE Transactions on Computational Imaging.

[26]  Lei Zhang,et al.  Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising , 2016, IEEE Transactions on Image Processing.

[27]  David Connah,et al.  Lookup-Table-Based Gradient Field Reconstruction , 2011, IEEE Transactions on Image Processing.

[28]  Jian Sun,et al.  ScribbleSup: Scribble-Supervised Convolutional Networks for Semantic Segmentation , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[29]  Junping Du,et al.  Low-Light Image Enhancement via a Deep Hybrid Network , 2019, IEEE Transactions on Image Processing.

[30]  Xiaoou Tang,et al.  Image Super-Resolution Using Deep Convolutional Networks , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[31]  Yücel Altunbasak,et al.  A Histogram Modification Framework and Its Application for Image Contrast Enhancement , 2009, IEEE Transactions on Image Processing.

[32]  Chen Wei,et al.  Deep Retinex Decomposition for Low-Light Enhancement , 2018, BMVC.

[33]  Kwanghoon Sohn,et al.  Unsupervised Deep Image Fusion With Structure Tensor Representations , 2020, IEEE Transactions on Image Processing.

[34]  Turgay Çelik,et al.  Contextual and Variational Contrast Enhancement , 2011, IEEE Transactions on Image Processing.

[35]  Kwanghoon Sohn,et al.  Deeply Aggregated Alternating Minimization for Image Restoration , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).