Deep Learning Based Exposure Correction for Image Exposure Correction with Application in Computer Vision for Robotics

This paper presents a convolutional neural network that is able to minimize the effects of poorly exposed digital images in post processing, outperforming state-of-the-art networks on two different conditions: images that are too dark, and images where the irradiance exceeds both the lower and upper bounds of the sensor. The scene's illumination, camera aperture, exposure time, and ISO sensitivity present a direct impact on the quality of the resulting image. Optimize these parameters, however, is a challenging task, which often leads to images that are either too bright, too dark or out of focus. In order to restore over-exposed and under-exposed images, we propose a Generative Adversarial Network (GAN) which learns how to restore an image, preserving and enhancing the contents of the input image, as well as complete regions where data was completely lost due to clipping. Our results show the proposed network is able to enhance color, fulfill regions with missing data, and preserve structural details.

[1]  Jan Kautz,et al.  Exposure Fusion , 2009, 15th Pacific Conference on Computer Graphics and Applications (PG'07).

[2]  Alan C. Bovik,et al.  A Two-Step Framework for Constructing Blind Image Quality Indices , 2010, IEEE Signal Processing Letters.

[3]  Christopher Joseph Pal,et al.  The Importance of Skip Connections in Biomedical Image Segmentation , 2016, LABELS/DLMIA@MICCAI.

[4]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[5]  Qiang Chen,et al.  Network In Network , 2013, ICLR.

[6]  Alexander M. Rush,et al.  Dilated Convolutions for Modeling Long-Distance Genomic Dependencies , 2017, bioRxiv.

[7]  Vladlen Koltun,et al.  Multi-Scale Context Aggregation by Dilated Convolutions , 2015, ICLR.

[8]  Alexei A. Efros,et al.  Unpaired Image-to-Image Translation Using Cycle-Consistent Adversarial Networks , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[9]  Alan C. Bovik,et al.  No-Reference Image Quality Assessment in the Spatial Domain , 2012, IEEE Transactions on Image Processing.

[10]  Robert L. Stevenson,et al.  Dynamic range improvement through multiple exposures , 1999, Proceedings 1999 International Conference on Image Processing (Cat. 99CH36348).

[11]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[12]  Seyed-Ahmad Ahmadi,et al.  V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation , 2016, 2016 Fourth International Conference on 3D Vision (3DV).

[13]  Jiawen Chen,et al.  Bilateral guided upsampling , 2016, ACM Trans. Graph..

[14]  Sylvain Paris,et al.  Learning photographic global tonal adjustment with a database of input / output image pairs , 2011, CVPR 2011.

[15]  Jitendra Malik,et al.  Recovering high dynamic range radiance maps from photographs , 1997, SIGGRAPH '08.

[16]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[17]  Jonathan T. Barron,et al.  Deep bilateral learning for real-time image enhancement , 2017, ACM Trans. Graph..

[18]  Lei Zhang,et al.  Learning without Human Scores for Blind Image Quality Assessment , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[19]  Alexei A. Efros,et al.  Context Encoders: Feature Learning by Inpainting , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[20]  Sepp Hochreiter,et al.  Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs) , 2015, ICLR.

[21]  Vincent Dumoulin,et al.  Deconvolution and Checkerboard Artifacts , 2016 .

[22]  Soumith Chintala,et al.  Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks , 2015, ICLR.

[23]  Kilian Q. Weinberger,et al.  Densely Connected Convolutional Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[24]  Jonathan T. Barron,et al.  Burst photography for high dynamic range and low-light imaging on mobile cameras , 2016, ACM Trans. Graph..