MBLLEN: Low-Light Image/Video Enhancement Using CNNs

We present a deep learning based method for low-light image enhancement. This problem is challenging due to the difficulty in handling various factors simultaneously including brightness, contrast, artifacts and noise. To address this task, we propose the multi-branch low-light enhancement network (MBLLEN). The key idea is to extract rich features up to different levels, so that we can apply enhancement via multiple subnets and finally produce the output image via multi-branch fusion. In this manner, image quality is improved from different aspects. Through extensive experiments, our proposed MBLLEN is found to outperform the state-of-art techniques by a large margin. We additionally show that our method can be directly extended to handle low-light videos.

[1]  Haidi Ibrahim,et al.  Brightness Preserving Dynamic Histogram Equalization for Image Contrast Enhancement , 2007, IEEE Transactions on Consumer Electronics.

[2]  Delu Zeng,et al.  A fusion-based enhancing method for weakly illuminated images , 2016, Signal Process..

[3]  Akira Taguchi,et al.  Color image contrast enhacement method based on differential intensity/saturation gray-levels histograms , 2013, 2013 International Symposium on Intelligent Signal Processing and Communication Systems.

[4]  Ronggang Wang,et al.  A New Image Contrast Enhancement Algorithm Using Exposure Fusion Framework , 2017, CAIP.

[5]  Alessandro Foi,et al.  Variance Stabilization for Noisy+Estimate Combination in Iterative Poisson Denoising , 2016, IEEE Signal Processing Letters.

[6]  Alan C. Bovik,et al.  Image information and visual quality , 2004, 2004 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[7]  Sangkeun Lee,et al.  Artifact-Free Low-Light Video Enhancement Using Temporal Similarity and Guide Map , 2017, IEEE Transactions on Industrial Electronics.

[8]  Li Tao,et al.  LLCNN: A convolutional neural network for low-light image enhancement , 2017, 2017 IEEE Visual Communications and Image Processing (VCIP).

[9]  Alexander M. Bronstein,et al.  Deep class-aware image denoising , 2017, 2017 IEEE International Conference on Image Processing (ICIP).

[10]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[11]  Lei Zhang,et al.  Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising , 2016, IEEE Transactions on Image Processing.

[12]  Zhou Wang,et al.  Objective Quality Assessment of Tone-Mapped Images , 2013, IEEE Transactions on Image Processing.

[13]  Chin-Chuan Han,et al.  Adaptive Multiscale Retinex for Image Contrast Enhancement , 2013, 2013 International Conference on Signal-Image Technology & Internet-Based Systems.

[14]  Yücel Altunbasak,et al.  A Histogram Modification Framework and Its Application for Image Contrast Enhancement , 2009, IEEE Transactions on Image Processing.

[15]  Yong Dou,et al.  A piecewise-based contrast enhancement framework for low lighting video , 2014, Proceedings 2014 IEEE International Conference on Security, Pattern Analysis, and Cybernetics (SPAC).

[16]  Huijie Liu,et al.  Low-light video image enhancement based on multiscale Retinex-like algorithm , 2016, 2016 Chinese Control and Decision Conference (CCDC).

[17]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[18]  Sheng Zhang,et al.  A novel framework for enhancement of the low lighting video , 2017, 2017 IEEE Symposium on Computers and Communications (ISCC).

[19]  Zhou Wang,et al.  Multiscale structural similarity for image quality assessment , 2003, The Thrity-Seventh Asilomar Conference on Signals, Systems & Computers, 2003.

[20]  Chul Lee,et al.  Contrast Enhancement Based on Layered Difference Representation of 2D Histograms , 2013, IEEE Transactions on Image Processing.

[21]  E. Land The retinex theory of color vision. , 1977, Scientific American.

[22]  Mongi A. Abidi,et al.  Gray-level grouping (GLG): an automatic method for optimized image contrast Enhancement-part I: the basic method , 2006, IEEE Transactions on Image Processing.

[23]  Yao Lu,et al.  Fast efficient algorithm for enhancement of low lighting video , 2011, ICME.

[24]  Alexei A. Efros,et al.  Image-to-Image Translation with Conditional Adversarial Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[25]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[26]  Luc Van Gool,et al.  The Pascal Visual Object Classes (VOC) Challenge , 2010, International Journal of Computer Vision.

[27]  Wen Gao,et al.  A Bio-Inspired Multi-Exposure Fusion Framework for Low-light Image Enhancement , 2017, ArXiv.

[28]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[29]  Tao Lu,et al.  Low-light image enhancement using CNN and bright channel prior , 2017, 2017 IEEE International Conference on Image Processing (ICIP).

[30]  Xiao-Ping Zhang,et al.  A Weighted Variational Model for Simultaneous Reflectance and Illumination Estimation , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[31]  Hanseok Ko,et al.  A novel approach for denoising and enhancement of extremely low-light video , 2015, IEEE Transactions on Consumer Electronics.

[32]  Alexei A. Efros,et al.  Unpaired Image-to-Image Translation Using Cycle-Consistent Adversarial Networks , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[33]  Yu Li,et al.  LIME: Low-Light Image Enhancement via Illumination Map Estimation , 2017, IEEE Transactions on Image Processing.

[34]  Christian Ledig,et al.  Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[35]  Martín Abadi,et al.  TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems , 2016, ArXiv.

[36]  Hai-Miao Hu,et al.  Naturalness Preserved Enhancement Algorithm for Non-Uniform Illumination Images , 2013, IEEE Transactions on Image Processing.

[37]  Christian Ledig,et al.  Real-Time Video Super-Resolution with Spatio-Temporal Networks and Motion Compensation , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[38]  Soumik Sarkar,et al.  LLNet: A deep autoencoder approach to natural low-light image enhancement , 2015, Pattern Recognit..

[39]  Turgay Çelik,et al.  Contextual and Variational Contrast Enhancement , 2011, IEEE Transactions on Image Processing.

[40]  Zia-ur Rahman,et al.  A multiscale retinex for bridging the gap between color images and the human observation of scenes , 1997, IEEE Trans. Image Process..

[41]  Zia-ur Rahman,et al.  Properties and performance of a center/surround retinex , 1997, IEEE Trans. Image Process..

[42]  Ronggang Wang,et al.  A New Low-Light Image Enhancement Algorithm Using Camera Response Model , 2017, 2017 IEEE International Conference on Computer Vision Workshops (ICCVW).