Progressive Retinex: Mutually Reinforced Illumination-Noise Perception Network for Low-Light Image Enhancement

Contrast enhancement and noise removal are coupled problems for low-light image enhancement. The existing Retinex based methods do not take the coupling relation into consideration, resulting in under or over-smoothing of the enhanced images. To address this issue, this paper presents a novel progressive Retinex framework, in which illumination and noise of low-light image are perceived in a mutually reinforced manner, leading to noise reduction low-light enhancement results. Specifically, two fully pointwise convolutional neural networks are devised to model the statistical regularities of ambient light and image noise respectively, and to leverage them as constraints to facilitate the mutual learning process. The proposed method not only suppresses the interference caused by the ambiguity between tiny textures and image noises, but also greatly improves the computational efficiency. Moreover, to solve the problem of insufficient training data, we propose an image synthesis strategy based on camera imaging model, which generates color images corrupted by illumination-dependent noises. Experimental results on both synthetic and real low-light images demonstrate the superiority of our proposed approaches against the State-Of-The-Art (SOTA) low-light enhancement methods.

[1]  Takeo Kanade,et al.  Statistical calibration of CCD imaging process , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[2]  Xiao-Ping Zhang,et al.  A Weighted Variational Model for Simultaneous Reflectance and Illumination Estimation , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[3]  Dacheng Tao,et al.  A Joint Intrinsic-Extrinsic Prior Model for Retinex , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[4]  Jie Ma,et al.  MSR-net: Low-light Image Enhancement Using Deep Convolutional Network , 2017, ArXiv.

[5]  Jian Sun,et al.  Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[6]  Chen Wei,et al.  Deep Retinex Decomposition for Low-Light Enhancement , 2018, BMVC.

[7]  David Zhang,et al.  FSIM: A Feature Similarity Index for Image Quality Assessment , 2011, IEEE Transactions on Image Processing.

[8]  Jean-Michel Morel,et al.  A PDE Formalization of Retinex Theory , 2010, IEEE Transactions on Image Processing.

[9]  Xiaoyan Sun,et al.  Structure-Revealing Low-Light Image Enhancement Via Robust Retinex Model , 2018, IEEE Transactions on Image Processing.

[10]  Wen Gao,et al.  A low-light image enhancement method for both denoising and contrast enlarging , 2015, 2015 IEEE International Conference on Image Processing (ICIP).

[11]  Wai-kuen Cham,et al.  Gradient-Directed Multiexposure Composition , 2012, IEEE Transactions on Image Processing.

[12]  Richard Szeliski,et al.  Automatic Estimation and Removal of Noise from a Single Image , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[13]  Yu Li,et al.  LIME: Low-Light Image Enhancement via Illumination Map Estimation , 2017, IEEE Transactions on Image Processing.

[14]  Lilong Shi,et al.  The Rehabilitation of MaxRGB , 2010, CIC.

[15]  Dacheng Tao,et al.  FAMED-Net: A Fast and Accurate Multi-Scale End-to-End Dehazing Network , 2019, IEEE Transactions on Image Processing.

[16]  Mark S. Drew,et al.  The Role of Bright Pixels in Illumination Estimation , 2012, Color Imaging Conference.

[17]  Jianbo Shi,et al.  Generalized Random Walks for Fusion of Multi-Exposure Images , 2011, IEEE Transactions on Image Processing.

[18]  Zia-ur Rahman,et al.  A multiscale retinex for bridging the gap between color images and the human observation of scenes , 1997, IEEE Trans. Image Process..

[19]  Michael Elad,et al.  Retinex by Two Bilateral Filters , 2005, Scale-Space.

[20]  Lei Zhang,et al.  Learning a Deep Single Image Contrast Enhancer from Multi-Exposure Images , 2018, IEEE Transactions on Image Processing.

[21]  Zia-ur Rahman,et al.  Retinex processing for automatic image enhancement , 2004, J. Electronic Imaging.

[22]  Wesley E. Snyder,et al.  Demosaicking methods for Bayer color arrays , 2002, J. Electronic Imaging.

[23]  Jean-Philippe Tarel,et al.  BLIND CONTRAST ENHANCEMENT ASSESSMENT BY GRADIENT RATIOING AT VISIBLE EDGES , 2011 .

[24]  Alessandro Foi,et al.  Image Denoising by Sparse 3-D Transform-Domain Collaborative Filtering , 2007, IEEE Transactions on Image Processing.

[25]  Neil D. B. Bruce ExpoBlend: Information preserving exposure blending based on normalized log-domain entropy , 2014, Comput. Graph..

[26]  Juan Song,et al.  Enhancement and noise reduction of very low light level images , 2012, Proceedings of the 21st International Conference on Pattern Recognition (ICPR2012).

[27]  Chenglin Wen,et al.  Fully Point-wise Convolutional Neural Network for Modeling Statistical Regularities in Natural Images , 2018, ACM Multimedia.

[28]  Qing Zhang,et al.  High-Quality Exposure Correction of Underexposed Photos , 2018, ACM Multimedia.

[29]  Joonki Paik,et al.  Dual Autoencoder Network for Retinex-Based Low-Light Image Enhancement , 2018, IEEE Access.

[30]  Glenn Healey,et al.  Radiometric CCD camera calibration and noise estimation , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[31]  Xuelong Li,et al.  Exposure Fusion Using Boosting Laplacian Pyramid , 2014, IEEE Transactions on Cybernetics.

[32]  Jian Sun,et al.  Guided Image Filtering , 2010, ECCV.

[33]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[34]  Gang Luo,et al.  Naturalness Preserved Image Enhancement Using a Priori Multi-Layer Lightness Statistics , 2018, IEEE Transactions on Image Processing.

[35]  Subhasis Chaudhuri,et al.  Bilateral Filter Based Compositing for Variable Exposure Photography , 2009, Eurographics.

[36]  Trevor Darrell,et al.  Caffe: Convolutional Architecture for Fast Feature Embedding , 2014, ACM Multimedia.

[37]  Shree K. Nayar,et al.  Modeling the space of camera response functions , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[38]  Marc Pollefeys,et al.  A Dataset of Flash and Ambient Illumination Pairs from the Crowd , 2018, ECCV.

[39]  Jing Zhang,et al.  Fast Haze Removal for Nighttime Image Using Maximum Reflectance Prior , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[40]  David A. Forsyth,et al.  A Novel Approach To Colour Constancy , 1988, [1988 Proceedings] Second International Conference on Computer Vision.

[41]  Soumik Sarkar,et al.  LLNet: A deep autoencoder approach to natural low-light image enhancement , 2015, Pattern Recognit..

[42]  Zhengguo Li,et al.  Multi-scale exposure fusion via gradient domain guided image filtering , 2017, 2017 IEEE International Conference on Multimedia and Expo (ICME).