Artifact-Free Low-Light Video Enhancement Using Temporal Similarity and Guide Map

This paper presents a low-light video restoration algorithm using similar patches among temporally adjacent frames. The proposed artifact-free low-light video restoration algorithm consists of three steps: 1) brightness enhancement using similar patches among temporally adjacent frames and adaptive accumulation; 2) improved color assignment to reduce color distortion; and 3) image fusion for saturation reduction using the guide map. The proposed brightness enhancement step guarantees not to produce any undesired artifacts because of searching the most similar patches among given set of temporally adjacent frames. The color assignment and fusion steps enable a fully automatic color preservation and average brightness control. Experimental results show that the proposed algorithm can better restore high-quality videos without undesired artifacts such as noise amplification, flicker, color distortion, and brightness saturation. As a result, the proposed algorithm can be implemented in a wide range of digital imaging applications such as video surveillance systems and advanced driver assistance systems.

[1]  E. Land,et al.  Lightness and retinex theory. , 1971, Journal of the Optical Society of America.

[2]  Jungyoup Yang,et al.  Flickering effect reduction for H.264/AVC intra frames , 2006, SPIE Optics East.

[3]  Hanseok Ko,et al.  A novel approach for denoising and enhancement of extremely low-light video , 2015, IEEE Transactions on Consumer Electronics.

[4]  Joonki Paik,et al.  Adaptive contrast enhancement using gain-controllable clipped histogram equalization , 2008, IEEE Transactions on Consumer Electronics.

[5]  Rahman Zia-ur,et al.  A Multiscale Retinex for Color Rendition and Dynamic Range Compression , 1996 .

[6]  Joost van de Weijer,et al.  Computational Color Constancy: Survey and Experiments , 2011, IEEE Transactions on Image Processing.

[7]  Stephen Lin,et al.  A New In-Camera Imaging Model for Color Computer Vision and Its Application , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[8]  Weisi Lin,et al.  The Analysis of Image Contrast: From Quality Assessment to Automatic Enhancement , 2016, IEEE Transactions on Cybernetics.

[9]  Wenjun Zhang,et al.  No-Reference Quality Metric of Contrast-Distorted Images Based on Information Maximization , 2017, IEEE Transactions on Cybernetics.

[10]  Shih-Chia Huang,et al.  Automatic Moving Object Extraction Through a Real-World Variable-Bandwidth Network for Traffic Monitoring Systems , 2014, IEEE Transactions on Industrial Electronics.

[11]  Yao Lu,et al.  Fast efficient algorithm for enhancement of low lighting video , 2011, ICME.

[12]  Joonki Paik,et al.  Flicker-free low-light video enhancement using patch-similarity and adaptive accumulation , 2016, 2016 IEEE International Conference on Consumer Electronics (ICCE).

[13]  Sven Loncaric,et al.  Light Random Sprays Retinex: Exploiting the Noisy Illumination Estimation , 2013, IEEE Signal Processing Letters.

[14]  Jean-Philippe Tarel,et al.  BLIND CONTRAST ENHANCEMENT ASSESSMENT BY GRADIENT RATIOING AT VISIBLE EDGES , 2011 .

[15]  Shih-Chia Huang,et al.  An Advanced Motion Detection Algorithm With Video Quality Analysis for Video Surveillance Systems , 2011, IEEE Transactions on Circuits and Systems for Video Technology.

[16]  Christophe Schlick,et al.  Quantization Techniques for Visualization of High Dynamic Range Pictures , 1995 .

[17]  Ren C. Luo,et al.  Multisensor Fusion-Based Concurrent Environment Mapping and Moving Object Detection for Intelligent Service Robotics , 2014, IEEE Transactions on Industrial Electronics.

[18]  Nikola Banić,et al.  Smart light random memory sprays Retinex: a fast Retinex implementation for high-quality brightness adjustment and color correction. , 2015, Journal of the Optical Society of America. A, Optics, image science, and vision.

[19]  Joonki Paik,et al.  Image Processing-based Validation of Unrecognizable Numbers in Severely Distorted License Plate Images , 2012 .

[20]  Zia-ur Rahman,et al.  A multiscale retinex for bridging the gap between color images and the human observation of scenes , 1997, IEEE Trans. Image Process..

[21]  Hanseok Ko,et al.  A novel framework for extremely low-light video enhancement , 2014, 2014 IEEE International Conference on Consumer Electronics (ICCE).

[22]  G. Buchsbaum.  A spatial processor model for object colour perception , 1980 .

[23]  J. Alex Stark,et al.  Adaptive image contrast enhancement using generalizations of histogram equalization , 2000, IEEE Trans. Image Process..

[24]  Young Shik Moon,et al.  Flickering Effect Reduction Based on the Modified Transformation Function for Video Contrast Enhancement , 2014 .

[25]  Sung-Jea Ko,et al.  Image Contrast Enhancement Based on a Multi-Cue Histogram , 2015 .

[26]  Weisi Lin,et al.  No-Reference Image Sharpness Assessment in Autoregressive Parameter Space , 2015, IEEE Transactions on Image Processing.

[27]  Sergio Goma,et al.  Camera Technology at the dawn of digital renascence era , 2010, 2010 Conference Record of the Forty Fourth Asilomar Conference on Signals, Systems and Computers.

[28]  Alessandro Rizzi,et al.  Random Spray Retinex: A New Retinex Implementation to Investigate the Local Properties of the Model , 2007, IEEE Transactions on Image Processing.

[29]  Raanan Fattal,et al.  Image and video upscaling from local self-examples , 2011, TOGS.

[30]  Aichi Chien,et al.  An L1-based variational model for Retinex theory and its application to medical images , 2011, CVPR 2011.

[31]  Wei Zeng,et al.  Night video enhancement using improved dark channel prior , 2013, 2013 IEEE International Conference on Image Processing.

[32]  Rae-Hong Park,et al.  Histogram-Based Locality-Preserving Contrast Enhancement , 2015, IEEE Signal Processing Letters.

[33]  Xiaoou Tang,et al.  Single Image Haze Removal Using Dark Channel Prior , 2011 .

[34]  Abd. Rahman Ramli,et al.  Contrast enhancement using recursive mean-separate histogram equalization for scalable brightness preservation , 2003, IEEE Trans. Consumer Electron..

[35]  Eunsung Lee,et al.  Color Enhancement of Low Exposure Images using Histogram Specification and its Application to Color Shift Model-Based Refocusing , 2012 .

[36]  Michael S. Brown,et al.  Globally Optimized Linear Windowed Tone Mapping , 2010, IEEE Transactions on Visualization and Computer Graphics.

[37]  E. Land.  The retinex theory of color vision. , 1977, Scientific American.

[38]  Joonki Paik,et al.  Flicker removal for CMOS wide dynamic range imaging based on alternating current component analysis , 2014, IEEE Transactions on Consumer Electronics.

[39]  Alessandro Rizzi,et al.  Mathematical definition and analysis of the retinex algorithm. , 2005, Journal of the Optical Society of America. A, Optics, image science, and vision.

[40]  D H Brainard,et al.  Analysis of the retinex theory of color vision. , 1986, Journal of the Optical Society of America. A, Optics and image science.

[41]  Zia-ur Rahman,et al.  Properties and performance of a center/surround retinex , 1997, IEEE Trans. Image Process..

[42]  Hai-Miao Hu,et al.  Naturalness Preserved Enhancement Algorithm for Non-Uniform Illumination Images , 2013, IEEE Transactions on Image Processing.

[43]  David A. Forsyth,et al.  Generalizing motion edits with Gaussian processes , 2009, ACM Trans. Graph..

[44]  Wenjun Zhang,et al.  Automatic Contrast Enhancement Technology With Saliency Preservation , 2015, IEEE Transactions on Circuits and Systems for Video Technology.

[45]  Kuldeep Singh,et al.  Enhancement of low exposure images via recursive histogram equalization algorithms , 2015 .

[46]  Farzaneh Abdollahi,et al.  A Decentralized Cooperative Control Scheme With Obstacle Avoidance for a Team of Mobile Robots , 2014, IEEE Transactions on Industrial Electronics.

[47]  Sabu Emmanuel,et al.  ACE–An Effective Anti-forensic Contrast Enhancement Technique , 2016, IEEE Signal Processing Letters.

[48]  Joonki Paik,et al.  Spatially Adaptive Histogram Equalization for Single Image-Based Ghost-free High Dynamic Range Imaging , 2014 .