Directed color transfer for low-light image enhancement

Abstract Underexposed, low-light, images are acquired when scene illumination is insufficient for a given camera. Camera limitation originates in the high chance of producing motion blurred images due to shaky hands. In this paper we suggest to actively use underexposing as a measure to prevent motion blurred images to appear and propose a novel color transfer as a method for low light image amplification. The proposed solution envisages a dual acquisition, containing a normally exposed, possibly blurred image and an underexposed/low-light, but sharp one. Good colors are learned from the normal exposed image and transferred to the low light one using a framework matching solution. To ensure that the transfer is spatially consistent, the images are divided into luminance perceptual consistent patches called frameworks and the optimal mapping is piece-wise approximated. The two image may differ by colors and subject to improve the robustness of the spatial matching, we added supplementary extreme channels. The proposed method shows robust results from both an objective and a subjective point of view.

[1]  Ge Li,et al.  LECARM: Low-Light Image Enhancement Using the Camera Response Model , 2019, IEEE Transactions on Circuits and Systems for Video Technology.

[2]  Charu C. Aggarwal,et al.  On the Surprising Behavior of Distance Metrics in High Dimensional Spaces , 2001, ICDT.

[3]  Massimo Bergamasco,et al.  Desktop Haptic Interface for Simulation of Hand-Tremor , 2016, IEEE Transactions on Haptics.

[4]  Soumik Sarkar,et al.  LLNet: A deep autoencoder approach to natural low-light image enhancement , 2015, Pattern Recognit..

[5]  Xuezhi Chi,et al.  An improved fuzzy C-means clustering algorithm based on simulated annealing , 2013, 2013 10th International Conference on Fuzzy Systems and Knowledge Discovery (FSKD).

[6]  Paul Crawford,et al.  Differentiation and diagnosis of tremor. , 2011, American family physician.

[7]  James C. Bezdek,et al.  Pattern Recognition with Fuzzy Objective Function Algorithms , 1981, Advanced Applications in Pattern Recognition.

[8]  W. T. Ang,et al.  Estimation and filtering of physiological tremor for real‐time compensation in surgical robotics applications , 2010, The international journal of medical robotics + computer assisted surgery : MRCAS.

[9]  Robert B. Fisher,et al.  Color Homography: Theory and Applications , 2019, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[11]  Pedro M. Domingos.  A few useful things to know about machine learning , 2012, Commun. ACM.

[12]  Frédo Durand,et al.  Understanding Blind Deconvolution Algorithms , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[13]  Rui Zhang,et al.  Global sparse gradient guided variational Retinex model for image enhancement , 2017, Signal Process. Image Commun..

[14]  Erik Reinhard,et al.  Progressive color transfer for images of arbitrary dynamic range , 2011, Comput. Graph..

[15]  Panagiotis Tsakalides,et al.  Low Light Image Enhancement via Sparse Representations , 2014, ICIAR.

[16]  Gabriel Peyré,et al.  Regularized Discrete Optimal Transport , 2014, SIAM J. Imaging Sci..

[17]  J. C. Dunn,et al.  A Fuzzy Relative of the ISODATA Process and Its Use in Detecting Compact Well-Separated Clusters , 1973 .

[18]  Yanning Zhang,et al.  Multi-image Blind Deblurring Using a Coupled Adaptive Sparse Prior , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[19]  Zhixun Su,et al.  Blur kernel estimation via salient edges and low rank prior for blind image deblurring , 2017, Signal Process. Image Commun..

[20]  Miguel Oliveira,et al.  Unsupervised local color correction for coarsely registered images , 2011, CVPR 2011.

[21]  Jian Sun,et al.  Single image haze removal using dark channel prior , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[22]  Erik Reinhard,et al.  Color Transfer between Images , 2001, IEEE Computer Graphics and Applications.

[23]  Cameron N. Riviere,et al.  Physiological tremor amplitude during retinal microsurgery , 2002, Proceedings of the IEEE 28th Annual Northeast Bioengineering Conference (IEEE Cat. No.02CH37342).

[24]  D. Ruderman,et al.  Statistics of cone responses to natural images: implications for visual coding , 1998 .

[25]  Sophie Triantaphillidou,et al.  Exposure and image control , 2011 .

[26]  Zhou Wang,et al.  Objective Quality Assessment of Tone-Mapped Images , 2013, IEEE Transactions on Image Processing.

[27]  Jitendra Malik,et al.  Recovering high dynamic range radiance maps from photographs , 1997, SIGGRAPH '08.

[28]  Miguel Oliveira,et al.  A Probabilistic Approach for Color Correction in Image Mosaicking Applications , 2015, IEEE Transactions on Image Processing.

[29]  S. B. Kang,et al.  Image deblurring using inertial measurement sensors , 2010, SIGGRAPH 2010.

[30]  Jean Ponce,et al.  Learning a convolutional neural network for non-uniform motion blur removal , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[31]  A. Gilchrist,et al.  An anchoring theory of lightness perception. , 1999 .

[32]  Bernhard Schölkopf,et al.  Learning to Deblur , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[33]  Michal Irani,et al.  Blind Deblurring Using Internal Patch Recurrence , 2014, ECCV.

[34]  Jean Ponce,et al.  Non-uniform Deblurring for Shaken Images , 2012, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[35]  Bernhard Schölkopf,et al.  Fast removal of non-uniform camera shake , 2011, 2011 International Conference on Computer Vision.

[36]  Ankit Gupta,et al.  Single Image Deblurring Using Motion Density Functions , 2010, ECCV.

[37]  Hans-Peter Seidel,et al.  Lightness Perception in Tone Reproduction for High Dynamic Range Images , 2005, Comput. Graph. Forum.

[38]  Nicolas Papadakis,et al.  Superpixel-based color transfer , 2017, 2017 IEEE International Conference on Image Processing (ICIP).

[39]  Xu Zhou,et al.  Variational Bayesian Blind Image Deconvolution: A review , 2015, Digit. Signal Process..

[40]  Yair Weiss,et al.  From learning models of natural image patches to whole image restoration , 2011, 2011 International Conference on Computer Vision.

[41]  Harry Shum,et al.  Blurred/Non-Blurred Image Alignment using Sparseness Prior , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[42]  Cheolkon Jung,et al.  Low light image enhancement with dual-tree complex wavelet transform , 2017, J. Vis. Commun. Image Represent..

[43]  Lihi Zelnik-Manor,et al.  Photorealistic Style Transfer with Screened Poisson Equation , 2017, BMVC.

[44]  Yacov Hel-Or,et al.  Piecewise-consistent color mappings of images acquired under various conditions , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[45]  Jan Kautz,et al.  Exposure Fusion , 2007 .

[46]  Corneliu Florea,et al.  Avoiding the Deconvolution: Framework Oriented Color Transfer for Enhancing Low-Light Images , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[47]  Delu Zeng,et al.  A fusion-based enhancing method for weakly illuminated images , 2016, Signal Process..

[48]  Joonki Paik,et al.  Variational framework for low-light image enhancement using optimal transmission map and combined ℓ2 and ℓ2-minimization , 2017, Signal Process. Image Commun..

[49]  Erik Reinhard,et al.  Colour Mapping: A Review of Recent Methods, Extensions and Applications , 2016, Comput. Graph. Forum.

[50]  Zhixun Su,et al.  Kernel estimation from salient structure for robust motion deblurring , 2012, Signal Process. Image Commun..

[51]  François Pitié,et al.  Automated colour grading using colour distribution transfer , 2007, Comput. Vis. Image Underst..