When Color Constancy Goes Wrong: Correcting Improperly White-Balanced Images

This paper focuses on correcting a camera image that has been improperly white-balanced. This situation occurs when a camera's auto white balance fails or when the wrong manual white-balance setting is used. Even after decades of computational color constancy research, there are no effective solutions to this problem. The challenge lies not in identifying what the correct white balance should have been, but in the fact that the in-camera white-balance procedure is followed by several camera-specific nonlinear color manipulations that make it challenging to correct the image's colors in post-processing. This paper introduces the first method to explicitly address this problem. Our method is enabled by a dataset of over 65,000 pairs of incorrectly white-balanced images and their corresponding correctly white-balanced images. Using this dataset, we introduce a k-nearest neighbor strategy that is able to compute a nonlinear color mapping function to correct the image's colors. We show our method is highly effective and generalizes well to camera models not in the training set.

[1]  Seoung Wug Oh,et al.  Approaching the computational color constancy as a classification problem through deep learning , 2016, Pattern Recognit..

[2]  Stephen Lin,et al.  Determining the radiometric response function from a single grayscale image , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[3]  Shree K. Nayar,et al.  What is the space of camera response functions? , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[4]  Andrew Zisserman,et al.  Three things everyone should know to improve object retrieval , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[5]  Trevor Darrell,et al.  From pixels to physics: Probabilistic color de-rendering , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[6]  Graham D. Finlayson,et al.  Shades of Gray and Colour Constancy , 2004, CIC.

[7]  Stephen Lin,et al.  FC^4: Fully Convolutional Color Constancy with Confidence-Weighted Pooling , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[8]  Peter A. Rhodes,et al.  A study of digital camera colorimetric characterisation based on polynomial modelling , 2001 .

[9]  Moncef Gabbouj,et al.  A Data Set for Camera-Independent Color Constancy , 2018, IEEE Transactions on Image Processing.

[10]  Stephen Lin,et al.  A New In-Camera Imaging Model for Color Computer Vision and Its Application , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  Marc Ebner,et al.  Color Constancy , 2007, Computer Vision, A Reference Guide.

[12]  Heung-Yeung Shum,et al.  Radiometric calibration from a single image , 2004, CVPR 2004.

[13]  Yun-Ta Tsai,et al.  Fast Fourier Color Constancy , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[14]  R. Hunt,et al.  Metamerism and Colour Constancy , 2011 .

[15]  Sylvain Paris,et al.  Learning photographic global tonal adjustment with a database of input / output image pairs , 2011, CVPR 2011.

[16]  Jitendra Malik,et al.  Recovering high dynamic range radiance maps from photographs , 1997, SIGGRAPH '08.

[17]  Trevor Darrell,et al.  Modeling Radiometric Uncertainty for Vision with Tone-Mapped Color Images , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[18]  Dilip K Prasad,et al.  Illuminant estimation for color constancy: why spatial-domain methods work and the role of the color distribution. , 2014, Journal of the Optical Society of America. A, Optics, image science, and vision.

[19]  Martin Fodslette Møller,et al.  A scaled conjugate gradient algorithm for fast supervised learning , 1993, Neural Networks.

[20]  Marc Pollefeys,et al.  Robust Radiometric Calibration and Vignetting Correction , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[21]  E.Y. Lam,et al.  Combining gray world and retinex theory for automatic white balance in digital photography , 2005, Proceedings of the Ninth International Symposium on Consumer Electronics, 2005. (ISCE 2005)..

[22]  David Connah,et al.  Weighted Constrained Hue-Plane Preserving Camera Characterization , 2016, IEEE Transactions on Image Processing.

[23]  Andrew Stockman.  Physiologically-based color matching functions , 2008, Color Imaging Conference.

[24]  Michael S. Brown,et al.  A Software Platform for Manipulating the Camera Imaging Pipeline , 2016, ECCV.

[25]  Gaurav Sharma.  Digital Color Imaging Handbook , 2002 .

[26]  Joost van de Weijer,et al.  Computational Color Constancy: Survey and Experiments , 2011, IEEE Transactions on Image Processing.

[27]  Roy S. Berns,et al.  Colorimetric characterization of a desktop drum scanner using a spectral model , 1995, J. Electronic Imaging.

[28]  Joost van de Weijer,et al.  Improving Color Constancy by Photometric Edge Weighting , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[29]  Matthew Anderson,et al.  Proposal for a Standard Default Color Space for the Internet - sRGB , 1996, CIC.

[30]  Michael S. Brown,et al.  RAW Image Reconstruction Using a Self-contained sRGB–JPEG Image with Small Memory Overhead , 2017, International Journal of Computer Vision.

[31]  Michael S. Brown,et al.  Revisiting radiometric calibration for color computer vision , 2011, 2011 International Conference on Computer Vision.

[32]  Wencheng Wu,et al.  The CIEDE2000 color-difference formula: Implementation notes, supplementary test data, and mathematical observations , 2005 .

[33]  Robert B. Fisher,et al.  Color Homography: Theory and Applications , 2019, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[34]  Seonghyeon Nam,et al.  Modelling the Scene Dependent Imaging in Cameras with a Deep Neural Network , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[35]  Michal Mackiewicz,et al.  Color Correction Using Root-Polynomial Regression , 2015, IEEE Transactions on Image Processing.

[36]  Raimondo Schettini,et al.  Adaptive Color Constancy Using Faces , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[37]  Andrew Blake,et al.  Bayesian color constancy revisited , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[38]  Geoffrey E. Hinton,et al.  Visualizing Data using t-SNE , 2008 .

[39]  Jonathan T. Barron,et al.  Convolutional Color Constancy , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[40]  D H Brainard,et al.  Analysis of the retinex theory of color vision. , 1986, Journal of the Optical Society of America. A, Optics and image science.

[41]  G. Buchsbaum.  A spatial processor model for object colour perception , 1980 .

[42]  Michael S. Brown,et al.  Beyond White: Ground Truth Colors for Color Constancy Correction , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[43]  Michael S. Brown,et al.  Improving Color Reproduction Accuracy on Cameras , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[44]  Mahdi Nezamabadi,et al.  Color Appearance Models , 2014, J. Electronic Imaging.