UvA-DARE (Digital Academic Repository) Color Constancy by Deep Learning Color Constancy by Deep Learning

Computational color constancy aims to estimate the color of the light source. The performance of many vision tasks, such as object detection and scene understanding, may benefit from color constancy by using the corrected object colors. Since traditional color constancy methods are based on specific assumptions, none of those methods can be used as a universal predictor. Further, shallow learning schemes are used for training-based color constancy, possibly suffering from limited learning capabilities. In this paper, we propose a new framework using Deep Neural Networks (DNNs) to obtain accurate light source estimation. We reformulate color constancy as a DNN-based regression approach to estimate the color of the light source. The model is trained using datasets of more than a million images. Experiments show that the proposed algorithm outperforms the state-of-the-art by 9%. Especially in cross dataset validation, our approach reduces the median angular error by 35%. Our algorithm operates at more than 100 fps during testing.

[1]  E. Land,et al.  Lightness and retinex theory. , 1971, Journal of the Optical Society of America.

[2]  G. Buchsbaum A spatial processor model for object colour perception , 1980 .

[3]  G D Finlayson,et al.  Spectral sharpening: sensor transformations for improved color constancy. , 1994, Journal of the Optical Society of America. A, Optics, image science, and vision.

[4]  Brian V. Funt Color constancy in digital imagery , 1999, Proceedings 1999 International Conference on Image Processing (Cat. 99CH36348).

[5]  Graham D. Finlayson,et al.  Selection for gamut mapping colour constancy , 1999, Image Vis. Comput..

[6]  B. Funt,et al.  Diagonal versus affine transformations for color correction. , 2000, Journal of the Optical Society of America. A, Optics, image science, and vision.

[7]  Joost van de Weijer,et al.  Colour Constancy from Hyper-Spectral Data , 2000, BMVC.

[8]  Brian V. Funt,et al.  A Large Image Database for Color Constancy Research , 2003, CIC.

[9]  Henrikas Vaitkevicius,et al.  Investigation of color constancy with a neural network , 2004, Neural Networks.

[10]  Graham D. Finlayson,et al.  Shades of Gray and Colour Constancy , 2004, CIC.

[11]  Brian V. Funt,et al.  Estimating Illumination Chromaticity via Support Vector Regression , 2004, Color Imaging Conference.

[12]  Ingeborg Tastl,et al.  Gamut Constrained Illuminant Estimation , 2006, International Journal of Computer Vision.

[13]  Joost van de Weijer,et al.  Author Manuscript, Published in "ieee Transactions on Image Processing Edge-based Color Constancy , 2022 .

[14]  Cordelia Schmid,et al.  Using High-Level Visual Information for Color Constancy , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[15]  Andrew Blake,et al.  Bayesian color constancy revisited , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[16]  Joost van de Weijer,et al.  Generalized Gamut Mapping using Image Derivative Structures for Color Constancy , 2008, International Journal of Computer Vision.

[17]  Gérard G. Medioni,et al.  Color constancy using denoising methods and cepstral analysis , 2009, 2009 16th IEEE International Conference on Image Processing (ICIP).

[18]  Fei-Fei Li,et al.  ImageNet: A large-scale hierarchical image database , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[19]  Sanjit K. Mitra,et al.  Color constancy in the compressed domain , 2009, 2009 16th IEEE International Conference on Image Processing (ICIP).

[20]  Theo Gevers,et al.  Color Constancy Using Natural Image Statistics and Scene Semantics , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[21]  Mark S. Drew,et al.  Specularity, the Zeta-image, and Information-Theoretic Illuminant Estimation , 2012, ECCV Workshops.

[22]  Mark S. Drew,et al.  White Patch Gamut Mapping Colour Constancy , 2012, 2012 19th IEEE International Conference on Image Processing.

[23]  Keigo Hirakawa,et al.  Color Constancy with Spatio-Spectral Statistics , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[24]  Mark S. Drew,et al.  Exemplar-Based Colour Constancy , 2012, BMVC.

[25]  Enhong Chen,et al.  Image Denoising and Inpainting with Deep Neural Networks , 2012, NIPS.

[26]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[27]  Dani Lischinski,et al.  Illuminant Chromaticity from Image Sequences , 2013, 2013 IEEE International Conference on Computer Vision.

[28]  Hongbin Zha,et al.  Canonicalized central absolute moment for edge-based color constancy , 2013, 2013 IEEE International Conference on Image Processing.

[29]  Graham D. Finlayson,et al.  Corrected-Moment Illuminant Estimation , 2013, 2013 IEEE International Conference on Computer Vision.

[30]  Mark S. Drew,et al.  Exemplar-Based Color Constancy and Multiple Illumination , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[31]  Yongjie Li,et al.  Efficient Color Constancy with Local Surface Reflectance Statistics , 2014, ECCV.

[32]  Theo Gevers,et al.  Color Constancy Using 3D Scene Geometry Derived From a Single Image , 2014, IEEE Transactions on Image Processing.

[33]  Dilip K Prasad,et al.  Illuminant estimation for color constancy: why spatial-domain methods work and the role of the color distribution. , 2014, Journal of the Optical Society of America. A, Optics, image science, and vision.

[34]  Raimondo Schettini,et al.  Color constancy using CNNs , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).