Computational Color Constancy: Survey and Experiments

Computational color constancy is a fundamental prerequisite for many computer vision applications. This paper presents a survey of many recent developments and state-of-the-art methods. Several criteria are proposed that are used to assess the approaches. A taxonomy of existing algorithms is proposed and methods are separated in three groups: static methods, gamut-based methods, and learning-based methods. Further, the experimental setup is discussed including an overview of publicly available datasets. Finally, various freely available methods, of which some are considered to be state of the art, are evaluated on two datasets.

[1]  E. Land,et al.  Lightness and retinex theory. , 1971, Journal of the Optical Society of America.

[2]  E. Land The retinex theory of color vision. , 1977, Scientific American.

[3]  G. Buchsbaum A spatial processor model for object colour perception , 1980 .

[4]  M. H. Brill,et al.  Necessary and sufficient conditions for Von Kries chromatic adaptation to give color constancy , 1982, Journal of mathematical biology.

[5]  Steven A. Shafer,et al.  Using color to separate reflection components , 1985 .

[6]  E. Land Recent advances in retinex theory , 1986, Vision Research.

[7]  H C Lee,et al.  Method for computing the scene-illuminant chromaticity from specular highlights. , 1986, Journal of the Optical Society of America. A, Optics and image science.

[8]  M. H. Brill,et al.  Heuristic analysis of von Kries color constancy. , 1986, Journal of the Optical Society of America. A, Optics and image science.

[9]  John K. Tsotsos,et al.  From [R, G, B] to Surface Reflectance: Computing Color Constant Descriptors in Images , 1987, IJCAI.

[10]  L. Maloney,et al.  Color constancy: a method for recovering surface spectral reflectance , 1987 .

[11]  B. Wandell,et al.  Standard surface-reflectance model and illuminant estimation , 1989 .

[12]  Edward H. Adelson,et al.  The Design and Use of Steerable Filters , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[13]  L. Arend,et al.  Simultaneous color constancy: paper with diverse Munsell values. , 1991, Journal of the Optical Society of America. A, Optics and image science.

[14]  Huang Yumin,et al.  A PHYSICAL APPROACH TO COLOR IMAGE UNDERSTANDING , 1991 .

[15]  Glenn Healey,et al.  Estimating spectral reflectance using highlights , 1991, Image Vis. Comput..

[16]  M. D'Zmura,et al.  Color constancy. II. Results for two-stage linear recovery of spectral descriptions for lights and surfaces. , 1993, Journal of the Optical Society of America. A, Optics, image science, and vision.

[17]  M. D'Zmura,et al.  Color constancy. I. Basic theory of two-stage linear recovery of spectral descriptions for lights and surfaces. , 1993, Journal of the Optical Society of America. A, Optics, image science, and vision.

[18]  G D Finlayson,et al.  Spectral sharpening: sensor transformations for improved color constancy. , 1994, Journal of the Optical Society of America. A, Optics, image science, and vision.

[19]  M. S. Drew,et al.  Color constancy - Generalized diagonal transforms suffice , 1994 .

[20]  Brian V. Funt,et al.  Color Constant Color Indexing , 1995, IEEE Trans. Pattern Anal. Mach. Intell..

[21]  Brian V. Funt,et al.  Color constancy under varying illumination , 1995, Proceedings of IEEE International Conference on Computer Vision.

[22]  Graham D. Finlayson,et al.  Color in Perspective , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[23]  Zia-ur Rahman,et al.  A multiscale retinex for bridging the gap between color images and the human observation of scenes , 1997, IEEE Trans. Image Process..

[24]  Brian V. Funt,et al.  Color Constancy for Scenes with Varying Illumination , 1997, Comput. Vis. Image Underst..

[25]  D H Brainard,et al.  Bayesian color constancy. , 1997, Journal of the Optical Society of America. A, Optics, image science, and vision.

[26]  Arnold W. M. Smeulders,et al.  Color Based Object Recognition , 1997, ICIAP.

[27]  Zia-ur Rahman,et al.  Properties and performance of a center/surround retinex , 1997, IEEE Trans. Image Process..

[28]  Alexander H. Waibel,et al.  Visual tracking for multimodal human computer interaction , 1998, CHI.

[29]  Brian V. Funt,et al.  Is Machine Colour Constancy Good Enough? , 1998, ECCV.

[30]  T Troscianko,et al.  Color and luminance information in natural scenes. , 1998, Journal of the Optical Society of America. A, Optics, image science, and vision.

[31]  Guillermo Sapiro,et al.  Color and Illuminant Voting , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[32]  Graham D. Finlayson,et al.  Selection for gamut mapping colour constancy , 1999, Image Vis. Comput..

[33]  Brian V. Funt,et al.  Committee-Based Color Constancy , 1999, Color Imaging Conference.

[34]  Brian V. Funt,et al.  Color Constancy with Specular and Non-Specular Surfaces , 1999, Color Imaging Conference.

[35]  D. Brainard,et al.  Mechanisms of color constancy under nearly natural viewing. , 1999, Proceedings of the National Academy of Sciences of the United States of America.

[36]  B. Funt,et al.  Diagonal versus affine transformations for color correction. , 2000, Journal of the Optical Society of America. A, Optics, image science, and vision.

[37]  Arnold W. M. Smeulders,et al.  PicToSeek: combining color and shape invariant features for image retrieval , 2000, IEEE Trans. Image Process..

[38]  Kobus Barnard,et al.  Improvements to Gamut Mapping Colour Constancy Algorithms , 2000, ECCV.

[39]  Graham D. Finlayson,et al.  Improving gamut mapping color constancy , 2000, IEEE Trans. Image Process..

[40]  Graham D. Finlayson,et al.  Color by Correlation: A Simple, Unifying Framework for Color Constancy , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[41]  Takeo Kanade,et al.  Bayesian color constancy for outdoor object recognition , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[42]  B A Wandell,et al.  Scene illuminant classification: brighter is better. , 2001, Journal of the Optical Society of America. A, Optics, image science, and vision.

[43]  Color constancy using KL-divergence , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[44]  K Barnard,et al.  Sensor sharpening for computational color constancy. , 2001, Journal of the Optical Society of America. A, Optics, image science, and vision.

[45]  Donald I. A. MacLeod,et al.  Influence of scene statistics on colour constancy , 2002, Nature.

[46]  Changjun Li,et al.  The CIECAM02 Color Appearance Model , 2002, CIC.

[47]  Brian V. Funt,et al.  A data set for color research , 2002 .

[48]  Kobus Barnard,et al.  Estimating the scene illumination chromaticity by using a neural network. , 2002, Journal of the Optical Society of America. A, Optics, image science, and vision.

[49]  Brian V. Funt,et al.  A comparison of computational color constancy algorithms. I: Methodology and experiments with synthesized data , 2002, IEEE Trans. Image Process..

[50]  Flávio P. Ferreira,et al.  Statistics of spatial cone-excitation ratios in natural scenes. , 2002, Journal of the Optical Society of America. A, Optics, image science, and vision.

[51]  Brian V. Funt,et al.  A comparison of computational color constancy Algorithms. II. Experiments with image data , 2002, IEEE Trans. Image Process..

[52]  Brian V. Funt,et al.  A Large Image Database for Color Constancy Research , 2003, CIC.

[53]  Arnold W. M. Smeulders,et al.  Color constancy from physical principles , 2003, Pattern Recognit. Lett..

[54]  G. Finlayson,et al.  Convex Programming Color Constancy , 2003 .

[55]  Brainard,et al.  Colour constancy: developing empirical tests of computational models , 2003 .

[56]  Hao Jiang,et al.  Nondiagonal color correction , 2003, Proceedings 2003 International Conference on Image Processing (Cat. No.03CH37429).

[57]  Tom Minka,et al.  Bayesian Color Constancy with Non-Gaussian Models , 2003, NIPS.

[58]  David A. Forsyth,et al.  A novel algorithm for color constancy , 1990, International Journal of Computer Vision.

[59]  Mark S. Drew,et al.  Color constancy from mutual reflection , 1991, International Journal of Computer Vision.

[60]  Peter B. Delahunt,et al.  Does human color constancy incorporate the statistical regularity of natural daylight? , 2004, Journal of vision.

[61]  Robby T. Tan,et al.  Color constancy through inverse-intensity chromaticity space. , 2004 .

[62]  Brian V. Funt,et al.  Failure of Luminance-Redness Correlation for Illuminant Estimation , 2004, Color Imaging Conference.

[63]  Gerald Schaefer,et al.  Solving for Colour Constancy using a Constrained Dichromatic Reflection Model , 2001, International Journal of Computer Vision.

[64]  Graham D. Finlayson,et al.  Shades of Gray and Colour Constancy , 2004, CIC.

[65]  K. Ikeuchi,et al.  Color constancy through inverse-intensity chromaticity space. , 2004, Journal of the Optical Society of America. A, Optics, image science, and vision.

[66]  Kinjiro Amano,et al.  Information limits on neural identification of colored surfaces in natural scenes , 2004, Visual Neuroscience.

[67]  Brian V. Funt,et al.  Estimating Illumination Chromaticity via Support Vector Regression , 2004, Color Imaging Conference.

[68]  Graham D. Finlayson,et al.  Colour constancy using the chromagenic constraint , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[69]  Ruixia Xu,et al.  Convex programming colour constancy with a diagonal-offset model , 2005, IEEE International Conference on Image Processing 2005.

[70]  Gerald Schaefer,et al.  A combined physical and statistical approach to colour constancy , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[71]  Frédéric Jurie,et al.  Learned color constancy from local correspondences , 2005, 2005 IEEE International Conference on Multimedia and Expo.

[72]  J.-P. Renno,et al.  Application and Evaluation of Colour Constancy in Visual Surveillance , 2005, 2005 IEEE International Workshop on Visual Surveillance and Performance Evaluation of Tracking and Surveillance.

[73]  D. Foster,et al.  Color constancy in natural scenes explained by global image statistics , 2006, Visual Neuroscience.

[74]  S. D. Hordley,et al.  Reevaluation of color constancy algorithm performance. , 2006, Journal of the Optical Society of America. A, Optics, image science, and vision.

[75]  Brian V. Funt,et al.  Stereo Retinex , 2006, The 3rd Canadian Conference on Computer and Robot Vision (CRV'06).

[76]  Ingeborg Tastl,et al.  Gamut Constrained Illuminant Estimation , 2006, International Journal of Computer Vision.

[77]  Vivek Agarwal,et al.  Estimating Illumination Chromaticity via Kernel Regression , 2006, 2006 International Conference on Image Processing.

[78]  Joost van de Weijer,et al.  Author Manuscript, Published in "ieee Transactions on Image Processing Edge-based Color Constancy , 2022 .

[79]  Byoung-Ho Kang,et al.  Automatic White Balancing via Gray Surface Identification , 2007, CIC.

[80]  Graham D. Finlayson,et al.  The bright-chromagenic algorithm for illuminant estimation , 2007, Color Imaging Conference.

[81]  Vivek Agarwal,et al.  Machine learning approach to color constancy , 2007, Neural Networks.

[82]  Homer H. Chen,et al.  Edge-based automatic white balancing with linear illuminant constraint , 2007, Electronic Imaging.

[83]  Cordelia Schmid,et al.  Using High-Level Visual Information for Color Constancy , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[84]  Theo Gevers,et al.  Color Constancy using Image Regions , 2007, 2007 IEEE International Conference on Image Processing.

[85]  Theo Gevers,et al.  Color Constancy using Natural Image Statistics , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[86]  Steven J. Gortler,et al.  The von Kries Hypothesis and a Basis for Color Constancy , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[87]  Javier Toro,et al.  A Multilinear Constraint on Dichromatic Planes for Illumination Estimation , 2007, IEEE Transactions on Image Processing.

[88]  Byoung-Ho Kang,et al.  Illumination Estimation via Thin-Plate Spline Interpolation , 2007, CIC.

[89]  T. Gevers,et al.  Color Constancy by Local Averaging , 2007, 14th International Conference of Image Analysis and Processing - Workshops (ICIAPW 2007).

[90]  M. H. Brill,et al.  Minimal Von Kries illuminant invariance , 2008 .

[91]  Gregor Fischer,et al.  whitebalPR - a New Method for Automatic White Balance , 2008, CGIV/MCS.

[92]  Xiaoyong Wang,et al.  Cluster Based Color Constancy , 2008, Color Imaging Conference.

[93]  Andrew Blake,et al.  Bayesian color constancy revisited , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[94]  Marc Ebner,et al.  Color constancy based on local space average color , 2009, Machine Vision and Applications.

[95]  Chun-Wang Sun,et al.  Representation or Context as a Cognitive Strategy in Colour Constancy? , 2008, Perception.

[96]  P. Bradley Constancy, Categories and Bayes: A New Approach to Representational Theories of Color Constancy , 2008 .

[97]  Carlo Gatta,et al.  A Spatially Variant White-Patch and Gray-World Method for Color Image Enhancement Driven by Local Contrast , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[98]  Raimondo Schettini,et al.  Improving Color Constancy Using Indoor–Outdoor Image Classification , 2008, IEEE Transactions on Image Processing.

[99]  Raimondo Schettini,et al.  Consensus-based framework for illuminant chromaticity estimation , 2008, J. Electronic Imaging.

[100]  Keigo Hirakawa,et al.  Color constancy beyond bags of pixels , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[101]  Jürgen Golz The role of chromatic scene statistics in color constancy: spatial integration. , 2008, Journal of vision.

[102]  Joost van de Weijer,et al.  Generalized Gamut Mapping using Image Derivative Structures for Color Constancy , 2008, International Journal of Computer Vision.

[103]  Frédo Durand,et al.  Light mixture estimation for spatially varying white balance , 2008, ACM Trans. Graph..

[104]  Lilong Shi,et al.  Dichromatic Illumination Estimation via Hough Transforms in 3D , 2008, CGIV/MCS.

[105]  Javier Toro Dichromatic illumination estimation without pre-segmentation , 2008, Pattern Recognit. Lett..

[106]  Javier Romero,et al.  Unsupervised illuminant estimation from natural scenes: an RGB digital camera suffices. , 2008, Applied optics.

[107]  Vasco M N de Almeida,et al.  Perception of Illuminant Colour Changes across Real Scenes , 2009, Perception.

[108]  Esa Rahtu,et al.  Applying Visual Object Categorization and Memory Colors for Automatic Color Constancy , 2009, ICIAP.

[109]  Achieving color constancy across multiple cameras , 2009, ACM Multimedia.

[110]  Alexa I Ruppertsberg,et al.  Color constancy improves for real 3D objects. , 2009, Journal of vision.

[111]  Jeroen J. M. Granzier,et al.  Can illumination estimates provide the basis for color constancy? , 2009, Journal of vision.

[112]  Javier Vazquez-Corral,et al.  Color Constancy Algorithms: Psychophysical Evaluation on a New Dataset , 2009 .

[113]  De Xu,et al.  Color constancy using stage classification , 2009, 2009 16th IEEE International Conference on Image Processing (ICIP).

[114]  Ning Wang,et al.  Edge-Based Color Constancy via Support Vector Regression , 2009, IEICE Trans. Inf. Syst..

[115]  Mongi A. Abidi,et al.  Illumination Chromaticity Estimation Using Linear Learning Methods , 2009 .

[116]  Gregor Fischer,et al.  Automatic white balance: whitebalPR using the dichromatic reflection model , 2009, Electronic Imaging.

[117]  Joost van de Weijer,et al.  Physics-based edge evaluation for improved color constancy , 2009, CVPR.

[118]  Stefano Messelodi,et al.  Computing von Kries Illuminant Changes by Piecewise Inversion of Cumulative Color Histograms , 2009 .

[119]  Raimondo Schettini,et al.  Region-Based Illuminant Estimation for Effective Color Correction , 2009, ICIAP.

[120]  Theo Gevers,et al.  Perceptual analysis of distance measures for color constancy algorithms. , 2009, Journal of the Optical Society of America. A, Optics, image science, and vision.

[121]  De Xu,et al.  Colour constancy based on texture similarity for natural images , 2009 .

[122]  De Xu,et al.  Color constancy using 3D scene geometry , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[123]  Jun Sun,et al.  Color constancy based on texture pyramid matching and regularized local regression. , 2010, Journal of the Optical Society of America. A, Optics, image science, and vision.

[124]  Lilong Shi,et al.  The Rehabilitation of MaxRGB , 2010, CIC.

[125]  Arnold W. M. Smeulders,et al.  Stages as Models of Scene Geometry , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[126]  De Xu,et al.  Color Constancy Using Achromatic Surface , 2010 .

[127]  F. Zaraga,et al.  White balance by tunable spectral responsivities. , 2010, Journal of the Optical Society of America. A, Optics, image science, and vision.

[128]  Raimondo Schettini,et al.  Automatic color constancy algorithm selection and combination , 2010, Pattern Recognit..

[129]  Lilong Shi,et al.  The effect of exposure on MaxRGB color constancy , 2010, Electronic Imaging.

[130]  Brian V. Funt,et al.  Cubical Gamut Mapping Colour Constancy , 2010, CGIV/MCS.

[131]  D. Foster Color constancy , 2011, Vision Research.

[132]  Theo Gevers,et al.  Color Constancy Using Natural Image Statistics and Scene Semantics , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[133]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[134]  R. Hunt,et al.  Metamerism and Colour Constancy , 2011 .

[135]  Donald D. Hoffman,et al.  Probabilistic Color Constancy , 2013 .