Color Constancy Using Natural Image Statistics and Scene Semantics

Existing color constancy methods are all based on specific assumptions such as the spatial and spectral characteristics of images. As a consequence, no algorithm can be considered as universal. However, with the large variety of available methods, the question is how to select the method that performs best for a specific image. To achieve selection and combining of color constancy algorithms, in this paper natural image statistics are used to identify the most important characteristics of color images. Then, based on these image characteristics, the proper color constancy algorithm (or best combination of algorithms) is selected for a specific image. To capture the image characteristics, the Weibull parameterization (e.g., grain size and contrast) is used. It is shown that the Weibull parameterization is related to the image attributes to which the used color constancy methods are sensitive. An MoG-classifier is used to learn the correlation and weighting between the Weibull-parameters and the image attributes (number of edges, amount of texture, and SNR). The output of the classifier is the selection of the best performing color constancy method for a certain image. Experimental results show a large improvement over state-of-the-art single algorithms. On a data set consisting of more than 11,000 images, an increase in color constancy performance up to 20 percent (median angular error) can be obtained compared to the best-performing single algorithm. Further, it is shown that for certain scene categories, one specific color constancy algorithm can be used instead of the classifier considering several algorithms.

[1]  E. Land,et al.  Lightness and retinex theory. , 1971, Journal of the Optical Society of America.

[2]  E. Land.  The retinex theory of color vision. , 1977, Scientific American.

[3]  G. Buchsbaum.  A spatial processor model for object colour perception , 1980 .

[4]  M. H. Brill,et al.  Necessary and sufficient conditions for Von Kries chromatic adaptation to give color constancy , 1982, Journal of mathematical biology.

[5]  Edward H. Adelson,et al.  The Design and Use of Steerable Filters , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[6]  Heekuck Oh,et al.  Neural Networks for Pattern Recognition , 1993, Adv. Comput..

[7]  G D Finlayson,et al.  Spectral sharpening: sensor transformations for improved color constancy. , 1994, Journal of the Optical Society of America. A, Optics, image science, and vision.

[8]  D H Brainard,et al.  Bayesian color constancy. , 1997, Journal of the Optical Society of America. A, Optics, image science, and vision.

[9]  Alexander H. Waibel,et al.  Visual tracking for multimodal human computer interaction , 1998, CHI.

[10]  D. Ruderman,et al.  Statistics of cone responses to natural images: implications for visual coding , 1998 .

[11]  Arnold W. M. Smeulders,et al.  Color-based object recognition , 1997, Pattern Recognit..

[12]  Brian V. Funt,et al.  Committee-Based Color Constancy , 1999, Color Imaging Conference.

[13]  B. Funt,et al.  Diagonal versus affine transformations for color correction. , 2000, Journal of the Optical Society of America. A, Optics, image science, and vision.

[14]  Arnold W. M. Smeulders,et al.  PicToSeek: combining color and shape invariant features for image retrieval , 2000, IEEE Trans. Image Process..

[15]  Graham D. Finlayson,et al.  Color by Correlation: A Simple, Unifying Framework for Color Constancy , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[16]  Changjun Li,et al.  The CIECAM02 Color Appearance Model , 2002, CIC.

[17]  Brian V. Funt,et al.  A data set for color research , 2002 .

[18]  Brian V. Funt,et al.  A comparison of computational color constancy algorithms. I: Methodology and experiments with synthesized data , 2002, IEEE Trans. Image Process..

[19]  Brian V. Funt,et al.  A comparison of computational color constancy Algorithms. II. Experiments with image data , 2002, IEEE Trans. Image Process..

[20]  Ingeborg Tastl,et al.  Gamut Constrained Illuminant Estimation , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[21]  Brian V. Funt,et al.  A Large Image Database for Color Constancy Research , 2003, CIC.

[22]  Antonio Torralba,et al.  Statistics of natural image categories , 2003, Network.

[23]  David A. Forsyth,et al.  A novel algorithm for color constancy , 1990, International Journal of Computer Vision.

[24]  Peter B. Delahunt,et al.  Does human color constancy incorporate the statistical regularity of natural daylight? , 2004, Journal of vision.

[25]  Antonio Torralba,et al.  Modeling the Shape of the Scene: A Holistic Representation of the Spatial Envelope , 2001, International Journal of Computer Vision.

[26]  Graham D. Finlayson,et al.  Shades of Gray and Colour Constancy , 2004, CIC.

[27]  Antonio Torralba,et al.  Contextual Priming for Object Detection , 2003, International Journal of Computer Vision.

[28]  Gerald Schaefer,et al.  A combined physical and statistical approach to colour constancy , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[29]  Arnold W. M. Smeulders,et al.  c ○ 2005 Springer Science + Business Media, Inc. Manufactured in The Netherlands. A Six-Stimulus Theory for Stochastic Texture , 2002 .

[30]  D. Foster,et al.  Color constancy in natural scenes explained by global image statistics , 2006, Visual Neuroscience.

[31]  Steven D. Hordley,et al.  Scene illuminant estimation: Past, present, and future , 2006 .

[32]  Jan-Mark Geusebroek,et al.  Compact Object Descriptors from Local Colour Invariant Histograms , 2006, BMVC.

[33]  Cor J. Veenman,et al.  Robust Scene Categorization by Learning Image Statistics in Context , 2006, 2006 Conference on Computer Vision and Pattern Recognition Workshop (CVPRW'06).

[34]  S. D. Hordley,et al.  Reevaluation of color constancy algorithm performance. , 2006, Journal of the Optical Society of America. A, Optics, image science, and vision.

[35]  Marc Ebner,et al.  Evolving color constancy , 2006, Pattern Recognit. Lett..

[36]  Joost van de Weijer,et al.  Author Manuscript, Published in "ieee Transactions on Image Processing Edge-based Color Constancy , 2022 .

[37]  Cordelia Schmid,et al.  Using High-Level Visual Information for Color Constancy , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[38]  Theo Gevers,et al.  Color Constancy using Natural Image Statistics , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[39]  Steven J. Gortler,et al.  The von Kries Hypothesis and a Basis for Color Constancy , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[40]  Joost van de Weijer,et al.  Generalized Gamut Mapping using Image Derivative Structures for Color Constancy , 2008, International Journal of Computer Vision.

[41]  Theo Gevers,et al.  Perceptual analysis of distance measures for color constancy algorithms. , 2009, Journal of the Optical Society of America. A, Optics, image science, and vision.

[42]  R. Hunt,et al.  Metamerism and Colour Constancy , 2011 .

[43]  Donald D. Hoffman,et al.  Probabilistic Color Constancy , 2013 .