Maximum Entropy Spectral Models for Color Constancy

Earlier work showed that maximum entropy models can be used to represent surface reflectance spectra of Munsell patches. Here, we introduce a new approach to color constancy which is based upon that work. To our knowledge, all color constancy approaches employing spectral models use linear basis function representations for surface and illuminant spectra. This means that a set of basis functions has to be specified in advance in these algorithms. The proposed maximum entropy approach does not require this a priori information and therefore has a major advantage over other spectral based color constancy approaches. We show that a maximum entropy approach can be used to estimate surface and illuminant spectra given only camera sensor responses. We test our approach both in simulation and experiment. We also show that the performance of the proposed approach is similar to the most successful spectral based color constancy approach. This comparison is carried out in simulation in the presence of noise. Introduction Color constancy is the ability of a vision system to compute a measure of a surface’s color that is independent of the spectrum of the light incident on a surface. Obtaining such a measure is essential when using color as a cue in machine vision tasks, such as object recognition. This measure can be in 3D vector form (RGB, CMY, YIQ, etc.) or in spectral form. The latter constitutes the surface reflectance spectrum, which is the amount of light reflected off the surface at each wavelength. Most recent work on color constancy has focused on using 3D color models [14, 15, 16] because the information they provide is sufficient for a large class of problems. However, color constancy using spectral models can be useful when there is a need for a more accurate representation of a color. For example, when classifying vegetation, the difference between the colors of leaves may be sufficiently small to require the extra information contained in the spectral models. Moreover, employing a spectral model for a surface color constitutes a universal representation across different color spaces (RGB, CMY, YIQ, etc.). This can be useful when images of the same scene are taken with cameras of sensor spectral sensitivities belonging to different color spaces. In this case, employing 3D color models of objects is cumbersome in the sense that different color spaces imply different sensor responses for the same object. Spectral models have been used in a number of color constancy approaches [3, 4, 5, 11, 12, 13]. All these approaches have one major aspect in common. They represent surface and illuminant spectra by linear combinations of spectral basis functions. These basis functions are typically obtained by performing principal components analysis (PCA) on the sets of surface and illuminant spectra. This means that these sets should be available prior to applying the color constancy approach. However, these databases might not be available in advance. Even if they are available, they might not be consistent with the data used in a certain application. We introduce a new maximum entropy spectral based approach to color constancy which does not require a set of basis functions to be specified in advance. We build upon previous work [1] in which maximum entropy models were successfully used to estimate Munsell patch reflectance spectra given only photoreceptor responses. In [1] the illuminant was assumed to be constant or white, which is not the case in this work. The use of maximum entropy models was inspired by Jaynes, who stated that a physical quantity frequently observed in practice will tend to a value that can be produced in the largest number of ways [2]. In the case of physical processes representing spectra, many surfaces observed in our everyday-life surroundings have spectra that have high entropy, as opposed to monochromatic surfaces which have low entropy spectra [1]. Since we are concerned with color constancy where the illuminant is unknown, we seek a suitable model for the illuminant spectrum. Applying Jaynes’ argument, we can say that illuminants observed in our everyday-life surroundings have high entropy spectra, which therefore can be produced in the largest number of ways. This motivates us to represent the illuminant spectra with maximum entropy models as is the case for the surface reflectance spectra. The paper is organized as follows. First, the proposed maximum entropy approach is explained and derived. Second, the performance of our approach is analyzed both in simulation and experiment. Third, the proposed approach is compared to the best color constancy algorithm employing spectral models. This comparison is performed in simulation in the presence of noise. Finally, the paper is ended with concluding remarks. Maximum Entropy Spectral Based Color Constancy A spectral based color constancy algorithm computes surface and illuminant spectra given sensor responses obtained from a camera. The responses can be computed by the following equation: Pk = M ∑ λ=1 Rk(λ )s(λ )e(λ ), k = 1,2, ..., p, (1) where p is the number of sensor classes, each denoted by k; λ denotes the wavelength, which is taken over the visible range. Pk is the computed sensor response for each sensor class k, each with a spectral sensitivity function Rk(λ ); s(λ ) is the surface reflectance spectrum; e(λ ) is the illuminant spectrum; M is the dimension of these spectra. Usually there are three sensor classes corresponding to each of the long-, medium-, and short-wavelength ranges. The proposed color constancy approach aims at recovering surface and illuminant spectra by representing them using maximum entropy models, given only sensor responses. For simplicity, we illustrate the approach with the case of one surface patch in a scene illuminated by a single light source. Note that color constancy problems are addressed for the cases when there are at least two surface patches in the scene, which makes our example a hypothetical case. In our hypothetical scene, s(λ ) is the surface spectrum and e(λ ) is the illuminant spectrum. We represent these spectra by probability distributions in order to compute their entropy. A light spectrum is a collection of photons that can be thought of as a histogram of photons over wavelength. Given a particular photon photon0, the spectral reflectance then represents the probability of a wavelength given photon0. Therefore s(λ ), for example, can be represented by the corresponding conditional probability distribution ps(λ |photon0). The same argument can be applied to the illuminant spectrum whose probability distribution representation can be denoted by pe(λ |photon0). In this paper we adopt the notations ps(λ ) and pe(λ ) for ps(λ |photon0) and pe(λ |photon0) for simplicity knowing that this does not affect the following derivations. Throughout this paper we may refer to the entropy of the probability distribution representation of a spectrum as the entropy of a spectrum for simplicity. ps(λ ) and pe(λ ) can be obtained from s(λ ) and e(λ ) by the following: ps(λ ) = s(λ )/ M ∑ λ=1 s(λ ), λ = 1, ...,M, (2a) pe(λ ) = e(λ )/ M ∑ λ=1 e(λ ), λ = 1, ...,M. (2b) Our goal is to estimate surface and illuminant spectra by representing each by a maximum entropy model. Jaynes showed that given measurements in the form of expectations, which is the case for the sensor responses given by Equation 1, the probability distribution which maximizes the entropy can be computed and is in the form of a product of exponentials [2]: