An Ideal Observer for Appearance: Reconstruction from Samples

Note: This report provides additional technical detail related to the Bayesian demosaicing method outlined in Brainard, D. H. (1994) Bayesian method for reconstructing color images from trichromatic samples. (1995) Reconstructing images from trichromatic samples: from basic research to practical applications. Image representation. I represent an image by the discrete function i(x i , y j , λ k). The coordinates x i (1 ≤ i ≤ N rows) and y j (1 ≤ j ≤ N cols) represent N rows N cols evenly spaced sample locations on a rectangular grid. This grid should be much denser than the sampling array being studied. The λ k (1 ≤ k ≤ N wls) represent the different color bands in the image. The λ k represent evenly spaced sample wavelengths throughout the visible spectrum, so that the entire function i(x i , y j , λ k) represents the spectrum at each image location. The use of discrete representations for both spatial position and wavelength has been discussed extensively elsewhere. i =1 +((n–1)moduloN rows) j =1 +((n–1)/N rows moduloN cols) k =1 + (n–1)/(N rows N cols) (1) This choice of indexing enumerates all of the values of the discrete function i(x i , y j , λ k). Sampling. Consider the relation between the response of a single receptor and the image data. Each receptor is completely characterized by a polychromatic receptive field, which specifies how strongly it responds to light from each color band at each location in the image. The receptive field of a single sensor may be described by a function s(x i , y j , λ k). The response r of the receptor is given by Σ i,j,k +e (2) Reconstruction from samples 3 where e is a random variable representing sensor noise. Note that this receptive field specification allows us to handle any optical blurring that precedes spatial samping, arbitrary sampling geometry, and the fact that different photoreceptors have different spectral sensitivities. We can rewrite Eq. 2 as r= s i+e (3) where the N rows N cols N wls dimensional row vector s is the vector representation of s(x i , y j , λ k). Suppose there are N rec sensors in the overall camera array, enumerated by the index m. The responses of all the sensors can be represented by a single N rec dimensional column vector r whose m th entry is …