A biologically inspired algorithm for the recovery of shading and reflectance images

We present an algorithm for separating the shading and reflectance images of photographed natural scenes. The algorithm exploits the constraint that in natural scenes chromatic and luminance variations that are co-aligned mainly arise from changes in surface reflectance, whereas near-pure luminance variations mainly arise from shading and shadows. The novel aspect of the algorithm is the initial separation of the image into luminance and chromatic image planes that correspond to the luminance, red–green, and blue–yellow channels of the primate visual system. The red–green and blue–yellow image planes are analysed to provide a map of the changes in surface reflectance, which is then used to separate the reflectance from shading changes in both the luminance and chromatic image planes. The final reflectance image is obtained by reconstructing the chromatic and luminance-reflectance-change maps, while the shading image is obtained by subtracting the reconstructed luminance-reflectance image from the original luminance image. A number of image examples are included to illustrate the successes and limitations of the algorithm.

[1]  J. Pokorny,et al.  Spectral sensitivity of the foveal cone photopigments between 400 and 500 nm , 1975, Vision Research.

[2]  A. Gilchrist,et al.  An anchoring theory of lightness perception. , 1999 .

[3]  L. Pessoa,et al.  Finding out about filling-in: a guide to perceptual completion for visual science and the philosophy of perception. , 1998, The Behavioral and brain sciences.

[4]  H. Barrow,et al.  RECOVERING INTRINSIC SCENE CHARACTERISTICS FROM IMAGES , 1978 .

[5]  P. Lennie,et al.  Chromatic mechanisms in lateral geniculate nucleus of macaque. , 1984, The Journal of physiology.

[6]  E. Land,et al.  Lightness and retinex theory. , 1971, Journal of the Optical Society of America.

[7]  V. S. Ramachandran,et al.  Perception of shape from shading , 1988, Nature.

[8]  J Sun,et al.  Shading and Stereo in Early Perception of Shape and Reflectance , 1997, Perception.

[9]  Yair Weiss,et al.  Deriving intrinsic images from image sequences , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[10]  J. Koenderink,et al.  Spatial and temporal discrimination ellipsoids in color space. , 1983, Journal of the Optical Society of America.

[11]  M. McCourt,et al.  A multiscale spatial filtering account of the White effect, simultaneous brightness contrast and grating induction , 1999, Vision Research.

[12]  J. M. Rubin,et al.  Color vision and image intensities: When are changes material? , 1982, Biological Cybernetics.

[13]  S. Grossberg,et al.  Neural dynamics of 1-D and 2-D brightness perception: A unified model of classical and recent phenomena , 1988, Perception & psychophysics.

[14]  Frederick A A Kingdom,et al.  Colour Vision Brings Clarity to Shadows , 2004, Perception.

[15]  A. Gilchrist,et al.  The classification and integration of edges as critical to the perception of reflectance and illumination , 1983, Perception & psychophysics.

[16]  Terrence J. Sejnowski,et al.  Network model of shape-from-shading: neural function arises from both receptive and projective fields , 1988, Nature.

[17]  Mark S. Drew,et al.  Removing Shadows from Images , 2002, ECCV.

[18]  W. McIlhagga,et al.  Detection mechanisms in L-, M-, and S-cone contrast space. , 1993, Journal of the Optical Society of America. A, Optics and image science.

[19]  P. Cavanagh,et al.  Shape from shadows. , 1989, Journal of experimental psychology. Human perception and performance.

[20]  Frederick A A Kingdom,et al.  Color brings relief to human vision , 2003, Nature Neuroscience.

[21]  Ione Fine,et al.  Surface segmentation based on the luminance and color statistics of natural scenes. , 2003, Journal of the Optical Society of America. A, Optics, image science, and vision.

[22]  C. F. Stromeyer,et al.  Second-site adaptation in the red-green chromatic pathways , 1985, Vision Research.

[23]  Mark S. Drew,et al.  Removing Shadows From Images using Retinex , 2002, CIC.

[24]  Edward H. Adelson,et al.  Recovering intrinsic images from a single image , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[25]  R. M. Boynton,et al.  Chromaticity diagram showing cone excitation by stimuli of equal luminance. , 1979, Journal of the Optical Society of America.

[26]  A. Gilchrist,et al.  An anchoring theory of lightness perception. , 1999, Psychological review.

[27]  T Troscianko,et al.  Color and luminance information in natural scenes. , 1998, Journal of the Optical Society of America. A, Optics, image science, and vision.

[28]  D. Tolhurst,et al.  Spatiochromatic Properties of Natural Images and Human Vision , 2002, Current Biology.

[29]  A. Gilchrist.  The perception of surface blacks and whites. , 1979, Scientific American.

[30]  D. Knill,et al.  Implications of a Bayesian formulation of visual information for processing for psychophysics , 1996 .

[31]  F. Kingdom,et al.  Border effects on brightness: a review of findings, models and issues. , 1988, Spatial vision.

[32]  K. Mullen,et al.  Postreceptoral chromatic detection mechanisms revealed by noise masking in three-dimensional cone contrast space. , 1997, Journal of the Optical Society of America. A, Optics, image science, and vision.

[33]  John F. Canny,et al.  A Computational Approach to Edge Detection , 1986, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[34]  R. L. de Valois,et al.  Analysis and coding of color vision in the primate visual system. , 1965, Cold Spring Harbor symposia on quantitative biology.

[35]  Theo Gevers,et al.  Classifying color transitions into shadow-geometry, illumination, highlight or material edges , 2000, Proceedings 2000 International Conference on Image Processing (Cat. No.00CH37101).