Color improves edge classification in human vision

Despite the complexity of the visual world, humans rarely confuse variations in illumination, for example shadows, from variations in material properties, such as paint or stain. This ability to distinguish illumination from material edges is crucial for determining the spatial layout of objects and surfaces in natural scenes. In this study, we explore the role that color (chromatic) cues play in edge classification. We conducted a psychophysical experiment that required subjects to classify edges into illumination and material, in patches taken from images of natural scenes that either contained or did not contain color information. The edge images were of various sizes and were pre-classified into illumination and material, based on inspection of the edge in the context of the whole image from which the edge was extracted. Edge classification performance was found to be superior for the color compared to grayscale images, in keeping with color acting as a cue for edge classification. We defined machine observers sensitive to simple image properties and found that they too classified the edges better with color information, although they failed to capture the effect of image size observed in the psychophysical experiment. Our findings are consistent with previous work suggesting that color information facilitates the identification of material properties, transparency, shadows and the perception of shape-from-shading.

[1]  W. Geisler.  Ideal Observer Analysis , 2002 .

[2]  Frederick A A Kingdom,et al.  Colour contrast influences perceived shape in combined shading and texture patterns. , 2006, Spatial vision.

[3]  E. Land,et al.  Lightness and retinex theory. , 1971, Journal of the Optical Society of America.

[4]  Edward H. Adelson,et al.  Recovering intrinsic images from a single image , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[5]  Jun-Wei Hsieh,et al.  Shadow elimination for effective moving object detection by Gaussian shadow modeling , 2003, Image Vis. Comput..

[6]  Neil A. Macmillan,et al.  Detection Theory: A User's Guide , 1991 .

[7]  Javier Gonzalez,et al.  Shadow detection in colour high‐resolution satellite images , 2008 .

[8]  Christopher DiMattina,et al.  Detecting natural occlusion boundaries using local cues. , 2012, Journal of vision.

[9]  Frederick A A Kingdom,et al.  Color brings relief to human vision , 2003, Nature Neuroscience.

[10]  J. M. Rubin,et al.  Color vision and image intensities: When are changes material? , 1982, Biological Cybernetics.

[11]  T Troscianko,et al.  Stability of the color-opponent signals under changes of illuminant in natural scenes. , 2005, Journal of the Optical Society of America. A, Optics, image science, and vision.

[12]  Mark A. Georgeson,et al.  Local luminance amplitude modulates the interpretation of shape-from-shading in textured surfaces , 2006, Vision Research.

[13]  J. Pokorny,et al.  Spectral sensitivity of the foveal cone photopigments between 400 and 500 nm , 1975, Vision Research.

[14]  Pascal Mamassian,et al.  Interaction of visual prior constraints , 2001, Vision Research.

[15]  D. M. Green,et al.  Signal detection theory and psychophysics , 1966 .

[16]  A. Oliva,et al.  From Blobs to Boundary Edges: Evidence for Time- and Spatial-Scale-Dependent Scene Recognition , 1994 .

[17]  Andriana Olmos,et al.  A biologically inspired algorithm for the recovery of shading and reflectance images , 2004 .

[18]  R. Fisher.  THE USE OF MULTIPLE MEASUREMENTS IN TAXONOMIC PROBLEMS , 1936 .

[19]  M. Chun,et al.  Contextual cueing of visual attention , 2022 .

[20]  Antonio Torralba,et al.  Top-down control of visual attention in object detection , 2003, Proceedings 2003 International Conference on Image Processing (Cat. No.03CH37429).

[21]  Cheng Lu,et al.  On the removal of shadows from images , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[22]  Tom Troscianko,et al.  On the role of blue shadows in the visual behaviour of tsetse flies , 2004, Proceedings of the Royal Society of London. Series B: Biological Sciences.

[23]  Jake K. Aggarwal,et al.  Human Shadow Removal with Unknown Light Source , 2010, 2010 20th International Conference on Pattern Recognition.

[24]  tephen E. Palmer.  The effects of contextual scenes on the identification of objects , 1975, Memory & cognition.

[25]  Karl R Gegenfurtner,et al.  Geometry in Nature , 1993 .

[26]  P. Lennie,et al.  Chromatic mechanisms in lateral geniculate nucleus of macaque. , 1984, The Journal of physiology.

[27]  Frederick A A Kingdom,et al.  Colour Vision Brings Clarity to Shadows , 2004, Perception.

[28]  Brian C. Lovell,et al.  Shadow detection: A survey and comparative evaluation of recent methods , 2012, Pattern Recognit..

[29]  Michael I. Jordan,et al.  Advances in Neural Information Processing Systems 30 , 1995 .

[30]  Wilson S Geisler,et al.  Region grouping in natural foliage scenes: image statistics and human performance. , 2010, Journal of vision.

[31]  Karl R Gegenfurtner,et al.  Color contributes to object-contour perception in natural scenes. , 2017, Journal of vision.

[32]  W. Nauta,et al.  The organization of the brain. , 1979, Scientific American.

[33]  Frederick A.A. Kingdom,et al.  Perceiving light versus material , 2008, Vision Research.

[34]  Peng Sun,et al.  What is second-order vision for? Discriminating illumination versus material changes. , 2010, Journal of vision.

[35]  Liu Zhi Fang,et al.  A method to segment moving vehicle cast shadow based on wavelet transform , 2008, Pattern Recognit. Lett..

[36]  Alan J Pegna,et al.  An electrophysiological study of conscious visual perception using progressively degraded stimuli. , 2010, Journal of vision.

[37]  D. Tolhurst,et al.  Spatiochromatic Properties of Natural Images and Human Vision , 2002, Current Biology.

[38]  A. Gilchrist.  The perception of surface blacks and whites. , 1979, Scientific American.

[39]  Antonio Torralba,et al.  Modeling the Shape of the Scene: A Holistic Representation of the Spatial Envelope , 2001, International Journal of Computer Vision.

[40]  Antonio Torralba,et al.  Contextual guidance of eye movements and attention in real-world scenes: the role of global features in object search. , 2006, Psychological review.

[41]  H. Damasio,et al.  IEEE Transactions on Pattern Analysis and Machine Intelligence: Special Issue on Perceptual Organization in Computer Vision , 1998 .

[42]  Mark S. Drew,et al.  Removing Shadows from Images , 2002, ECCV.

[43]  A. Gilchrist,et al.  The classification and integration of edges as critical to the perception of reflectance and illumination , 1983, Perception & psychophysics.

[44]  M. Wilscy,et al.  Detection of moving cast shadows using edge information , 2010, 2010 The 2nd International Conference on Computer and Automation Engineering (ICCAE).

[45]  Sanjeev Kumar.  ALORITHM FOR SHADOW DETECTION IN REAL COLOUR IMAGES , 2010 .

[46]  Martin D. Levine,et al.  Removing shadows , 2005, Pattern Recognit. Lett..

[47]  Ione Fine,et al.  Surface segmentation based on the luminance and color statistics of natural scenes. , 2003, Journal of the Optical Society of America. A, Optics, image science, and vision.

[48]  Kedarnath P. Vilankar,et al.  Local edge statistics provide information regarding occlusion and nonocclusion edges in natural scenes. , 2014, Journal of vision.

[49]  Frederick A A Kingdom,et al.  Spatiochromatic statistics of natural scenes: first- and second-order information and their correlational structure. , 2005, Journal of the Optical Society of America. A, Optics, image science, and vision.

[50]  Touradj Ebrahimi,et al.  Cast shadow segmentation using invariant color features , 2004, Comput. Vis. Image Underst..

[51]  R. van Ee,et al.  United we sense, divided we fail: context-driven perception of ambiguous visual stimuli , 2012, Philosophical Transactions of the Royal Society B: Biological Sciences.

[52]  M. Bar.  Visual objects in context , 2004, Nature Reviews Neuroscience.