Color contributes to object-contour perception in natural scenes.

The magnitudes of chromatic and achromatic edge contrast are statistically independent and thus provide independent information, which can be used for object-contour perception. However, it is unclear if and how much object-contour perception benefits from chromatic edge contrast. To address this question, we investigated how well human-marked object contours can be predicted from achromatic and chromatic edge contrast. We used four data sets of human-marked object contours with a total of 824 images. We converted the images to the Derrington-Krauskopf-Lennie color space to separate chromatic from achromatic information in a physiologically meaningful way. Edges were detected in the three dimensions of the color space (one achromatic and two chromatic) and compared to human-marked object contours using receiver operating-characteristic (ROC) analysis for a threshold-independent evaluation. Performance was quantified by the difference of the area under the ROC curves (ΔAUC). Results were consistent across different data sets and edge-detection methods. If chromatic edges were used in addition to achromatic edges, predictions were better for 83% of the images, with a prediction advantage of 3.5% ΔAUC, averaged across all data sets and edge detectors. For some images the prediction advantage was considerably higher, up to 52% ΔAUC. Interestingly, if achromatic edges were used in addition to chromatic edges, the average prediction advantage was smaller (2.4% ΔAUC). We interpret our results such that chromatic information is important for object-contour perception.

[1]  S. Zeki The functional organization of projections from striate to prestriate visual cortex in the rhesus monkey. , 1976, Cold Spring Harbor symposia on quantitative biology.

[2]  K. Gegenfurtner,et al.  The contributions of color to recognition memory for natural scenes. , 2002, Journal of experimental psychology. Learning, memory, and cognition.

[3]  Paul V McGraw,et al.  Luminance cues constrain chromatic blur discrimination in natural scene stimuli. , 2013, Journal of vision.

[4]  D. Ruderman,et al.  Statistics of cone responses to natural images: implications for visual coding , 1998 .

[5]  Michael S. Lewicki,et al.  Is Early Vision Optimized for Extracting Higher-order Dependencies? , 2005, NIPS.

[6]  J. P. Jones,et al.  An evaluation of the two-dimensional Gabor filter model of simple receptive fields in cat striate cortex. , 1987, Journal of neurophysiology.

[7]  R. Shapley,et al.  Color in the Cortex: single- and double-opponent cells , 2011, Vision Research.

[8]  Refractor Vision , 2000, The Lancet.

[9]  Karl R Gegenfurtner,et al.  Higher order color mechanisms: evidence from noise-masking experiments in cone contrast space. , 2013, Journal of vision.

[10]  J. H. Hateren Spatial, temporal and spectral pre-processing for colour vision , 1993 .

[11]  James H. Elder,et al.  Design and perceptual validation of performance measures for salient object segmentation , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops.

[12]  Edward H. Adelson,et al.  Recovering intrinsic images from a single image , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[13]  Zhaoping Li,et al.  Understanding Retinal Color Coding from First Principles , 1992, Neural Computation.

[14]  J. Rieger,et al.  Sensory and cognitive contributions of color to the recognition of natural scenes , 2000, Current Biology.

[15]  D. Bamber The area above the ordinal dominance graph and the area below the receiver operating characteristic graph , 1975 .

[16]  Eero P. Simoncelli,et al.  Natural signal statistics and sensory gain control , 2001, Nature Neuroscience.

[17]  Tom Fawcett,et al.  An introduction to ROC analysis , 2006, Pattern Recognit. Lett..

[18]  D. Champernowne,et al.  On Economic Inequality. , 1974 .

[19]  D. H. Kelly Spatiotemporal variation of chromatic and achromatic contrast thresholds. , 1983, Journal of the Optical Society of America.

[20]  Frederick A A Kingdom,et al.  Spatiochromatic statistics of natural scenes: first- and second-order information and their correlational structure. , 2005, Journal of the Optical Society of America. A, Optics, image science, and vision.

[21]  Jean Ponce,et al.  Computer Vision: A Modern Approach , 2002 .

[22]  D. Hubel,et al.  Receptive fields and functional architecture of monkey striate cortex , 1968, The Journal of physiology.

[23]  A. Akbarinia Biologically-inspired Edge Detection Through Surround Modulation , 2016 .

[24]  K. Mullen The contrast sensitivity of human colour vision to red‐green and blue‐yellow chromatic gratings. , 1985, The Journal of physiology.

[25]  R. L. Valois Color Vision Mechanisms in the Monkey , 1960 .

[26]  J. Wolfe,et al.  What Can 1 Million Trials Tell Us About Visual Search? , 1998 .

[27]  Jitendra Malik,et al.  A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[28]  Andriana Olmos,et al.  A biologically inspired algorithm for the recovery of shading and reflectance images , 2004 .

[29]  Karl R Gegenfurtner,et al.  Geometry in Nature , 1993 .

[30]  D. Weiskopf,et al.  The role of color in high-level vision , 2001, Trends in Cognitive Sciences.

[31]  D. Tolhurst,et al.  Spatiochromatic Properties of Natural Images and Human Vision , 2002, Current Biology.

[32]  T. W. Lee,et al.  Chromatic structure of natural scenes. , 2001, Journal of the Optical Society of America. A, Optics, image science, and vision.

[33]  Karl R. Gegenfurtner,et al.  Geometric-optical illusions at isoluminance , 2007, Vision Research.

[34]  G. Buchsbaum,et al.  Trichromacy, opponent colours coding and optimum colour information transmission in the retina , 1983, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[35]  K. D. De Valois,et al.  Orientation and spatial-frequency discrimination for luminance and chromatic gratings. , 1990, Journal of the Optical Society of America. A, Optics and image science.

[36]  Colin W G Clifford,et al.  Interactions between color and luminance in the perception of orientation. , 2003, Journal of vision.

[37]  B. Pinna,et al.  Surface color from boundaries: a new ‘watercolor’ illusion , 2001, Vision Research.

[38]  A Bradley,et al.  Contrast dependence and mechanisms of masking interactions among chromatic and luminance gratings. , 1988, Journal of the Optical Society of America. A, Optics and image science.

[39]  K. Mullen,et al.  Color and luminance spatial tuning estimated by noise masking in the absence of off-frequency looking. , 1995, Journal of the Optical Society of America. A, Optics, image science, and vision.

[40]  D. Brainard,et al.  Aberration-free measurements of the visibility of isoluminant gratings. , 1993, Journal of the Optical Society of America. A, Optics, image science, and vision.

[41]  J. Mollon "Tho' she kneel'd in that place where they grew..." The uses and origins of primate colour vision. , 1989, The Journal of experimental biology.

[42]  J. Hanley,et al.  The meaning and use of the area under a receiver operating characteristic (ROC) curve. , 1982, Radiology.

[43]  I.E. Abdou,et al.  Quantitative design and evaluation of enhancement/thresholding edge detectors , 1979, Proceedings of the IEEE.

[44]  Josée Rivest,et al.  Localizing contours defined by more than one attribute , 1996, Vision Research.

[45]  J. M. Rubin,et al.  Color vision and image intensities: When are changes material? , 1982, Biological Cybernetics.

[46]  Paul V McGraw,et al.  Cue Combination of Conflicting Color and Luminance Edges , 2013, i-Perception.

[47]  Gunther Heidemann,et al.  The principal components of natural images revisited , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[48]  Heiko Neumann,et al.  A simple cell model with dominating opponent inhibition for robust image processing , 2004, Neural Networks.

[49]  Kathy T. Mullen,et al.  Contour integration with colour and luminance contrast , 1996, Vision Research.

[50]  Kathy T Mullen,et al.  Contour integration in color vision: a common process for the blue–yellow, red–green and luminance mechanisms? , 2000, Vision Research.

[51]  S. Shevell,et al.  Color in complex scenes. , 2008, Annual review of psychology.

[52]  Kathy T Mullen,et al.  Comparison of color and luminance vision on a global shape discrimination task , 2002, Vision Research.

[53]  Jitendra Malik,et al.  Learning to detect natural image boundaries using local brightness, color, and texture cues , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[54]  John F. Canny,et al.  A Computational Approach to Edge Detection , 1986, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[55]  Arthur Bradley,et al.  Orientation and spatial frequency selectivity of adaptation to color and luminance gratings , 1988, Vision Research.

[56]  A. Treisman,et al.  A feature-integration theory of attention , 1980, Cognitive Psychology.

[57]  Sean Dougherty,et al.  Edge Detector Evaluation Using Empirical ROC Curves , 2001, Comput. Vis. Image Underst..

[58]  Daniel A. Pollen,et al.  Visual cortical neurons as localized spatial frequency filters , 1983, IEEE Transactions on Systems, Man, and Cybernetics.

[59]  Heiko Neumann,et al.  Neural Mechanisms for the Robust Representation of Junctions , 2004, Neural Computation.

[60]  Karl R Gegenfurtner,et al.  Parallel visual search and rapid animal detection in natural scenes. , 2011, Journal of vision.

[61]  Christopher DiMattina,et al.  Detecting natural occlusion boundaries using local cues. , 2012, Journal of vision.

[62]  Arnold W. M. Smeulders,et al.  Color Invariance , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[63]  Frederick A A Kingdom,et al.  Color brings relief to human vision , 2003, Nature Neuroscience.

[64]  George Azzopardi,et al.  A CORF computational model of a simple cell that relies on LGN input outperforms the Gabor function model , 2012, Biological Cybernetics.

[65]  M. Webster,et al.  Adaptation and the color statistics of natural images , 1997, Vision Research.

[66]  Peter A. Flach,et al.  Repairing Concavities in ROC Curves , 2005, IJCAI.

[67]  D. M. Green,et al.  Signal detection theory and psychophysics , 1966 .

[68]  Bartlett W. Mel,et al.  Cue combination and color edge detection in natural scenes. , 2008, Journal of vision.

[69]  M. Pepe The Statistical Evaluation of Medical Tests for Classification and Prediction , 2003 .

[70]  Elena Gheorghiu,et al.  Chromatic tuning of contour-shape mechanisms revealed through the shape-frequency and shape-amplitude after-effects , 2007, Vision Research.