Distributed processing of color and form in the visual cortex

To what extent does the visual system process color and form separately? Proponents of the segregation view claim that distinct regions of the cortex are dedicated to each of these two dimensions separately. However, evidence is accumulating that color and form processing may, at least to some extent, be intertwined in the brain. In this perspective, we review psychophysical and neurophysiological studies on color and form perception and evaluate their results in light of recent developments in population coding.

[1]  Zhaoping Li.  A saliency map in primary visual cortex , 2002, Trends in Cognitive Sciences.

[2]  Margaret S. Livingstone,et al.  Two-Dimensional Substructure of Stereo and Motion Interactions in Macaque Visual Cortex , 2003, Neuron.

[3]  S. C Dakin,et al.  Summation of concentric orientation structure: seeing the Glass or the window? , 2002, Vision Research.

[4]  A. Leventhal,et al.  Concomitant sensitivity to orientation, direction, and color of cells in layers 2, 3, and 4 of monkey striate cortex , 1995, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[5]  H. Adesnik,et al.  A neural circuit for spatial summation in visual cortex , 2012, Nature.

[6]  A. Borst.  Seeing smells: imaging olfactory learning in bees , 1999, Nature Neuroscience.

[7]  M. Meister,et al.  Dynamic predictive coding by the retina , 2005, Nature.

[8]  L. Glass.  Moiré Effect from Random Dots , 1969, Nature.

[9]  Derek H. Arnold,et al.  A paradox of temporal perception revealed by a stimulus oscillating in colour and orientation , 2003, Vision Research.

[10]  D. Hubel,et al.  Complex–unoriented cells in a subregion of primate area 18 , 1985, Nature.

[11]  J. Nelson,et al.  Orientation-selective inhibition from beyond the classic visual receptive field , 1978, Brain Research.

[12]  Brad Wyble,et al.  Detecting meaning in RSVP at 13 ms per picture , 2013, Attention, perception & psychophysics.

[13]  Victor A. F. Lamme,et al.  Synchrony and covariation of firing rates in the primary visual cortex during contour grouping , 2004, Nature Neuroscience.

[14]  E. DeYoe,et al.  Segregation of efferent connections and receptive field properties in visual area V2 of the macaque , 1985, Nature.

[15]  H. Kennedy,et al.  A double-labeling investigation of the afferent connectivity to cortical areas V1 and V2 of the macaque monkey , 1985, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[16]  S. Dakin,et al.  Local and global visual grouping: tuning for spatial frequency and contrast. , 2001, Journal of vision.

[17]  Victor A. F. Lamme.  The neurophysiology of figure-ground segregation in primary visual cortex , 1995, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[18]  K. Mullen,et al.  Orientation selectivity in luminance and color vision assessed using 2-d band-pass filtered spatial noise , 2005, Vision Research.

[19]  S. Zeki,et al.  Colour coding in the superior temporal sulcus of rhesus monkey visual cortex , 1977, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[20]  K. Gegenfurtner,et al.  Cortical mechanisms of colour vision , 2003, Nature Reviews Neuroscience.

[21]  Christof Koch,et al.  Single-neuron correlates of subjective vision in the human medial temporal lobe , 2002, Proceedings of the National Academy of Sciences of the United States of America.

[22]  R. Shapley,et al.  Color in the Cortex: single- and double-opponent cells , 2011, Vision Research.

[23]  I. Ohzawa,et al.  Length and width tuning of neurons in the cat's primary visual cortex. , 1994, Journal of neurophysiology.

[24]  D. Alexander,et al.  Mapping of contextual modulation in the population response of primary visual cortex , 2010, Cognitive Neurodynamics.

[25]  W. Newsome,et al.  Context-dependent computation by recurrent dynamics in prefrontal cortex , 2013, Nature.

[26]  Semir Zeki,et al.  The theory of multistage integration in the visual brain , 1998, Proceedings of the Royal Society of London. Series B: Biological Sciences.

[27]  K. H. Britten,et al.  A relationship between behavioral choice and the visual responses of neurons in macaque MT , 1996, Visual Neuroscience.

[28]  G. Orban,et al.  How well do response changes of striate neurons signal differences in orientation: a study in the discriminating monkey , 1990, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[29]  C. Blakemore,et al.  Lateral inhibition between orientation detectors in the cat's visual cortex , 2004, Experimental Brain Research.

[30]  Pieter R Roelfsema,et al.  Separable Codes for Attention and Luminance Contrast in the Primary Visual Cortex , 2010, The Journal of Neuroscience.

[31]  Carrie J. McAdams,et al.  Effects of Attention on the Reliability of Individual Neurons in Monkey Visual Cortex , 1999, Neuron.

[32]  H. Komatsu,et al.  Disparity sensitivity of neurons in monkey extrastriate area MST , 1992, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[33]  Krista A. Ehinger,et al.  Rethinking the Role of Top-Down Attention in Vision: Effects Attributable to a Lossy Representation in Peripheral Vision , 2011, Front. Psychology.

[34]  A. Treisman,et al.  A feature-integration theory of attention , 1980, Cognitive Psychology.

[35]  DH Hubel,et al.  Segregation of form, color, and stereopsis in primate area 18 , 1987, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[36]  S. Zeki,et al.  The Organization of Connections between Areas V5 and V1 in Macaque Monkey Visual Cortex , 1989, The European journal of neuroscience.

[37]  Ruxandra Sireteanu,et al.  Perceptual learning in visual search: Fast, enduring, but non-specific , 1995, Vision Research.

[38]  Jim M. Monti,et al.  Neural repetition suppression reflects fulfilled perceptual expectations , 2008, Nature Neuroscience.

[39]  Christian K. Machens,et al.  Demixing Population Activity in Higher Cortical Areas , 2009, Front. Comput. Neurosci..

[40]  Anthony J. Movshon,et al.  Signals in Macaque Striate Cortical Neurons that Support the Perception of Glass Patterns , 2002, The Journal of Neuroscience.

[41]  R. Desimone,et al.  Selective attention gates visual processing in the extrastriate cortex. , 1985, Science.

[42]  V. Casagrande,et al.  Organization of cytochrome oxidase staining in the visual cortex of nocturnal primates (Galago crassicaudatus and Galago senegalensis): I. Adult Patterns , 1990, The Journal of comparative neurology.

[43]  T. Sejnowski,et al.  Representation of Color Stimuli in Awake Macaque Primary Visual Cortex , 2003, Neuron.

[44]  Terrence J. Sejnowski,et al.  Network model of shape-from-shading: neural function arises from both receptive and projective fields , 1988, Nature.

[45]  J. Movshon,et al.  The statistical reliability of signals in single neurons in cat and monkey visual cortex , 1983, Vision Research.

[46]  DH Hubel,et al.  Psychophysical evidence for separate channels for the perception of form, color, movement, and depth , 1987, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[47]  C. Koch,et al.  Invariant visual representation by single neurons in the human brain , 2005, Nature.

[48]  S. Zeki.  Functional organization of a visual area in the posterior bank of the superior temporal sulcus of the rhesus monkey , 1974, The Journal of physiology.

[49]  J. Wolfe,et al.  Guided Search 2.0 A revised model of visual search , 1994, Psychonomic bulletin & review.

[50]  D. Hubel,et al.  Thalamic inputs to cytochrome oxidase-rich regions in monkey visual cortex. , 1982, Proceedings of the National Academy of Sciences of the United States of America.

[51]  N. P. Bichot,et al.  A visual salience map in the primate frontal eye field. , 2005, Progress in brain research.

[52]  R. Tootell,et al.  Functional anatomy of the second visual area (V2) in the macaque , 1989, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[53]  H. Spitzer,et al.  Increased attention enhances both behavioral and neuronal performance. , 1988, Science.

[54]  D. Kiper,et al.  Chromatic properties of neurons in macaque area V2 , 1997, Visual Neuroscience.

[55]  R. Shapley,et al.  The Orientation Selectivity of Color-Responsive Neurons in Macaque V1 , 2008, The Journal of Neuroscience.

[56]  Ruth Rosenholtz,et al.  What your visual system sees where you are not looking , 2011, Electronic Imaging.

[57]  T. Albright,et al.  Representation of Color , 2009 .

[58]  G. Paramei,et al.  Processing bimodal stimuli: integrality/separability of color and orientation , 2013, Front. Psychol..

[59]  D. Hubel,et al.  Anatomy and physiology of a color system in the primate visual cortex , 1984, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[60]  Leslie G. Ungerleider,et al.  Object vision and spatial vision: two cortical pathways , 1983, Trends in Neurosciences.

[61]  W. Maass,et al.  State-dependent computations: spatiotemporal processing in cortical networks , 2009, Nature Reviews Neuroscience.

[62]  Janneke F. M. Jehee,et al.  Less Is More: Expectation Sharpens Representations in the Primary Visual Cortex , 2012, Neuron.

[63]  Xiao-Jing Wang,et al.  The importance of mixed selectivity in complex cognitive tasks , 2013, Nature.

[64]  S. Zeki,et al.  Responses of spectrally selective cells in macaque area V2 to wavelengths and colors. , 2002, Journal of neurophysiology.

[65]  J. B. Levitt,et al.  Contrast dependence of contextual effects in primate visual cortex , 1997, nature.

[66]  P. Lennie.  Single Units and Visual Cortical Organization , 1998, Perception.

[67]  D. Hubel,et al.  Receptive fields of single neurones in the cat's striate cortex , 1959, The Journal of physiology.

[68]  L. P. O'Keefe,et al.  Functional organization of owl monkey lateral geniculate nucleus and visual cortex. , 1998, Journal of neurophysiology.

[69]  Bernice E. Rogowitz,et al.  Front Matter: Volume 7865 , 2011, Human Vision and Electronic Imaging.

[70]  L M Optican,et al.  Primate striate and prestriate cortical neurons during discrimination. I. simultaneous temporal encoding of information about color and pattern. , 1996, Journal of neurophysiology.

[71]  S. Zeki,et al.  Colour coding in rhesus monkey prestriate cortex. , 1973, Brain research.

[72]  A. Clark.  Whatever next? Predictive brains, situated agents, and the future of cognitive science. , 2013, The Behavioral and brain sciences.

[73]  S Ullman,et al.  Shifts in selective visual attention: towards the underlying neural circuitry. , 1985, Human neurobiology.

[74]  P. Fldik,et al.  The Speed of Sight , 2001, Journal of Cognitive Neuroscience.

[75]  C. Gilbert,et al.  Spatial integration and cortical dynamics. , 1996, Proceedings of the National Academy of Sciences of the United States of America.

[76]  S. Zeki,et al.  The Organization of Connections between Areas V5 and V2 in Macaque Monkey Visual Cortex , 1989, The European journal of neuroscience.

[77]  F. D. de Lange,et al.  Prior Expectations Bias Sensory Representations in Visual Cortex , 2013, The Journal of Neuroscience.

[78]  John Duncan,et al.  Hierarchical coding for sequential task events in the monkey prefrontal cortex , 2008, Proceedings of the National Academy of Sciences.

[79]  H. Barlow,et al.  Single Units and Sensation: A Neuron Doctrine for Perceptual Psychology? , 1972, Perception.

[80]  S. Zeki,et al.  Segregation of pathways leading from area V2 to areas V4 and V5 of macaque monkey visual cortex , 1985, Nature.

[81]  K. Mullen,et al.  The spatial tuning of chromatic mechanisms identified by simultaneous masking , 1994, Vision Research.

[82]  J. B. Levitt,et al.  Intrinsic cortical connections in macaque visual area V2: Evidence for interaction between different functional streams , 1994, The Journal of comparative neurology.

[83]  Yoichi Sugita,et al.  Grouping of image fragments in primary visual cortex , 1999, Nature.

[84]  G. DeAngelis,et al.  Cortical area MT and the perception of stereoscopic depth , 1998, Nature.

[85]  R. Shapley,et al.  Cone inputs in macaque primary visual cortex. , 2004, Journal of Neurophysiology.

[86]  S. Zeki.  Functional specialisation in the visual cortex of the rhesus monkey , 1978, Nature.

[87]  P. Roelfsema,et al.  Incremental grouping of image elements in vision , 2011, Attention, perception & psychophysics.

[88]  D H Hubel,et al.  Connections between layer 4B of area 17 and the thick cytochrome oxidase stripes of area 18 in the squirrel monkey , 1987, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[89]  H. Wilson,et al.  Concentric orientation summation in human form vision , 1997, Vision Research.

[90]  R. Desimone,et al.  Neural mechanisms of spatial selective attention in areas V1, V2, and V4 of macaque visual cortex. , 1997, Journal of neurophysiology.

[91]  R. Shapley,et al.  The spatial transformation of color in the primary visual cortex of the macaque monkey , 2001, Nature Neuroscience.

[92]  Zhaoping Li,et al.  Neural Activities in V1 Create a Bottom-Up Saliency Map , 2012, Neuron.

[93]  M. Goldberg,et al.  The representation of visual salience in monkey parietal cortex , 1998, Nature.

[94]  H. Wilson,et al.  Detection of global structure in Glass patterns: implications for form vision , 1998, Vision Research.

[95]  Jim M. Monti,et al.  Expectation and Surprise Determine Neural Population Responses in the Ventral Visual Stream , 2010, The Journal of Neuroscience.

[96]  Alexander Grunewald,et al.  The Integration of Multiple Stimulus Features by V1 Neurons , 2004, The Journal of Neuroscience.

[97]  P. Lennie,et al.  Chromatic mechanisms in striate cortex of macaque , 1990, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[98]  Izumi Ohzawa,et al.  Joint-encoding of motion and depth by visual cortical neurons: neural basis of the Pulfrich effect , 2001, Nature Neuroscience.

[99]  E Switkes,et al.  Spatial frequency specific interaction of dot patterns and gratings. , 1980, Proceedings of the National Academy of Sciences of the United States of America.

[100]  D. Hubel,et al.  Receptive fields, binocular interaction and functional architecture in the cat's visual cortex , 1962, The Journal of physiology.

[101]  H B Barlow,et al.  Single units and sensation: a neuron doctrine for perceptual psychology? , 1972, Perception.

[102]  Ronald A. Rensink,et al.  Influence of scene-based properties on visual search. , 1990, Science.

[103]  Rafael Yuste,et al.  Dendritic Spines and Distributed Circuits , 2011, Neuron.

[104]  D. Ts'o,et al.  Functional organization of primate visual cortex revealed by high resolution optical imaging. , 1990, Science.

[105]  K R Gegenfurtner,et al.  Processing of color, form, and motion in macaque area V2 , 1996, Visual Neuroscience.

[106]  Wu Li,et al.  Adaptive shape processing in primary visual cortex , 2011, Proceedings of the National Academy of Sciences.

[107]  John Duncan,et al.  A neural basis for visual search in inferior temporal cortex , 1993, Nature.

[108]  Lawrence C. Sincich,et al.  Orientation Tuning of Cytochrome Oxidase Patches in Macaque Primary Visual Cortex , 2011, Nature Neuroscience.

[109]  Christof Koch,et al.  A Model of Saliency-Based Visual Attention for Rapid Scene Analysis , 2009 .

[110]  Ilias Rentzeperis,et al.  Evidence for color and luminance invariance of global form mechanisms. , 2010, Journal of vision.

[111]  S. Laughlin,et al.  Predictive coding: a fresh view of inhibition in the retina , 1982, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[112]  George R. Mangun,et al.  Anterior Intraparietal Sulcus is Sensitive to Bottom–Up Attention Driven by Stimulus Salience , 2009, Journal of Cognitive Neuroscience.

[113]  W Singer,et al.  Visual feature integration and the temporal correlation hypothesis. , 1995, Annual review of neuroscience.

[114]  Jon Driver,et al.  Visual search for a conjunction of movement and form is parallel , 1988, Nature.

[115]  D. Robinson,et al.  Shared neural control of attentional shifts and eye movements , 1996, Nature.

[116]  S. Kastner,et al.  Two hierarchically organized neural systems for object information in human visual cortex , 2008, Nature Neuroscience.

[117]  Christoph von der Malsburg,et al.  The Correlation Theory of Brain Function , 1994 .

[118]  Colin W G Clifford,et al.  Interactions between color and luminance in the perception of orientation. , 2003, Journal of vision.

[119]  Danny Keogan,et al.  Distributed hierarchical processing , 2002, Photomask Japan.

[120]  Patrick Cavanagh,et al.  Early binding of feature pairs for visual perception , 2001, Nature Neuroscience.

[121]  C. Gilbert,et al.  Top-down influences on visual processing , 2013, Nature Reviews Neuroscience.

[122]  K. D. De Valois,et al.  Orientation and spatial-frequency discrimination for luminance and chromatic gratings. , 1990, Journal of the Optical Society of America. A, Optics and image science.

[123]  Marie-Juliette F Mandelli,et al.  The local and global processing of chromatic Glass patterns. , 2005, Journal of vision.

[124]  Christian K. Machens,et al.  Behavioral / Systems / Cognitive Functional , But Not Anatomical , Separation of “ What ” and “ When ” in Prefrontal Cortex , 2009 .

[125]  K. Moutoussis,et al.  Functional segregation and temporal hierarchy of the visual perceptive systems , 1997, Proceedings of the Royal Society of London. Series B: Biological Sciences.

[126]  D H HUBEL,et al.  RECEPTIVE FIELDS AND FUNCTIONAL ARCHITECTURE IN TWO NONSTRIATE VISUAL AREAS (18 AND 19) OF THE CAT. , 1965, Journal of neurophysiology.

[127]  R. Malach,et al.  Relationship between orientation domains, cytochrome oxidase stripes, and intrinsic horizontal connections in squirrel monkey area V2. , 1994, Cerebral cortex.

[128]  Eugene Switkes,et al.  Integration of differing chromaticities in early and midlevel spatial vision. , 2005, Journal of the Optical Society of America. A, Optics, image science, and vision.

[129]  N. Sigala,et al.  Dynamic Coding for Cognitive Control in Prefrontal Cortex , 2013, Neuron.

[130]  Kathy T. Mullen,et al.  Contour integration with colour and luminance contrast , 1996, Vision Research.

[131]  A. P. Georgopoulos,et al.  Neuronal population coding of movement direction. , 1986, Science.

[132]  Daniel C. Kiper,et al.  Relationship between neural response and adaptation selectivity to form and color: an ERP study , 2012, Front. Hum. Neurosci..

[133]  Sam T. Pickering.  A DIRECT DEMONSTRATION , 1909 .

[134]  D. Hubel,et al.  Regular patchy distribution of cytochrome oxidase staining in primary visual cortex of macaque monkey , 1981, Nature.

[135]  Pieter R. Roelfsema,et al.  Object-based attention in the primary visual cortex of the macaque monkey , 1998, Nature.

[136]  J. Movshon,et al.  A computational analysis of the relationship between neuronal and behavioral responses to visual motion , 1996, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[137]  S Zeki,et al.  The autonomy of the visual systems and the modularity of conscious vision. , 1998, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[138]  Georg B. Keller,et al.  Sensorimotor Mismatch Signals in Primary Visual Cortex of the Behaving Mouse , 2012, Neuron.

[139]  Denis Fize,et al.  Speed of processing in the human visual system , 1996, Nature.

[140]  K. Mullen.  The contrast sensitivity of human colour vision to red‐green and blue‐yellow chromatic gratings. , 1985, The Journal of physiology.

[141]  S. Zeki,et al.  Response properties and receptive fields of cells in an anatomically defined region of the superior temporal sulcus in the monkey. , 1971, Brain research.

[142]  S. Zeki,et al.  A direct demonstration of perceptual asynchrony in vision , 1997, Proceedings of the Royal Society of London. Series B: Biological Sciences.

[143]  James J. DiCarlo,et al.  How Does the Brain Solve Visual Object Recognition? , 2012, Neuron.

[144]  T. Poggio,et al.  Predicting the visual world: silence is golden , 1999, Nature Neuroscience.

[145]  J. Gibson,et al.  Adaptation, after-effect and contrast in the perception of tilted lines. I. Quantitative studies , 1937 .

[146]  Z Li,et al.  Contextual influences in V1 as a basis for pop out and asymmetry in visual search. , 1999, Proceedings of the National Academy of Sciences of the United States of America.

[147]  P. Roelfsema.  Cortical algorithms for perceptual grouping. , 2006, Annual review of neuroscience.

[148]  D. J. Felleman,et al.  Distributed hierarchical processing in the primate cerebral cortex. , 1991, Cerebral cortex.

[149]  E. Peterhans,et al.  Functional Organization of Area V2 in the Alert Macaque , 1993, The European journal of neuroscience.

[150]  Arthur Bradley,et al.  Orientation and spatial frequency selectivity of adaptation to color and luminance gratings , 1988, Vision Research.

[151]  S. Yantis,et al.  Spatially selective representations of voluntary and stimulus-driven attentional priority in human occipital, parietal, and frontal cortex. , 2007, Cerebral cortex.

[152]  Patrick Cavanagh,et al.  Independent orientation-selective mechanisms for the cardinal directions of colour space , 1990, Vision Research.

[153]  D. V. van Essen,et al.  Neuronal responses to static texture patterns in area V1 of the alert macaque monkey. , 1992, Journal of neurophysiology.

[154]  S C Dakin,et al.  Glass Patterns: Some Contrast Effects Re-Evaluated , 1997, Perception.

[155]  Tobias E. Reisbeck,et al.  PII: S0042-6989(97)00240-X , 2003 .

[156]  D. Hubel,et al.  Segregation of form, color, movement, and depth: anatomy, physiology, and perception. , 1988, Science.

[157]  Karl J. Friston.  Learning and inference in the brain , 2003, Neural Networks.

[158]  P. Roelfsema,et al.  Automatic spread of attentional response modulation along Gestalt criteria in primary visual cortex , 2011, Nature Neuroscience.

[159]  M. Silverman,et al.  Functional organization of the second cortical visual area in primates. , 1983, Science.

[160]  J. Movshon,et al.  Glass pattern responses in macaque V2 neurons. , 2007, Journal of vision.

[161]  A. Angelucci,et al.  Contribution of feedforward, lateral and feedback connections to the classical receptive field center and extra-classical receptive field surround of primate V1 neurons. , 2006, Progress in brain research.

[162]  D. Purves,et al.  Iterated patterns of brain circuitry (or how the cortex gets its spots) , 1992, Trends in Neurosciences.

[163]  L. Glass,et al.  Pattern Recognition in Humans: Correlations Which Cannot be Perceived , 1976, Perception.

[164]  D. Ts'o,et al.  Visual topography in primate V2: multiple representation across functional stripes , 1995, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[165]  Stefano Fusi,et al.  The Sparseness of Mixed Selectivity Neurons Controls the Generalization–Discrimination Trade-Off , 2013, The Journal of Neuroscience.

[166]  Hong Zhou,et al.  The coding of uniform colour figures in monkey visual cortex , 2003, The Journal of physiology.

[167]  A. Leventhal,et al.  Signal timing across the macaque visual system. , 1998, Journal of neurophysiology.

[168]  B Julesz,et al.  Depth, motion, and static-flow perception at metaisoluminant color contrast. , 1992, Proceedings of the National Academy of Sciences of the United States of America.

[169]  L M Optican,et al.  Primate striate and prestriate cortical neurons during discrimination. II. separable temporal codes for color and pattern. , 1996, Journal of neurophysiology.

[170]  J. Gallant,et al.  Goal-Related Activity in V4 during Free Viewing Visual Search Evidence for a Ventral Stream Visual Salience Map , 2003, Neuron.

[171]  J. Mollon.  Color vision. , 1982, Annual review of psychology.