Multiplicative modulations enhance diversity of hue-selective cells

There is still much to understand about the brain’s colour processing mechanisms and the transformation from cone-opponent representations to perceptual hues. Moreover, it is unclear which area(s) in the brain represent unique hues. We propose a hierarchical model inspired by the neuronal mechanisms in the brain for local hue representation, which reveals the contributions of each visual cortical area in hue representation. Hue encoding is achieved through incrementally increasing processing nonlinearities beginning with cone input. Besides employing nonlinear rectifications, we propose multiplicative modulations as a form of nonlinearity. Our simulation results indicate that multiplicative modulations have significant contributions in encoding of hues along intermediate directions in the MacLeod-Boynton diagram and that our model V2 neurons have the capacity to encode unique hues. Additionally, responses of our model neurons resemble those of biological colour cells, suggesting that our model provides a novel formulation of the brain’s colour processing pathway.

[1]  R. Shapley,et al.  Cone inputs in macaque primary visual cortex. , 2004, Journal of Neurophysiology.

[2]  S. Wuerger,et al.  The cone inputs to the unique-hue mechanisms , 2005, Vision Research.

[3]  Bevil R. Conway,et al.  Spatial Structure of Cone Inputs to Color Cells in Alert Macaque Primary Visual Cortex (V-1) , 2001, The Journal of Neuroscience.

[4]  Gouki Okazawa,et al.  Effects of Luminance Contrast on the Color Selectivity of Neurons in the Macaque Area V4 and Inferior Temporal Cortex , 2014, The Journal of Neuroscience.

[5]  P. Lennie,et al.  Chromatic mechanisms in lateral geniculate nucleus of macaque. , 1984, The Journal of physiology.

[6]  T. Sejnowski,et al.  Representation of Color Stimuli in Awake Macaque Primary Visual Cortex , 2003, Neuron.

[7]  G. Edelman,et al.  A model of color vision based on cortical reentry. , 1996, Cerebral cortex.

[8]  S. Zeki The representation of colours in the cerebral cortex , 1980, Nature.

[9]  Bolei Zhou,et al.  Network Dissection: Quantifying Interpretability of Deep Visual Representations , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[10]  David A. Forsyth,et al.  Shape, Contour and Grouping in Computer Vision , 1999, Lecture Notes in Computer Science.

[11]  B. Batlogg,et al.  Auditory Spatial Receptive Fields Created by Multiplication , 2022 .

[12]  R. Shapley,et al.  Space and Time Maps of Cone Photoreceptor Signals in Macaque Lateral Geniculate Nucleus , 2002, The Journal of Neuroscience.

[13]  M. Webster,et al.  Variations in normal color vision. II. Unique hues. , 2000, Journal of the Optical Society of America. A, Optics, image science, and vision.

[14]  Stephen P. Boyd,et al.  An Interior-Point Method for Large-Scale l1-Regularized Logistic Regression , 2007, J. Mach. Learn. Res..

[15]  S Yamane,et al.  Color selectivity of neurons in the inferior temporal cortex of the awake macaque monkey , 1992, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[16]  Qasim Zaidi,et al.  Geometrical structure of perceptual color space: Mental representations and adaptation invariance , 2018, bioRxiv.

[17]  D. Kiper,et al.  Chromatic properties of neurons in macaque area V2 , 1997, Visual Neuroscience.

[18]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[19]  R. Desimone,et al.  Spectral properties of V4 neurons in the macaque , 1990, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[20]  H. Barlow Why have multiple cortical areas? , 1986, Vision Research.

[21]  Terrence J. Sejnowski,et al.  Seeing White: Qualia in the Context of Decoding Population Codes , 1999, Neural Computation.

[22]  R. Shapley,et al.  Color in the Cortex: single- and double-opponent cells , 2011, Vision Research.

[23]  Li Lin,et al.  Corrigendum: Selectively enhanced photocurrent generation in twisted bilayer graphene with van Hove singularity , 2016, Nature Communications.

[24]  Lauren E. Wool,et al.  Connectomic identification and three-dimensional color tuning of S-OFF midget ganglion cells in the primate retina , 2018 .

[25]  A. Linksz Outlines of a Theory of the Light Sense. , 1965 .

[26]  E. Miyahara Focal Colors and Unique Hues , 2003, Perceptual and motor skills.

[27]  Bevil R. Conway,et al.  Representation of Perceptual Color Space in Macaque Posterior Inferior Temporal Cortex (the V4 Complex) , 2016, eNeuro.

[28]  John K. Tsotsos,et al.  TarzaNN: A General Purpose Neural Network Simulator for Visual Attention Modeling , 2004, WAPCV.

[29]  P. Lennie,et al.  Chromatic mechanisms in striate cortex of macaque , 1990, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[30]  Leif H. Finkel,et al.  A multistage neural network for color constancy and color induction , 1995, IEEE Trans. Neural Networks.

[31]  Kenichi Ueno,et al.  Hue Selectivity in Human Visual Cortex Revealed by Functional Magnetic Resonance Imaging , 2015, Cerebral cortex.

[32]  Bevil R. Conway,et al.  Evolution of neural computations: Mantis shrimp and human color decoding , 2014, i-Perception.

[33]  R. L. Valois,et al.  Some transformations of color information from lateral geniculate nucleus to striate cortex. , 2000, Proceedings of the National Academy of Sciences of the United States of America.

[34]  J. D. Paula Converting RGB Images to LMS Cone Activations , 2006 .

[35]  H. Komatsu,et al.  Neural selectivity for hue and saturation of colour in the primary visual cortex of the monkey , 2000, The European journal of neuroscience.

[36]  Barry B. Lee,et al.  Specificity of cone inputs to macaque retinal ganglion cells. , 2006, Journal of neurophysiology.

[37]  Bevil R. Conway,et al.  Specialized Color Modules in Macaque Extrastriate Cortex , 2007, Neuron.

[38]  K. D. De Valois,et al.  A multi-stage color model. , 1993, Vision research.

[39]  Bevil R. Conway,et al.  Color-tuned neurons are spatially clustered according to color preference within alert macaque posterior inferior temporal cortex , 2009, Proceedings of the National Academy of Sciences.

[40]  Joshua W. Brown,et al.  The tale of the neuroscientists and the computer: why mechanistic theory matters , 2014, Front. Neurosci..

[41]  Multiplying two numbers together in your head is a difficult task if you did not learn multiplication tables as a child. On the face of it, this is somewhat surprising given the remarkable power of the brain to perform , 2010 .

[42]  J. Alonso,et al.  Functional implications of orientation maps in primary visual cortex , 2016, Nature Communications.

[43]  J. Mollon,et al.  A neural basis for unique hues? , 2009, Current Biology.

[44]  Eero P. Simoncelli,et al.  Metamers of the ventral stream , 2011, Nature Neuroscience.

[45]  Lauren E. Wool,et al.  Nonselective Wiring Accounts for Red-Green Opponency in Midget Ganglion Cells of the Primate Retina , 2018, The Journal of Neuroscience.

[46]  R. M. Boynton,et al.  Chromaticity diagram showing cone excitation by stimuli of equal luminance. , 1979, Journal of the Optical Society of America.

[47]  C. Koch,et al.  Multiplicative computation in a visual neuron sensitive to looming , 2002, Nature.

[48]  Arvind Satyanarayan,et al.  The Building Blocks of Interpretability , 2018 .

[49]  Fang Liu,et al.  Perceptual Color Map in Macaque Visual Area V4 , 2014, The Journal of Neuroscience.

[50]  P. A. Dufort,et al.  Color categorization and color constancy in a neural network model of V4 , 1991, Biological Cybernetics.

[51]  D. J. Felleman,et al.  A spatially organized representation of colour in macaque cortical area V2 , 2003, Nature.

[52]  Yoshua Bengio,et al.  Object Recognition with Gradient-Based Learning , 1999, Shape, Contour and Grouping in Computer Vision.

[53]  Michael S. Bernstein,et al.  ImageNet Large Scale Visual Recognition Challenge , 2014, International Journal of Computer Vision.

[54]  Robert Shapley,et al.  Neural mechanisms for color perception in the primary visual cortex , 2002, Current Opinion in Neurobiology.

[55]  Bevil R. Conway,et al.  Steps towards neural decoding of colors , 2019, Current Opinion in Behavioral Sciences.

[56]  Qasim Zaidi,et al.  Salience of unique hues and implications for color theory. , 2015, Journal of vision.