Neural dynamics at successive stages of the ventral visual stream are consistent with hierarchical error signals

Ventral visual stream neural responses are dynamic, even for static image presentations. However, dynamical neural models of visual cortex are lacking as most progress has been made modeling static, time-averaged responses. Here, we studied population neural dynamics during face detection across three cortical processing stages. Remarkably, ~30 milliseconds after the initially evoked response, we found that neurons in intermediate level areas decreased their preference for faces, becoming anti-face preferring on average even while neurons in higher level areas achieved and maintained a face preference. This pattern of hierarchical neural dynamics was inconsistent with extensions of standard feedforward circuits that implemented recurrence within a cortical stage. Rather, recurrent models computing errors between stages captured the observed temporal signatures. Without additional parameter fitting, this model of neural dynamics, which simply augments the standard feedforward model of online vision to encode errors, also explained seemingly disparate dynamical phenomena in the ventral stream.

[1]  Doris Y. Tsao,et al.  Functional Compartmentalization and Viewpoint Generalization Within the Macaque Face-Processing System , 2010, Science.

[2]  Kenji Kawano,et al.  Global and fine information coded by single neurons in the temporal visual cortex , 1999, Nature.

[3]  O. Schwartz,et al.  Flexible Gating of Contextual Influences in Natural Vision , 2015, Nature Neuroscience.

[4]  Geoffrey E. Hinton,et al.  A Learning Algorithm for Boltzmann Machines , 1985, Cogn. Sci..

[5]  Doris Y. Tsao,et al.  A Cortical Region Consisting Entirely of Face-Selective Cells , 2006, Science.

[6]  Tai Sing Lee,et al.  Hierarchical Bayesian inference in the visual cortex. , 2003, Journal of the Optical Society of America. A, Optics, image science, and vision.

[7]  Karl J. Friston,et al.  Cortical circuits for perceptual inference , 2009, Neural Networks.

[8]  Caspar M. Schwiedrzik,et al.  High-Level Prediction Signals in a Low-Level Area of the Macaque Face-Processing Hierarchy , 2017, Neuron.

[9]  Ha Hong,et al.  Performance-optimized hierarchical models predict neural responses in higher visual cortex , 2014, Proceedings of the National Academy of Sciences.

[10]  Doris Y. Tsao,et al.  The Code for Facial Identity in the Primate Brain , 2017, Cell.

[11]  Doris Y. Tsao,et al.  What Makes a Cell Face Selective? The Importance of Contrast , 2012, Neuron.

[12]  Doris Y. Tsao,et al.  The effect of face patch microstimulation on perception of faces and objects , 2017, Nature Neuroscience.

[13]  Pascal Vincent,et al.  Contractive Auto-Encoders: Explicit Invariance During Feature Extraction , 2011, ICML.

[14]  David D. Cox,et al.  High-resolution three-dimensional microelectrode brain mapping using stereo microfocal X-ray imaging. , 2008, Journal of neurophysiology.

[15]  M. Carandini,et al.  Normalization as a canonical neural computation , 2011, Nature Reviews Neuroscience.

[16]  D. Mumford,et al.  Neural activity in early visual cortex reflects behavioral experience and higher-order perceptual saliency , 2002, Nature Neuroscience.

[17]  Elias B. Issa,et al.  Large-Scale, High-Resolution Neurophysiological Maps Underlying fMRI of Macaque Temporal Lobe , 2013, The Journal of Neuroscience.

[18]  Elias B. Issa,et al.  Neurophysiological Organization of the Middle Face Patch in Macaque Inferior Temporal Cortex , 2016, The Journal of Neuroscience.

[19]  R. von der Heydt,et al.  Analysis of the Context Integration Mechanisms Underlying Figure–Ground Organization in the Visual Cortex , 2010, The Journal of Neuroscience.

[20]  Hualou Liang,et al.  Incremental Integration of Global Contours through Interplay between Visual Cortical Areas , 2014, Neuron.

[21]  Rajesh P. N. Rao,et al.  Predictive coding in the visual cortex: a functional interpretation of some extra-classical receptive-field effects. , 1999 .

[22]  R. Kiani,et al.  Microstimulation of inferotemporal cortex influences face categorization , 2006, Nature.

[23]  Doris Y. Tsao,et al.  A face feature space in the macaque temporal lobe , 2009, Nature Neuroscience.

[24]  Eero P. Simoncelli,et al.  Natural signal statistics and sensory gain control , 2001, Nature Neuroscience.

[25]  E. Miller,et al.  Experience-dependent sharpening of visual shape selectivity in inferior temporal cortex. , 2005, Cerebral cortex.

[26]  Geoffrey E. Hinton,et al.  An Efficient Learning Procedure for Deep Boltzmann Machines , 2012, Neural Computation.

[27]  J. Movshon,et al.  Linearity and Normalization in Simple Cells of the Macaque Primary Visual Cortex , 1997, The Journal of Neuroscience.

[28]  Geoffrey E. Hinton,et al.  Learning representations by back-propagating errors , 1986, Nature.

[29]  Daniel L. K. Yamins,et al.  Deep Neural Networks Rival the Representation of Primate IT Cortex for Core Visual Object Recognition , 2014, PLoS Comput. Biol..

[30]  David J. Field,et al.  Emergence of simple-cell receptive field properties by learning a sparse code for natural images , 1996, Nature.

[31]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[32]  Scott L. Brincat,et al.  Dynamic Shape Synthesis in Posterior Inferotemporal Cortex , 2006, Neuron.

[33]  Doris Y. Tsao,et al.  Comparing face patch systems in macaques and humans , 2008, Proceedings of the National Academy of Sciences.

[34]  J. Nassi,et al.  Corticocortical feedback increases the spatial extent of normalization , 2014, Front. Syst. Neurosci..

[35]  Rob Fergus,et al.  Stochastic Pooling for Regularization of Deep Convolutional Neural Networks , 2013, ICLR.

[36]  Doris Y. Tsao,et al.  Anatomical Connections of the Functionally Defined “Face Patches” in the Macaque Monkey , 2016, Neuron.

[37]  S. Yantis,et al.  Visual attention: control, representation, and time course. , 1997, Annual review of psychology.

[38]  M. Giese,et al.  Norm-based face encoding by single neurons in the monkey inferotemporal cortex , 2006, Nature.

[39]  Wilbert Zarco,et al.  A causal relationship between face-patch activity and face-detection behavior , 2017, eLife.

[40]  Doris Y. Tsao,et al.  Patches with Links: A Unified System for Processing Faces in the Macaque Temporal Lobe , 2008, Science.

[41]  Shimon Ullman,et al.  Image interpretation by a single bottom-up top-down cycle , 2008, Proceedings of the National Academy of Sciences.

[42]  Konrad P. Körding,et al.  Toward an Integration of Deep Learning and Neuroscience , 2016, bioRxiv.

[43]  Doris Y. Tsao,et al.  Intelligent Information Loss: The Coding of Facial Identity, Head Pose, and Non-Face Information in the Macaque Face Patch System , 2015, The Journal of Neuroscience.

[44]  Elias B. Issa,et al.  Precedence of the Eye Region in Neural Processing of Faces , 2012, The Journal of Neuroscience.

[45]  Ha Hong,et al.  Simple Learned Weighted Sums of Inferior Temporal Neuronal Firing Rates Accurately Predict Human Core Object Recognition Performance , 2015, The Journal of Neuroscience.

[46]  Tomaso Poggio,et al.  Fast Readout of Object Identity from Macaque Inferior Temporal Cortex , 2005, Science.

[47]  C. Olson,et al.  Statistical learning of visual transitions in monkey inferotemporal cortex , 2011, Proceedings of the National Academy of Sciences.

[48]  C. Olson,et al.  Image familiarization sharpens response dynamics of neurons in inferotemporal cortex , 2014, Nature Neuroscience.

[49]  David L. Sheinberg,et al.  Effects of Long-Term Visual Experience on Responses of Distinct Classes of Single Units in Inferior Temporal Cortex , 2012, Neuron.

[50]  L. Saksida,et al.  Visual perception and memory: a new view of medial temporal lobe function in primates and rodents. , 2007, Annual review of neuroscience.