Topographic deep artificial neural networks reproduce the hallmarks of the primate inferior temporal cortex face processing network

A salient characteristic of monkey inferior temporal (IT) cortex is the IT face processing network. Its hallmarks include: “face neurons” that respond more to faces than non-face objects, strong spatial clustering of those neurons in foci at each IT anatomical level (“face patches”), and the preferential interconnection of those foci. While some deep artificial neural networks (ANNs) are good predictors of IT neuronal responses, including face neurons, they do not explain those face network hallmarks. Here we ask if they might be explained with a simple, metabolically motivated addition to current ANN ventral stream models. Specifically, we designed and successfully trained topographic deep ANNs (TDANNs) to solve real-world visual recognition tasks (as in prior work), but, in addition, we also optimized each network to minimize a proxy for neuronal wiring length within its IT layers. We report that after this dual optimization, the model IT layers of TDANNs reproduce the hallmarks of the IT face network: the presence of face neurons, clusters of face neurons that quantitatively match those found in IT face patches, connectivity between those patches, and the emergence of face viewpoint invariance along the network hierarchy. We find that these phenomena emerge for a range of naturalistic experience, but not for highly unnatural training. Taken together, these results show that the IT face processing network could be a consequence of a basic hierarchical anatomy along the ventral stream, selection pressure on the visual system to accomplish general object categorization, and selection pressure to minimize axonal wiring length.

[1]  Aran Nayebi,et al.  Brain-Like Object Recognition with High-Performing Shallow Recurrent ANNs , 2019, NeurIPS.

[2]  Kendrick Kay,et al.  The Functional Neuroanatomy of Human Face Perception. , 2017, Annual review of vision science.

[3]  Keiji Tanaka,et al.  Neuronal selectivities to complex object features in the ventral visual pathway of the macaque cerebral cortex. , 1994, Journal of neurophysiology.

[4]  Christiane Fellbaum,et al.  Book Reviews: WordNet: An Electronic Lexical Database , 1999, CL.

[5]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[6]  Keiji Tanaka,et al.  Object category structure in response patterns of neuronal population in monkey inferior temporal cortex. , 2007, Journal of neurophysiology.

[7]  Richard Durbin,et al.  A dimension reduction framework for understanding cortical maps , 1990, Nature.

[8]  Nikolaus Kriegeskorte,et al.  Deep neural networks: a new framework for modelling biological vision and brain information processing , 2015, bioRxiv.

[9]  Michael I. Jordan,et al.  Computational Consequences of a Bias toward Short Connections , 1992, Journal of Cognitive Neuroscience.

[10]  N. Swindale,et al.  Application of Kohonen's self–organizing feature map algorithm to cortical maps of orientation and direction preference , 1998, Proceedings of the Royal Society of London. Series B: Biological Sciences.

[11]  Y. Sugita Face perception in monkeys reared with no exposure to faces , 2008, Proceedings of the National Academy of Sciences.

[12]  Mason McGill,et al.  A map of object space in primate inferotemporal cortex , 2020, Nature.

[13]  Elias B. Issa,et al.  Large-Scale, High-Resolution Neurophysiological Maps Underlying fMRI of Macaque Temporal Lobe , 2013, The Journal of Neuroscience.

[14]  J. Haxby,et al.  Attribute-based neural substrates in temporal cortex for perceiving and knowing about objects , 1999, Nature Neuroscience.

[15]  Zoe Kourtzi,et al.  Development of visually evoked cortical activity in infant macaque monkeys studied longitudinally with fMRI. , 2006, Magnetic resonance imaging.

[16]  Leon A. Gatys,et al.  Deep convolutional models improve predictions of macaque V1 responses to natural images , 2019, PLoS Comput. Biol..

[17]  Michael S. Bernstein,et al.  ImageNet Large Scale Visual Recognition Challenge , 2014, International Journal of Computer Vision.

[18]  Baoyuan Wu,et al.  Tencent ML-Images: A Large-Scale Multi-Label Image Database for Visual Representation Learning , 2019, IEEE Access.

[19]  Talma Hendler,et al.  Eccentricity Bias as an Organizing Principle for Human High-Order Object Areas , 2002, Neuron.

[20]  M. Cynader,et al.  Surface organization of orientation and direction selectivity in cat area 18 , 1987, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[21]  W. Freiwald,et al.  Face Processing Systems: From Neurons to Real-World Social Perception. , 2016, Annual review of neuroscience.

[22]  Doris Y. Tsao,et al.  Faces and objects in macaque cerebral cortex , 2003, Nature Neuroscience.

[23]  Dmitri B. Chklovskii,et al.  Orientation Preference Patterns in Mammalian Visual Cortex A Wire Length Minimization Approach , 2001, Neuron.

[24]  Leila Reddy,et al.  Coding of visual objects in the ventral stream , 2006, Current Opinion in Neurobiology.

[25]  C. Foth,et al.  Birds have peramorphic skulls, too: anatomical network analyses reveal oppositional heterochronies in avian skull evolution , 2020, Communications Biology.

[26]  Surya Ganguli,et al.  Task-Driven Convolutional Recurrent Models of the Visual System , 2018, NeurIPS.

[27]  R. Vogels Categorization of complex visual images by rhesus monkeys. Part 2: single‐cell study , 1999, The European journal of neuroscience.

[28]  Bevil R. Conway,et al.  Parallel, multi-stage processing of colors, faces and shapes in macaque inferior temporal cortex , 2013, Nature Neuroscience.

[29]  Aran Nayebi,et al.  CORnet: Modeling the Neural Mechanisms of Core Object Recognition , 2018, bioRxiv.

[30]  N. Kanwisher,et al.  The Fusiform Face Area: A Module in Human Extrastriate Cortex Specialized for Face Perception , 1997, The Journal of Neuroscience.

[31]  Doris Y. Tsao,et al.  Functional Compartmentalization and Viewpoint Generalization Within the Macaque Face-Processing System , 2010, Science.

[32]  Ha Hong,et al.  Hierarchical Modular Optimization of Convolutional Networks Achieves Representations Similar to Macaque IT and Human Ventral Stream , 2013, NIPS.

[33]  K. Grill-Spector,et al.  The functional architecture of the ventral temporal cortex and its role in categorization , 2014, Nature Reviews Neuroscience.

[34]  A. Ishai,et al.  Distributed and Overlapping Representations of Faces and Objects in Ventral Temporal Cortex , 2001, Science.

[35]  James J. DiCarlo,et al.  How Does the Brain Solve Visual Object Recognition? , 2012, Neuron.

[36]  R. Desimone,et al.  Stimulus-selective properties of inferior temporal neurons in the macaque , 1984, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[37]  J. Haxby,et al.  The distributed human neural system for face perception , 2000, Trends in Cognitive Sciences.

[38]  Nancy Kanwisher,et al.  A cortical representation of the local visual environment , 1998, Nature.

[39]  H. O. D. Beeck,et al.  Development of visual category selectivity in ventral visual cortex does not require visual experience , 2017 .

[40]  Marc A Sommer,et al.  Spatial and Temporal Scales of Neuronal Correlation in Visual Area V4 , 2013, The Journal of Neuroscience.

[41]  J. DiCarlo,et al.  Using goal-driven deep learning models to understand sensory cortex , 2016, Nature Neuroscience.

[42]  Ha Hong,et al.  Simple Learned Weighted Sums of Inferior Temporal Neuronal Firing Rates Accurately Predict Human Core Object Recognition Performance , 2015, The Journal of Neuroscience.

[43]  Kalanit Grill-Spector,et al.  Temporal Processing Capacity in High-Level Visual Cortex Is Domain Specific , 2015, The Journal of Neuroscience.

[44]  Marwan Mattar,et al.  Labeled Faces in the Wild: A Database forStudying Face Recognition in Unconstrained Environments , 2008 .

[45]  N. Kanwisher,et al.  The Human Body , 2001 .

[46]  G. Orban,et al.  Search for color 'center(s)' in macaque visual cortex. , 2004, Cerebral cortex.

[47]  Xueqi Cheng,et al.  A Network for Scene Processing in the Macaque Temporal Lobe , 2013, Neuron.

[48]  Nancy Kanwisher,et al.  Visual experience is not necessary for the development of face-selectivity in the lateral fusiform gyrus , 2020, Proceedings of the National Academy of Sciences.

[49]  Ha Hong,et al.  Performance-optimized hierarchical models predict neural responses in higher visual cortex , 2014, Proceedings of the National Academy of Sciences.

[50]  L. Merabet,et al.  Development of the Visual Word Form Area Requires Visual Experience: Evidence from Blind Braille Readers , 2017, The Journal of Neuroscience.

[51]  Doris Y. Tsao,et al.  The Code for Facial Identity in the Primate Brain , 2017, Cell.

[52]  Hans P. Op de Beeck,et al.  Development of visual category selectivity in ventral visual cortex does not require visual experience , 2017, Proceedings of the National Academy of Sciences.

[53]  D. J. Felleman,et al.  Distributed hierarchical processing in the primate cerebral cortex. , 1991, Cerebral cortex.

[54]  K. Grill-Spector,et al.  Neural representations of faces and limbs neighbor in human high-level visual cortex: evidence for a new organization principle , 2011, Psychological Research.

[55]  Y. Kamatani,et al.  12 new susceptibility loci for prostate cancer identified by genome-wide association study in Japanese population , 2019, Nature Communications.

[56]  Rajani Raman,et al.  Convolutional neural networks explain tuning properties of anterior, but not middle, face-processing areas in macaque inferotemporal cortex , 2020, Communications Biology.

[57]  Elias B. Issa,et al.  Neurophysiological Organization of the Middle Face Patch in Macaque Inferior Temporal Cortex , 2016, The Journal of Neuroscience.

[58]  P. Goldman-Rakic,et al.  Preface: Cerebral Cortex Has Come of Age , 1991 .

[59]  Chengxu Zhuang,et al.  Local Aggregation for Unsupervised Learning of Visual Embeddings , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[60]  Thomas R. Clandinin,et al.  The Influence of Wiring Economy on Nervous System Evolution , 2016, Current Biology.

[61]  Bevil R. Conway,et al.  Color-Biased Regions of the Ventral Visual Pathway Lie between Face- and Place-Selective Regions in Humans, as in Macaques , 2016, The Journal of Neuroscience.

[62]  Leslie G. Ungerleider,et al.  The Effect of Face Inversion on Activity in Human Neural Systems for Face and Object Perception , 1999, Neuron.

[63]  D I Perrett,et al.  Organization and functions of cells responsive to faces in the temporal cortex. , 1992, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[64]  Dmitri B. Chklovskii,et al.  Wiring Optimization in Cortical Circuits , 2002, Neuron.

[65]  J. Hegdé,et al.  A comparative study of shape representation in macaque visual areas v2 and v4. , 2007, Cerebral cortex.

[66]  M. Tarr,et al.  Visual Object Recognition , 1996, ISTCS.

[67]  Alice J. O'Toole,et al.  Partially Distributed Representations of Objects and Faces in Ventral Temporal Cortex , 2005, Journal of Cognitive Neuroscience.

[68]  Michal Irani,et al.  Convergent evolution of face spaces across human face-selective neuronal groups and deep convolutional networks , 2019, Nature Communications.

[69]  C. Gross,et al.  Representations of faces and body parts in macaque temporal cortex: a functional MRI study. , 2005, Proceedings of the National Academy of Sciences of the United States of America.

[70]  M. A. Smith,et al.  Spatial and Temporal Scales of Neuronal Correlation in Primary Visual Cortex , 2008, The Journal of Neuroscience.

[71]  Leslie G. Ungerleider,et al.  Object vision and spatial vision: two cortical pathways , 1983, Trends in Neurosciences.

[72]  M. Silverman,et al.  Spatial frequency columns in primary visual cortex. , 1981, Science.

[73]  Doris Y. Tsao,et al.  Mechanisms of face perception. , 2008, Annual review of neuroscience.

[74]  James J. DiCarlo,et al.  Pinwheel-like Iso-Orientation Domains in a Convolutional Neural Network Model , 2018 .

[75]  Bolei Zhou,et al.  Places: A 10 Million Image Database for Scene Recognition , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[76]  D. Hubel,et al.  Segregation of form, color, movement, and depth: anatomy, physiology, and perception. , 1988, Science.

[77]  N. Kanwisher,et al.  A stable topography of selectivity for unfamiliar shape classes in monkey inferior temporal cortex. , 2008, Cerebral cortex.

[78]  Joshua B. Tenenbaum,et al.  Efficient inverse graphics in biological face processing , 2018, Science Advances.

[79]  Margaret S Livingstone,et al.  Retinotopic Organization of Scene Areas in Macaque Inferior Temporal Cortex , 2017, The Journal of Neuroscience.

[80]  Doris Y. Tsao,et al.  A Cortical Region Consisting Entirely of Face-Selective Cells , 2006, Science.

[81]  N. Kanwisher,et al.  Domain specificity in visual cortex. , 2006, Cerebral cortex.

[82]  Ha Hong,et al.  The Neural Representation Benchmark and its Evaluation on Brain and Machine , 2013, ICLR.

[83]  Doris Y. Tsao,et al.  Patches with Links: A Unified System for Processing Faces in the Macaque Temporal Lobe , 2008, Science.

[84]  T. Poggio,et al.  Object Selectivity of Local Field Potentials and Spikes in the Macaque Inferior Temporal Cortex , 2006, Neuron.

[85]  D. Chklovskii,et al.  Maps in the brain: what can we learn from them? , 2004, Annual review of neuroscience.