Feature-based encoding of face identity by single neurons in the human amygdala and hippocampus

Neurons in the human medial temporal lobe (MTL) that are selective for the identity of specific people are classically thought to encode identity invariant to visual features. However, it remains largely unknown how visual information from higher visual cortex is translated into a semantic representation of an individual person. Here, we show that some MTL neurons are selective to multiple different face identities on the basis of shared features that form clusters in the representation of a deep neural network trained to recognize faces. Contrary to prevailing views, we find that these neurons represent an individual’s face with feature-based encoding, rather than through association with concepts. The response of feature neurons did not depend on face identity nor face familiarity, and the region of feature space to which they are tuned predicted their response to new face stimuli. Our results provide critical evidence bridging the perception-driven representation of facial features in the higher visual cortex and the memory-driven representation of semantics in the MTL, which may form the basis for declarative memory.

[1]  Shuo Wang,et al.  Neurons in the human amygdala selective for perceived emotion , 2014, Proceedings of the National Academy of Sciences.

[2]  Charles L. Wilson,et al.  Single Neuron Activity in Human Hippocampus and Amygdala during Recognition of Faces and Objects , 1997, Neuron.

[3]  Terrence J. Sejnowski,et al.  The Computational Brain , 1996, Artif. Intell..

[4]  Donald O. Walter,et al.  Mass action in the nervous system , 1975 .

[5]  Doris Y. Tsao,et al.  A face feature space in the macaque temporal lobe , 2009, Nature Neuroscience.

[6]  H. Barlow,et al.  Single Units and Sensation: A Neuron Doctrine for Perceptual Psychology? , 1972, Perception.

[7]  A. Todorov,et al.  The functional basis of face evaluation , 2008, Proceedings of the National Academy of Sciences.

[8]  A. Treves,et al.  The representational capacity of the distributed encoding of information provided by populations of neurons in primate temporal visual cortex , 1997, Experimental Brain Research.

[9]  M. Giese,et al.  Norm-based face encoding by single neurons in the monkey inferotemporal cortex , 2006, Nature.

[10]  T. Valentine.  The Quarterly Journal of Experimental Psychology Section A: Human Experimental Psychology a Unified Account of the Effects of Distinctiveness, Inversion, and Race in Face Recognition , 2022 .

[11]  I. Fried,et al.  Long-term coding of personal and universal associations underlying the memory web in the human brain , 2016, Nature Communications.

[12]  C. Koch,et al.  Category-specific visual responses of single neurons in the human medial temporal lobe , 2000, Nature Neuroscience.

[13]  R. Quiroga.  Concept cells: the building blocks of declarative memory functions , 2012, Nature Reviews Neuroscience.

[14]  S T Roweis,et al.  Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.

[15]  James J DiCarlo,et al.  Neural population control via deep image synthesis , 2018, Science.

[16]  J. Tenenbaum,et al.  A global geometric framework for nonlinear dimensionality reduction. , 2000, Science.

[17]  S. Kochen,et al.  Single Neuron Coding of Identity in the Human Hippocampal Formation , 2020, Current Biology.

[18]  Ha Hong,et al.  Performance-optimized hierarchical models predict neural responses in higher visual cortex , 2014, Proceedings of the National Academy of Sciences.

[19]  Doris Y. Tsao,et al.  The Code for Facial Identity in the Primate Brain , 2017, Cell.

[20]  Vaidehi S. Natu,et al.  The functional neuroanatomy of face perception: from brain measurements to deep neural networks , 2018, Interface Focus.

[21]  Shuo Wang,et al.  The human amygdala parametrically encodes the intensity of specific facial emotions and their categorical ambiguity , 2017, Nature Communications.

[22]  C. Koch,et al.  Invariant visual representation by single neurons in the human brain , 2005, Nature.

[23]  Mason McGill,et al.  A map of object space in primate inferotemporal cortex , 2020, Nature.

[24]  Ueli Rutishauser,et al.  Testing Models of Human Declarative Memory at the Single-Neuron Level , 2019, Trends in Cognitive Sciences.

[25]  Michal Irani,et al.  Convergent evolution of face spaces across human face-selective neuronal groups and deep convolutional networks , 2019, Nature Communications.

[26]  Shuo Wang,et al.  Encoding of Target Detection during Visual Search by Single Neurons in the Human Brain , 2018, Current Biology.

[27]  H B Barlow,et al.  Single units and sensation: a neuron doctrine for perceptual psychology? , 1972, Perception.

[28]  Geoffrey E. Hinton,et al.  Distributed Representations , 1986, The Philosophy of Artificial Intelligence.

[29]  R. Jindra.  Mass action in the nervous system W. J. Freeman, Academic Press, New York (1975), 489 pp., (hard covers). $34.50 , 1976, Neuroscience.

[30]  Carlos R. Ponce,et al.  Evolving Images for Visual Neurons Using a Deep Generative Network Reveals Coding Principles and Neuronal Preferences , 2019, Cell.