A specialized face-processing network consistent with the representational geometry of monkey face patches

Ample evidence suggests that face processing in human and non-human primates is performed differently compared with other objects. Converging reports, both physiologically and psychophysically, indicate that faces are processed in specialized neural networks in the brain -i.e. face patches in monkeys and the fusiform face area (FFA) in humans. We are all expert face-processing agents, and able to identify very subtle differences within the category of faces, despite substantial visual and featural similarities. Identification is performed rapidly and accurately after viewing a whole face, while significantly drops if some of the face configurations (e.g. inversion, misalignment) are manipulated or if partial views of faces are shown due to occlusion. This refers to a hotly-debated, yet highly-supported concept, known as holistic face processing. We built a hierarchical computational model of face-processing based on evidence from recent neuronal and behavioural studies on faces processing in primates. Representational geometries of the last three layers of the model have characteristics similar to those observed in monkey face patches (posterior, middle and anterior patches). Furthermore, several face-processing-related phenomena reported in the literature automatically emerge as properties of this model. The representations are evolved through several computational layers, using biologically plausible learning rules. The model satisfies face inversion effect, composite face effect, other race effect, view and identity selectivity, and canonical face views. To our knowledge, no models have so far been proposed with this performance and agreement with biological data.

[1]  Doris Y. Tsao,et al.  A Cortical Region Consisting Entirely of Face-Selective Cells , 2006, Science.

[2]  D. Hubel,et al.  Uniformity of monkey striate cortex: A parallel relationship between field size, scatter, and magnification factor , 1974, The Journal of comparative neurology.

[3]  Doris Y. Tsao,et al.  Patches with Links: A Unified System for Processing Faces in the Macaque Temporal Lobe , 2008, Science.

[4]  Nikolaus Kriegeskorte,et al.  Frontiers in Systems Neuroscience Systems Neuroscience , 2022 .

[5]  Seyed-Mahdi Khaligh-Razavi,et al.  What you need to know about the state-of-the-art computational models of object-vision: A tour through the models , 2014, ArXiv.

[6]  Nikolaus Kriegeskorte,et al.  Deep Supervised, but Not Unsupervised, Models May Explain IT Cortical Representation , 2014, PLoS Comput. Biol..

[7]  Thomas Serre,et al.  Hierarchical Models of the Visual System , 2014, Encyclopedia of Computational Neuroscience.

[8]  R N Shepard,et al.  Multidimensional Scaling, Tree-Fitting, and Clustering , 1980, Science.

[9]  Reza Ebrahimpour,et al.  Feedforward object-vision models only tolerate small image variations compared to human , 2014, Front. Comput. Neurosci..

[10]  Nancy Kanwisher,et al.  A cortical representation of the local visual environment , 1998, Nature.

[11]  Guy Wallis,et al.  Toward a unified model of face and object recognition in the human visual system , 2013, Front. Psychol..

[12]  Nicolás Pinto Forward engineering object recognition : a scalable approach , 2011 .

[13]  B. Rossion The composite face illusion: A whole window into our understanding of holistic face perception , 2013 .

[14]  Nicolas Pinto,et al.  Comparing state-of-the-art visual features on invariant object recognition tasks , 2011, 2011 IEEE Workshop on Applications of Computer Vision (WACV).

[15]  E. Rolls The representation of information about faces in the temporal and frontal lobes , 2007, Neuropsychologia.

[16]  Radoslaw Martin Cichy,et al.  Resolving human object recognition in space and time , 2014, Nature Neuroscience.

[17]  M. D’Esposito,et al.  The Inferential Impact of Global Signal Covariates in Functional Neuroimaging Analyses , 1998, NeuroImage.

[18]  B. Rossion,et al.  Nonlinear relationship between holistic processing of individual faces and picture-plane rotation: evidence from the face composite illusion. , 2008, Journal of vision.

[19]  M. Riesenhuber,et al.  Face processing in humans is compatible with a simple shape–based model of vision , 2004, Proceedings of the Royal Society of London. Series B: Biological Sciences.

[20]  D. Maurer,et al.  The many faces of configural processing , 2002, Trends in Cognitive Sciences.

[21]  N. Kanwisher,et al.  The Fusiform Face Area: A Module in Human Extrastriate Cortex Specialized for Face Perception , 1997, The Journal of Neuroscience.

[22]  Doris Y. Tsao,et al.  Functional Compartmentalization and Viewpoint Generalization Within the Macaque Face-Processing System , 2010, Science.

[23]  Seyed-Mahdi Khaligh-Razavi,et al.  A Stable Biologically Motivated Learning Mechanism for Visual Feature Extraction to Handle Facial Categorization , 2012, PloS one.

[24]  N. Kanwisher,et al.  The Human Body , 2001 .

[25]  A. O'Toole,et al.  Stimulus-specific effects in face recognition over changes in viewpoint , 1998, Vision Research.

[26]  Bruno Rossion,et al.  Early Deafness Increases the Face Inversion Effect But Does Not Modulate the Composite Face Effect , 2012, Front. Psychology.

[27]  P. Thompson,et al.  Margaret Thatcher: A New Illusion , 1980, Perception.

[28]  M. Farah,et al.  What is "special" about face perception? , 1998, Psychological review.

[29]  K. Nakayama,et al.  The effect of face inversion on the human fusiform face area , 1998, Cognition.

[30]  Joan Y. Chiao,et al.  Differential responses in the fusiform region to same-race and other-race faces , 2001, Nature Neuroscience.

[31]  Bruno Rossion,et al.  Same-race faces are perceived more holistically than other-race faces , 2004 .

[32]  Thomas Serre,et al.  Robust Object Recognition with Cortex-Like Mechanisms , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[33]  Joel Z. Leibo,et al.  Learning and disrupting invariance in visual recognition with a temporal association rule , 2011, Front. Comput. Neurosci..

[34]  A. Ishai,et al.  Distributed and Overlapping Representations of Faces and Objects in Ventral Temporal Cortex , 2001, Science.

[35]  N. Kanwisher,et al.  The fusiform face area subserves face perception, not generic within-category identification , 2004, Nature Neuroscience.

[36]  Bruno Rossion,et al.  An Experience-Based Holistic Account , 2011 .

[37]  T. Poggio,et al.  Hierarchical models of object recognition in cortex , 1999, Nature Neuroscience.

[38]  Leslie G. Ungerleider,et al.  Distributed Neural Systems for the Generation of Visual Images , 2000, Neuron.

[39]  M. Merzenich,et al.  Cortical plasticity and memory , 1993, Current Opinion in Neurobiology.

[40]  Tomaso Poggio,et al.  Faces as a "Model Category" for Visual Object Recognition , 2013 .

[41]  M. Riesenhuber,et al.  Evaluation of a Shape-Based Model of Human Face Discrimination Using fMRI and Behavioral Techniques , 2006, Neuron.

[42]  C. Koch,et al.  Category-specific visual responses of single neurons in the human medial temporal lobe , 2000, Nature Neuroscience.

[43]  Thomas Serre,et al.  Models of visual cortex , 2013, Scholarpedia.

[44]  T. Poggio,et al.  Vision: are models of object recognition catching up with the brain? , 2013, Annals of the New York Academy of Sciences.

[45]  Nancy Kanwisher,et al.  The distribution of category and location information across object-selective regions in human visual cortex , 2008, Proceedings of the National Academy of Sciences.

[46]  Doris Y. Tsao,et al.  Faces and objects in macaque cerebral cortex , 2003, Nature Neuroscience.

[47]  N. Logothetis,et al.  Psychophysical and physiological evidence for viewer-centered object representations in the primate. , 1995, Cerebral cortex.

[48]  PoggioTomaso,et al.  Robust Object Recognition with Cortex-Like Mechanisms , 2007 .

[49]  D. Leopold,et al.  Face-selective neurons maintain consistent visual responses across months , 2014, Proceedings of the National Academy of Sciences.

[50]  A. Ishai,et al.  Distributed neural systems for the generation of visual images , 2000, NeuroImage.

[51]  B. Rossion Picture-plane inversion leads to qualitative changes of face perception. , 2008, Acta psychologica.

[52]  Ju-Chin Chen,et al.  A view-based statistical system for multi-view face detection and pose estimation , 2009, Image Vis. Comput..

[53]  D I Perrett,et al.  Organization and functions of cells responsive to faces in the temporal cortex. , 1992, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[54]  C. Gilbert,et al.  Adult Visual Cortical Plasticity , 2012, Neuron.

[55]  Isabel Gauthier,et al.  Race-Specific Perceptual Discrimination Improvement Following Short Individuation Training With Faces , 2010, Cogn. Sci..

[56]  V. Goffaux,et al.  Spatio-temporal localization of the face inversion effect: an event-related potentials study , 1999, Biological Psychology.

[57]  Avi Chaudhuri,et al.  Reassessing the 3/4 view effect in face recognition , 2002, Cognition.

[58]  R. Yin Looking at Upside-down Faces , 1969 .

[59]  Li Su,et al.  A Toolbox for Representational Similarity Analysis , 2014, PLoS Comput. Biol..

[60]  D. Hubel,et al.  Shape and arrangement of columns in cat's striate cortex , 1963, The Journal of physiology.

[61]  N. Kanwisher,et al.  How Distributed Is Visual Category Information in Human Occipito-Temporal Cortex? An fMRI Study , 2002, Neuron.

[62]  J. DiCarlo,et al.  Learning and neural plasticity in visual object recognition , 2006, Current Opinion in Neurobiology.

[63]  V. Bruce,et al.  Local and Relational Aspects of Face Distinctiveness , 1998, The Quarterly journal of experimental psychology. A, Human experimental psychology.

[64]  G. Rhodes,et al.  Processes Underlying the Cross-Race Effect: An Investigation of Holistic, Featural, and Relational Processing of Own-Race versus Other-Race Faces , 2010, Perception.

[65]  N. Kanwisher,et al.  The Neural Basis of the Behavioral Face-Inversion Effect , 2005, Current Biology.

[66]  S. Carey,et al.  From piecemeal to configurational representation of faces. , 1977, Science.

[67]  Rafael Malach,et al.  Large-Scale Mirror-Symmetry Organization of Human Occipito-Temporal Object Areas , 2003, Neuron.

[68]  C. Gross,et al.  Representations of faces and body parts in macaque temporal cortex: a functional MRI study. , 2005, Proceedings of the National Academy of Sciences of the United States of America.

[69]  Bruno Rossion,et al.  Face Perception is Whole or None: Disentangling the Role of Spatial Contiguity and Interfeature Distances in the Composite Face Illusion , 2013, Perception.

[70]  Doris Y. Tsao,et al.  Mechanisms of face perception. , 2008, Annual review of neuroscience.

[71]  M J Tarr,et al.  What Object Attributes Determine Canonical Views? , 1999, Perception.

[72]  C. Gilbert Plasticity in visual perception and physiology , 1996, Current Opinion in Neurobiology.

[73]  Seyed-Mahdi Khaligh-Razavi,et al.  How Can Selection of Biologically Inspired Features Improve the Performance of a Robust Object Recognition Model? , 2012, PloS one.

[74]  Jeffrey S Bowers,et al.  On the biological plausibility of grandmother cells: implications for neural network theories in psychology and neuroscience. , 2009, Psychological review.

[75]  D. Hubel,et al.  The pattern of ocular dominance columns in macaque visual cortex revealed by a reduced silver stain , 1975, The Journal of comparative neurology.

[76]  M. Farah,et al.  Parts and Wholes in Face Recognition , 1993, The Quarterly journal of experimental psychology. A, Human experimental psychology.

[77]  Gary L. Allen,et al.  Seeking a Common Gestalt Approach to the Perception of Faces, Objects, and Scenes , 2006 .

[78]  E. Maguire,et al.  Knowing Where Things Are: Parahippocampal Involvement in Encoding Object Locations in Virtual Large-Scale Space , 1998, Journal of Cognitive Neuroscience.

[79]  Keiji Tanaka,et al.  Matching Categorical Object Representations in Inferior Temporal Cortex of Man and Monkey , 2008, Neuron.

[80]  Elias B. Issa,et al.  Precedence of the Eye Region in Neural Processing of Faces , 2012, The Journal of Neuroscience.

[81]  Joel Z. Leibo,et al.  Why The Brain Separates Face Recognition From Object Recognition , 2011, NIPS.

[82]  B. Rossion Understanding face perception by means of human electrophysiology , 2014, Trends in Cognitive Sciences.

[83]  Thomas Serre,et al.  What are the Visual Features Underlying Rapid Object Recognition? , 2011, Front. Psychology.

[84]  Stephen Grossberg,et al.  Adaptive Resonance Theory: How a brain learns to consciously attend, learn, and recognize a changing world , 2013, Neural Networks.

[85]  Reza Ebrahimpour,et al.  The importance of visual features in generic vs. specialized object recognition: a computational study , 2014, Front. Comput. Neurosci..

[86]  C. Koch,et al.  Invariant visual representation by single neurons in the human brain , 2005, Nature.