Sensory and Motor Systems The Neural Dynamics of Facial Identity Processing : Insights from EEG-Based Pattern Analysis and Image Reconstruction

Uncovering the neural dynamics of facial identity processing along with its representational basis outlines a major endeavor in the study of visual processing. To this end, here, we record human electroencephalography (EEG) data associated with viewing face stimuli; then, we exploit spatiotemporal EEG information to determine the neural correlates of facial identity representations and to reconstruct the appearance of the corresponding stimuli. Our findings indicate that multiple temporal intervals support: facial identity classification, face space estimation, visual feature extraction and image reconstruction. In particular, we note that both classification and reconstruction accuracy peak in the proximity of the N170 component. Further, aggregate data from a larger interval (50–650 ms after stimulus onset) support robust reconstruction results, consistent with the availability of distinct visual information over time. Thus, theoretically, our findings shed light on the time course of face processing while, methodologically, they demonstrate the feasibility of EEG-based image reconstruction.

[1]  T. Valentine The Quarterly Journal of Experimental Psychology Section A: Human Experimental Psychology a Unified Account of the Effects of Distinctiveness, Inversion, and Race in Face Recognition , 2022 .

[2]  Donald A. Jackson PROTEST: A PROcrustean Randomization TEST of community environment concordance , 1995 .

[3]  D H Brainard,et al.  The Psychophysics Toolbox. , 1997, Spatial vision.

[4]  D G Pelli,et al.  The VideoToolbox software for visual psychophysics: transforming numbers into movies. , 1997, Spatial vision.

[5]  T. Valentine,et al.  Recognizing Unfamiliar Faces: The Effects of Distinctiveness and View , 1999, The Quarterly journal of experimental psychology. A, Human experimental psychology.

[6]  S. R. Lehky Fine Discrimination of Faces can be Performed Rapidly , 2000, Journal of Cognitive Neuroscience.

[7]  L. Deouell,et al.  STRUCTURAL ENCODING AND IDENTIFICATION IN FACE PROCESSING: ERP EVIDENCE FOR SEPARATE MECHANISMS , 2000, Cognitive neuropsychology.

[8]  J. Tanaka,et al.  A Neural Basis for Expert Object Recognition , 2001, Psychological science.

[9]  A. Burton,et al.  Human brain potential correlates of repetition priming in face and name recognition , 2002, Neuropsychologia.

[10]  Margot J. Taylor,et al.  Inversion and Contrast Polarity Reversal Affect both Encoding and Recognition Processes of Unfamiliar Faces: A Repetition Study Using ERPs , 2002, NeuroImage.

[11]  A. Burton,et al.  Matching faces for semantic information and names: an event-related brain potentials study. , 2003, Brain research. Cognitive brain research.

[12]  P. Schyns,et al.  Superstitious Perceptions Reveal Properties of Internal Representations , 2003, Psychological science.

[13]  P. Bennett,et al.  Inversion Leads to Quantitative, Not Qualitative, Changes in Face Processing , 2004, Current Biology.

[14]  A. Treves,et al.  Morphing Marilyn into Maggie dissociates physical and identity face representations in the brain , 2005, Nature Neuroscience.

[15]  Tomaso Poggio,et al.  Fast Readout of Object Identity from Macaque Inferior Temporal Cortex , 2005, Science.

[16]  H. Wilson,et al.  fMRI evidence for the neural representation of faces , 2005, Nature Neuroscience.

[17]  K. Nakayama,et al.  The Cambridge Face Memory Test: Results for neurologically intact individuals and an investigation of its validity using inverted face stimuli and prosopagnosic participants , 2006, Neuropsychologia.

[18]  James W. Tanaka,et al.  Activation of Preexisting and Acquired Face Representations: The N250 Event-related Potential as an Index of Face Familiarity , 2006, Journal of Cognitive Neuroscience.

[19]  M. Giese,et al.  Norm-based face encoding by single neurons in the monkey inferotemporal cortex , 2006, Nature.

[20]  J. Shedden,et al.  Progressive N170 habituation to unattended repeated faces , 2006, Vision Research.

[21]  A. O'Toole,et al.  Probing the Visual Representation of Faces With Adaptation , 2006, Psychological science.

[22]  Michael C. Mangini,et al.  Do Humans and Baboons Use the Same Information When Categorizing Human and Baboon Faces? , 2006, Psychological science.

[23]  C. Jacques,et al.  The time course of the inversion effect during individual face discrimination. , 2007, Journal of vision.

[24]  Edgar Erdfelder,et al.  G*Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences , 2007, Behavior research methods.

[25]  Terrence J. Sejnowski,et al.  Enhanced detection of artifacts in EEG data using higher-order statistics and independent component analysis , 2007, NeuroImage.

[26]  Yaroslav O. Halchenko,et al.  Brain Reading Using Full Brain Support Vector Machines for Object Recognition: There Is No Face Identification Area , 2008, Neural Computation.

[27]  Nikolaus Kriegeskorte,et al.  Frontiers in Systems Neuroscience Systems Neuroscience , 2022 .

[28]  A Mouraux,et al.  Across-trial averaging of event-related EEG responses and beyond. , 2008, Magnetic resonance imaging.

[29]  Masa-aki Sato,et al.  Visual Image Reconstruction from Human Brain Activity using a Combination of Multiscale Local Image Decoders , 2008, Neuron.

[30]  G. Kreiman,et al.  Timing, Timing, Timing: Fast Decoding of Object Information from Intracranial Field Potentials in Human Visual Cortex , 2009, Neuron.

[31]  G. Yovel,et al.  Diagnosing prosopagnosia: Effects of ageing, sex, and participant–stimulus ethnic match on the Cambridge Face Memory Test and Cambridge Face Perception Test , 2009, Cognitive neuropsychology.

[32]  C. Jacques,et al.  Early adaptation to repeated unfamiliar faces across viewpoint changes in the right hemisphere: Evidence from the N170 ERP component , 2009, Neuropsychologia.

[33]  Doris Y. Tsao,et al.  A face feature space in the macaque temporal lobe , 2009, Nature Neuroscience.

[34]  Ryan J. Prenger,et al.  Bayesian Reconstruction of Natural Images from Human Brain Activity , 2009, Neuron.

[35]  Skyler T. Hawk,et al.  Presentation and validation of the Radboud Faces Database , 2010 .

[36]  Sébastien M. Crouzet,et al.  Fast saccades toward faces: face detection in just 100 ms. , 2010, Journal of vision.

[37]  C. Thomaz,et al.  A new ranking method for principal components analysis and its application to face image analysis , 2010, Image Vis. Comput..

[38]  Marlene Behrmann,et al.  Unraveling the distributed neural code of facial identity through spatiotemporal pattern analysis , 2011, Proceedings of the National Academy of Sciences.

[39]  I. Blank,et al.  The structure of face--space is tolerant to lighting and viewpoint transformations. , 2011, Journal of vision.

[40]  Stefan Haufe,et al.  Single-trial analysis and classification of ERP components — A tutorial , 2011, NeuroImage.

[41]  J. Gallant,et al.  Reconstructing Visual Experiences from Brain Activity Evoked by Natural Movies , 2011, Current Biology.

[42]  Richard F Murray,et al.  Classification images: A review. , 2011, Journal of vision.

[43]  B. Rossion,et al.  ERP evidence for the speed of face categorization in the human brain: Disentangling the contribution of low-level visual cues from face perception , 2011, Vision Research.

[44]  P. Schyns,et al.  Measuring Internal Representations from Behavioral and Brain Data , 2012, Current Biology.

[45]  C. Mondloch,et al.  The timing of individual face recognition in the brain , 2012, Neuropsychologia.

[46]  D. Plaut,et al.  Face-Space Architectures , 2013, Psychological science.

[47]  Marcel van Gerven,et al.  MEG-based decoding of the spatiotemporal dynamics of visual category perception , 2013, NeuroImage.

[48]  H. P. Op de Beeck,et al.  Representations of Facial Identity Information in the Ventral Visual Stream Investigated with Multivoxel Pattern Analyses , 2013, The Journal of Neuroscience.

[49]  H. Wilson,et al.  The neural representation of face space dimensions , 2013, Neuropsychologia.

[50]  Garrison W. Cottrell,et al.  What Evidence Supports Special Processing for Faces? A Cautionary Tale for fMRI Interpretation , 2013, Journal of Cognitive Neuroscience.

[51]  David A. Tovar,et al.  Representational dynamics of object vision: the first 1000 ms. , 2013, Journal of vision.

[52]  Thomas Serre,et al.  The Neural Dynamics of Face Detection in the Wild Revealed by MVPA , 2014, The Journal of Neuroscience.

[53]  J. Pearson New Directions in Mental-Imagery Research , 2014 .

[54]  Radoslaw Martin Cichy,et al.  Resolving human object recognition in space and time , 2014, Nature Neuroscience.

[55]  David M. Groppe,et al.  Exemplar selectivity reflects perceptual similarities in the human fusiform cortex. , 2014, Cerebral cortex.

[56]  John A. Pyles,et al.  Dynamic Encoding of Face Information in the Human Fusiform Gyrus , 2014, Nature Communications.

[57]  Brice A. Kuhl,et al.  Neural portraits of perception: Reconstructing face images from evoked brain activity , 2014, NeuroImage.

[58]  Joel Z. Leibo,et al.  The dynamics of invariant object recognition in the human visual system. , 2014, Journal of neurophysiology.

[59]  A. Caramazza,et al.  Decoding representations of face identity that are tolerant to rotation. , 2014, Cerebral cortex.

[60]  Marlene Behrmann,et al.  Feature-based face representations and image reconstruction from behavioral and neural data , 2015, Proceedings of the National Academy of Sciences.

[61]  A. Norcia,et al.  A Representational Similarity Analysis of the Dynamics of Object Processing Using Single-Trial EEG Classification , 2015, PloS one.

[62]  D. Plaut,et al.  Spatiotemporal dynamics of similarity-based neural representations of facial identity , 2016, Proceedings of the National Academy of Sciences.

[63]  Matthias Niemeier,et al.  The time course of individual face recognition: A pattern analysis of ERP signals , 2016, NeuroImage.

[64]  Isabel Gauthier,et al.  Are Face and Object Recognition Independent? A Neurocomputational Modeling Exploration , 2016, Journal of Cognitive Neuroscience.

[65]  A. Young,et al.  Contributions of feature shapes and surface cues to the recognition and neural representation of facial identity , 2016, Cortex.

[66]  Susan G. Wardle,et al.  Decoding Dynamic Brain Patterns from Evoked Responses: A Tutorial on Multivariate Pattern Analysis Applied to Time Series Neuroimaging Data , 2016, Journal of Cognitive Neuroscience.

[67]  Doris Y. Tsao,et al.  The Code for Facial Identity in the Primate Brain , 2017, Cell.