The time course of individual face recognition: A pattern analysis of ERP signals

An extensive body of work documents the time course of neural face processing in the human visual cortex. However, the majority of this work has focused on specific temporal landmarks, such as N170 and N250 components, derived through univariate analyses of EEG data. Here, we take on a broader evaluation of ERP signals related to individual face recognition as we attempt to move beyond the leading theoretical and methodological framework through the application of pattern analysis to ERP data. Specifically, we investigate the spatiotemporal profile of identity recognition across variation in emotional expression. To this end, we apply pattern classification to ERP signals both in time, for any single electrode, and in space, across multiple electrodes. Our results confirm the significance of traditional ERP components in face processing. At the same time though, they support the idea that the temporal profile of face recognition is incompletely described by such components. First, we show that signals associated with different facial identities can be discriminated from each other outside the scope of these components, as early as 70ms following stimulus presentation. Next, electrodes associated with traditional ERP components as well as, critically, those not associated with such components are shown to contribute information to stimulus discriminability. And last, the levels of ERP-based pattern discrimination are found to correlate with recognition accuracy across subjects confirming the relevance of these methods for bridging brain and behavior data. Altogether, the current results shed new light on the fine-grained time course of neural face processing and showcase the value of novel methods for pattern analysis to investigating fundamental aspects of visual recognition.

[1]  James W. Tanaka,et al.  Activation of Preexisting and Acquired Face Representations: The N250 Event-related Potential as an Index of Face Familiarity , 2006, Journal of Cognitive Neuroscience.

[2]  K. Nakayama,et al.  Rapid adaptation of the m170 response: importance of face parts. , 2008, Cerebral cortex.

[3]  Guillaume A. Rousselet,et al.  Quantifying the Time Course of Visual Object Processing Using ERPs: It's Time to Up the Game , 2011, Front. Psychology.

[4]  C. Mondloch,et al.  The timing of individual face recognition in the brain , 2012, Neuropsychologia.

[5]  Daniel J. Acheson,et al.  Increased Alpha-Band Power during the Retention of Shapes and Shape-Location Associations in Visual Short-Term Memory , 2011, Front. Psychology.

[6]  Guillaume Thierry,et al.  Is the N170 peak of visual event-related brain potentials car-selective? , 2009, Neuroreport.

[7]  Bruno Rossion,et al.  Early Visually Evoked Electrophysiological Responses Over the Human Brain (P1, N170) Show Stable Patterns of Face-Sensitivity from 4 years to Adulthood , 2009, Front. Hum. Neurosci..

[8]  A. Caramazza,et al.  Decoding representations of face identity that are tolerant to rotation. , 2014, Cerebral cortex.

[9]  E. Halgren,et al.  Cognitive response profile of the human fusiform face area as determined by MEG. , 2000, Cerebral cortex.

[10]  Y. Benjamini,et al.  Controlling the false discovery rate: a practical and powerful approach to multiple testing , 1995 .

[11]  Benjamin Balas,et al.  EEG correlates of categorical and graded face perception , 2011, Neuropsychologia.

[12]  Guillaume Thierry,et al.  Face-Sensitive Processes One Hundred Milliseconds after Picture Onset , 2011, Front. Hum. Neurosci..

[13]  J. Tanaka,et al.  A Neural Basis for Expert Object Recognition , 2001, Psychological science.

[14]  Doris Y. Tsao,et al.  A face feature space in the macaque temporal lobe , 2009, Nature Neuroscience.

[15]  A. Ishai,et al.  Distributed and Overlapping Representations of Faces and Objects in Ventral Temporal Cortex , 2001, Science.

[16]  D G Pelli,et al.  The VideoToolbox software for visual psychophysics: transforming numbers into movies. , 1997, Spatial vision.

[17]  Bruno Rossion,et al.  Controlling interstimulus perceptual variance does not abolish N170 face sensitivity , 2007, Nature Neuroscience.

[18]  Katsumi Aoki,et al.  Recent development of flow visualization , 2004, J. Vis..

[19]  Marlene Behrmann,et al.  Unraveling the distributed neural code of facial identity through spatiotemporal pattern analysis , 2011, Proceedings of the National Academy of Sciences.

[20]  Daniel A. Braun,et al.  Risk-Sensitivity in Sensorimotor Control , 2011, Front. Hum. Neurosci..

[21]  M. Herrmann,et al.  Early stages (P100) of face perception in humans as measured with event-related potentials (ERPs) , 2004, Journal of Neural Transmission.

[22]  Hyeonjoon Moon,et al.  The FERET Evaluation Methodology for Face-Recognition Algorithms , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[23]  Margot J. Taylor,et al.  Inversion and Contrast Polarity Reversal Affect both Encoding and Recognition Processes of Unfamiliar Faces: A Repetition Study Using ERPs , 2002, NeuroImage.

[24]  Alice J. O'Toole,et al.  Dissociable Neural Patterns of Facial Identity across Changes in Viewpoint , 2010, Journal of Cognitive Neuroscience.

[25]  B. Balas,et al.  N170 face specificity and face memory depend on hometown size , 2015, Neuropsychologia.

[26]  Philippe G Schyns,et al.  Decoding face categories in diagnostic subregions of primary visual cortex , 2013, The European journal of neuroscience.

[27]  Salil H. Patel,et al.  Characterization of N200 and P300: Selected Studies of the Event-Related Potential , 2005, International journal of medical sciences.

[28]  Dan Nemrodov,et al.  Is the rapid adaptation paradigm too rapid? Implications for face and object processing , 2012, NeuroImage.

[29]  Guido Sanguinetti,et al.  Single-trial classification of EEG in a visual object task using ICA and machine learning , 2014, Journal of Neuroscience Methods.

[30]  L. Deouell,et al.  STRUCTURAL ENCODING AND IDENTIFICATION IN FACE PROCESSING: ERP EVIDENCE FOR SEPARATE MECHANISMS , 2000, Cognitive neuropsychology.

[31]  H. P. Op de Beeck,et al.  Representations of Facial Identity Information in the Ventral Visual Stream Investigated with Multivoxel Pattern Analyses , 2013, The Journal of Neuroscience.

[32]  Nikolaus Kriegeskorte,et al.  Comparison of multivariate classifiers and response normalizations for pattern-information fMRI , 2010, NeuroImage.

[33]  A. Burton,et al.  Human brain potential correlates of repetition priming in face and name recognition , 2002, Neuropsychologia.

[34]  C. Jacques,et al.  Early adaptation to repeated unfamiliar faces across viewpoint changes in the right hemisphere: Evidence from the N170 ERP component , 2009, Neuropsychologia.

[35]  A. Burton,et al.  Matching faces for semantic information and names: an event-related brain potentials study. , 2003, Brain research. Cognitive brain research.

[36]  Yufei Huang,et al.  Characterization and Robust Classification of EEG Signal from Image RSVP Events with Independent Time-Frequency Features , 2012, PloS one.

[37]  Sidney R. Lehky.  Fine Discrimination of Faces can be Performed Rapidly , 2000, Journal of Cognitive Neuroscience.

[38]  T. Sejnowski,et al.  Removing electroencephalographic artifacts by blind source separation. , 2000, Psychophysiology.

[39]  Christopher M. Bishop,et al.  Pattern Recognition and Machine Learning (Information Science and Statistics) , 2006 .

[40]  Bruno Rossion,et al.  Does physical interstimulus variance account for early electrophysiological face sensitive responses in the human brain? Ten lessons on the N170 , 2008, NeuroImage.

[41]  S. Tobimatsu,et al.  Early ERP components differentially extract facial features: Evidence for spatial frequency-and-contrast detectors , 2008, Neurosciences research.

[42]  C. Jacques,et al.  The N170 : understanding the time-course of face perception in the human brain , 2011 .

[43]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[44]  Aleix M. Martinez,et al.  The AR face database , 1998 .

[45]  Harry Wechsler,et al.  The FERET database and evaluation procedure for face-recognition algorithms , 1998, Image Vis. Comput..

[46]  Bruno Rossion,et al.  Early lateralization and orientation tuning for face, word, and object processing in the visual cortex , 2003, NeuroImage.

[47]  Robert F. Hess,et al.  Extracting the internal representation of faces from human brain activity: An analogue to reverse correlation , 2010, NeuroImage.

[48]  K. Linkenkaer-Hansen,et al.  Face-selective processing in human extrastriate cortex around 120 ms after stimulus onset revealed by magneto- and electroencephalography , 1998, Neuroscience Letters.

[49]  B. Renault,et al.  ERPs and chronometry of face recognition: following‐up Seeck et al. and George et al. , 1998, Neuroreport.

[50]  D H Brainard,et al.  The Psychophysics Toolbox. , 1997, Spatial vision.

[51]  T. Allison,et al.  Electrophysiological Studies of Face Perception in Humans , 1996, Journal of Cognitive Neuroscience.

[52]  Skyler T. Hawk,et al.  Presentation and validation of the Radboud Faces Database , 2010 .

[53]  N. Kanwisher,et al.  Stages of processing in face perception: an MEG study , 2002, Nature Neuroscience.

[54]  J. Ford,et al.  ERPs to response production and inhibition. , 1985, Electroencephalography and clinical neurophysiology.

[55]  P. Schyns,et al.  Measuring Internal Representations from Behavioral and Brain Data , 2012, Current Biology.

[56]  S. Hsieh,et al.  A factor-adjusted multiple testing procedure for ERP data analysis , 2012, Behavior research methods.

[57]  M. Eimer.  The Face-Sensitive N170 Component of the Event-Related Brain Potential , 2011 .

[58]  B. Rossion,et al.  ERP evidence for the speed of face categorization in the human brain: Disentangling the contribution of low-level visual cues from face perception , 2011, Vision Research.

[59]  F. Eustache,et al.  The time course of repetition effects for familiar faces and objects: An ERP study , 2009, Brain Research.

[60]  J. Shedden,et al.  Automatic face identity encoding at the N170 , 2006, Vision Research.

[61]  C. Jacques,et al.  The time course of the inversion effect during individual face discrimination. , 2007, Journal of vision.

[62]  M. Giard,et al.  Electrophysiological Correlates of Age and Gender Perception on Human Faces , 2003, Journal of Cognitive Neuroscience.

[63]  R. Goebel,et al.  Individual faces elicit distinct response patterns in human anterior temporal cortex , 2007, Proceedings of the National Academy of Sciences.

[64]  Bernadette M. Jansma,et al.  Temporal dynamics of face selection mechanism in the context of similar and dissimilar faces: ERP evidence for biased competition within the ventral occipito-temporal cortex using ICA , 2012, NeuroImage.