Unraveling the distributed neural code of facial identity through spatiotemporal pattern analysis

Face individuation is one of the most impressive achievements of our visual system, and yet uncovering the neural mechanisms subserving this feat appears to elude traditional approaches to functional brain data analysis. The present study investigates the neural code of facial identity perception with the aim of ascertaining its distributed nature and informational basis. To this end, we use a sequence of multivariate pattern analyses applied to functional magnetic resonance imaging (fMRI) data. First, we combine information-based brain mapping and dynamic discrimination analysis to locate spatiotemporal patterns that support face classification at the individual level. This analysis reveals a network of fusiform and anterior temporal areas that carry information about facial identity and provides evidence that the fusiform face area responds with distinct patterns of activation to different face identities. Second, we assess the information structure of the network using recursive feature elimination. We find that diagnostic information is distributed evenly among anterior regions of the mapped network and that a right anterior region of the fusiform gyrus plays a central role within the information network mediating face individuation. These findings serve to map out and characterize a cortical system responsible for individuation. More generally, in the context of functionally defined networks, they provide an account of distributed processing grounded in information-based architectures.

[1]  M. Tarr,et al.  Gender Recognition of Human Faces Using Color , 2008, Psychological science.

[2]  R. Goebel,et al.  Individual faces elicit distinct response patterns in human anterior temporal cortex , 2007, Proceedings of the National Academy of Sciences.

[3]  Tom M. Mitchell,et al.  Learning to Decode Cognitive States from Brain Images , 2004, Machine Learning.

[4]  T. Allison,et al.  Differential Sensitivity of Human Visual Cortex to Faces, Letterstrings, and Textures: A Functional Magnetic Resonance Imaging Study , 1996, The Journal of Neuroscience.

[5]  Talma Hendler,et al.  Eccentricity Bias as an Organizing Principle for Human High-Order Object Areas , 2002, Neuron.

[6]  T. Schormann,et al.  Activation Reduction in Anterior Temporal Cortices during Repeated Recognition of Faces of Personal Acquaintances , 2001, NeuroImage.

[7]  Timothy J. Andrews,et al.  Internal and External Features of the Face Are Represented Holistically in Face-Selective Regions of Visual Cortex , 2010, The Journal of Neuroscience.

[8]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[9]  M. Turk,et al.  Eigenfaces for Recognition , 1991, Journal of Cognitive Neuroscience.

[10]  Jascha D. Swisher,et al.  Multiscale Pattern Analysis of Orientation-Selective Activity in the Primary Visual Cortex , 2010, The Journal of Neuroscience.

[11]  P. Sinha,et al.  Contribution of Color to Face Recognition , 2002, Perception.

[12]  M. Giese,et al.  Norm-based face encoding by single neurons in the monkey inferotemporal cortex , 2006, Nature.

[13]  Michael McGill,et al.  Introduction to Modern Information Retrieval , 1983 .

[14]  Adrian Nestor,et al.  The segmental structure of faces and its use in gender recognition. , 2008, Journal of vision.

[15]  E. Halgren,et al.  Top-down facilitation of visual recognition. , 2006, Proceedings of the National Academy of Sciences of the United States of America.

[16]  Galia Avidan,et al.  Reduced structural connectivity in ventral visual cortex in congenital prosopagnosia , 2009, Nature Neuroscience.

[17]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[18]  A. Treves,et al.  Morphing Marilyn into Maggie dissociates physical and identity face representations in the brain , 2005, Nature Neuroscience.

[19]  Nikolaus Kriegeskorte,et al.  Analyzing for information, not activation, to exploit high-resolution fMRI , 2007, NeuroImage.

[20]  John T. Clark,et al.  Contextually Evoked Object-Specific Responses in Human Visual Cortex , 2004 .

[21]  Nikolaus Kriegeskorte,et al.  How does an fMRI voxel sample the neuronal activity pattern: Compact-kernel or complex spatiotemporal filter? , 2010, NeuroImage.

[22]  D G Pelli,et al.  The VideoToolbox software for visual psychophysics: transforming numbers into movies. , 1997, Spatial vision.

[23]  Kenneth A. Norman,et al.  Recollection, Familiarity, and Cortical Reinstatement: A Multivoxel Pattern Analysis , 2009, Neuron.

[24]  I. Gauthier,et al.  Expertise for cars and birds recruits brain areas involved in face recognition , 2000, Nature Neuroscience.

[25]  S. Dehaene,et al.  How Learning to Read Changes the Cortical Networks for Vision and Language , 2010, Science.

[26]  W. K. Simmons,et al.  The Selectivity and Functional Connectivity of the Anterior Temporal Lobes , 2009, Cerebral cortex.

[27]  R W Cox,et al.  AFNI: software for analysis and visualization of functional magnetic resonance neuroimages. , 1996, Computers and biomedical research, an international journal.

[28]  A. Ishai,et al.  Distributed and Overlapping Representations of Faces and Objects in Ventral Temporal Cortex , 2001, Science.

[29]  Timothy J. Andrews,et al.  Distinct representations for facial identity and changeable aspects of faces in the human temporal lobe , 2004, NeuroImage.

[30]  Giuseppe Iaria,et al.  The correlates of subjective perception of identity and expression in the face network: An fMRI adaptation study , 2009, NeuroImage.

[31]  Nikolaus Kriegeskorte,et al.  Comparison of multivariate classifiers and response normalizations for pattern-information fMRI , 2010, NeuroImage.

[32]  J. Keenan,et al.  Lesions of the fusiform face area impair perception of facial configuration in prosopagnosia , 2002, Neurology.

[33]  N. Na THE SCIENCE OF COLOR , 1952 .

[34]  Bernard Mazoyer,et al.  Word and non-word reading: What role for the Visual Word Form Area? , 2005, NeuroImage.

[35]  M. Tarr,et al.  The Fusiform Face Area is Part of a Network that Processes Faces at the Individual Level , 2000, Journal of Cognitive Neuroscience.

[36]  Mark D'Esposito,et al.  Variation of BOLD hemodynamic responses across subjects and brain regions and their effects on statistical analyses , 2004, NeuroImage.

[37]  A. Young,et al.  Understanding the recognition of facial identity and facial expression , 2005, Nature Reviews Neuroscience.

[38]  V. Bruce,et al.  Human and automatic face recognition: a comparison across image formats , 2001, Vision Research.

[39]  M. Riesenhuber,et al.  Evaluation of a Shape-Based Model of Human Face Discrimination Using fMRI and Behavioral Techniques , 2006, Neuron.

[40]  Denis Schluppeck,et al.  A comparison of fMRI adaptation and multivariate pattern classification analysis in visual cortex , 2010, NeuroImage.

[41]  D. Heeger,et al.  Linear Systems Analysis of Functional Magnetic Resonance Imaging in Human V1 , 1996, The Journal of Neuroscience.

[42]  Rainer Goebel,et al.  Combining multivariate voxel selection and support vector machines for mapping and classification of fMRI spatial patterns , 2008, NeuroImage.

[43]  Doris Y. Tsao,et al.  A face feature space in the macaque temporal lobe , 2009, Nature Neuroscience.

[44]  Markus Bindemann,et al.  The Role of Color in Human Face Detection , 2009, Cogn. Sci..

[45]  Karl J. Friston,et al.  Statistical parametric maps in functional imaging: A general linear approach , 1994 .

[46]  Doris Y. Tsao,et al.  Patches of face-selective cortex in the macaque frontal lobe , 2008, Nature Neuroscience.

[47]  Taketoshi Ono,et al.  Neural Correlates of Associative Face Memory in the Anterior Inferior Temporal Cortex of Monkeys , 2010, The Journal of Neuroscience.

[48]  N. Kanwisher Functional specificity in the human brain: A window into the functional architecture of the mind , 2010, Proceedings of the National Academy of Sciences.

[49]  J. Haxby,et al.  The distributed human neural system for face perception , 2000, Trends in Cognitive Sciences.

[50]  N. Kanwisher,et al.  The Fusiform Face Area: A Module in Human Extrastriate Cortex Specialized for Face Perception , 1997, The Journal of Neuroscience.

[51]  David D. Cox,et al.  Untangling invariant object recognition , 2007, Trends in Cognitive Sciences.

[52]  C. Gross,et al.  Neural representations of faces and body parts in macaque and human cortex: a comparative FMRI study. , 2009, Journal of neurophysiology.

[53]  S. Shevell The Science of Color , 2003 .

[54]  Thomas E. Nichols,et al.  Optimization of experimental design in fMRI: a general framework using a genetic algorithm , 2003, NeuroImage.

[55]  Francisco Pereira,et al.  Information mapping with pattern classifiers: A comparative study , 2011, NeuroImage.

[56]  A. Ishai,et al.  Effective connectivity within the distributed cortical network for face perception. , 2007, Cerebral cortex.

[57]  Johan Wagemans,et al.  Distributed subordinate specificity for bodies, faces, and buildings in human ventral visual cortex , 2010, NeuroImage.

[58]  Chih-Jen Lin,et al.  A comparison of methods for multiclass support vector machines , 2002, IEEE Trans. Neural Networks.

[59]  Karl J. Friston,et al.  Analysis of functional MRI time‐series , 1994, Human Brain Mapping.

[60]  Martin A. Lindquist,et al.  Modeling the hemodynamic response function in fMRI: Efficiency, bias and mis-modeling , 2009, NeuroImage.

[61]  Alice J. O'Toole,et al.  Dissociable Neural Patterns of Facial Identity across Changes in Viewpoint , 2010, Journal of Cognitive Neuroscience.

[62]  Yaroslav O. Halchenko,et al.  Brain Reading Using Full Brain Support Vector Machines for Object Recognition: There Is No Face Identification Area , 2008, Neural Computation.

[63]  Department of Electrical,et al.  Computational and Performance Aspects of PCA-Based Face-Recognition Algorithms , 2001, Perception.

[64]  Brittany S. Cassidy,et al.  Lower-Level Stimulus Features Strongly Influence Responses in the Fusiform Face Area , 2010, Cerebral cortex.

[65]  J. Stiles,et al.  Human Neuroscience , 2022 .

[66]  Kalanit Grill-Spector,et al.  Sparsely-distributed organization of face and limb activations in human ventral temporal cortex , 2010, NeuroImage.

[67]  Mark S. Seidenberg,et al.  Neural Systems Underlying the Recognition of Familiar and Newly Learned Faces , 2000, The Journal of Neuroscience.

[68]  Galia Avidan,et al.  Functional MRI Reveals Compromised Neural Integrity of the Face Processing Network in Congenital Prosopagnosia , 2009, Current Biology.

[69]  N. Kanwisher,et al.  The fusiform face area: a cortical region specialized for the perception of faces , 2006, Philosophical Transactions of the Royal Society B: Biological Sciences.

[70]  Thomas E. Nichols,et al.  Thresholding of Statistical Maps in Functional Neuroimaging Using the False Discovery Rate , 2002, NeuroImage.

[71]  Michael J. Tarr,et al.  Task-Specific Codes for Face Recognition: How they Shape the Neural Representation of Features for Detection and Individuation , 2008, PloS one.

[72]  Jiye G. Kim,et al.  Adaptation in the fusiform face area (FFA): Image or person? , 2009, Vision Research.

[73]  R. Adolphs,et al.  Neural systems behind word and concept retrieval , 2004, Cognition.

[74]  R. Tootell,et al.  An anterior temporal face patch in human cortex, predicted by macaque maps , 2009, Proceedings of the National Academy of Sciences.

[75]  Jason Weston,et al.  Gene Selection for Cancer Classification using Support Vector Machines , 2002, Machine Learning.

[76]  R. Goebel,et al.  Tracking cognitive processes with functional MRI mental chronometry , 2003, Current Opinion in Neurobiology.

[77]  Gilles Pourtois,et al.  View-independent coding of face identity in frontal and temporal cortices is modulated by familiarity: an event-related fMRI study , 2005, NeuroImage.

[78]  D H Brainard,et al.  The Psychophysics Toolbox. , 1997, Spatial vision.

[79]  Leslie G. Ungerleider,et al.  Perception of emotional expressions is independent of face selectivity in monkey inferior temporal cortex , 2008, Proceedings of the National Academy of Sciences.

[80]  Thad A Polk,et al.  Functional MRI evidence for an abstract, not perceptual, word-form area. , 2002, Journal of experimental psychology. General.

[81]  M. Seghier,et al.  A network of occipito-temporal face-sensitive areas besides the right middle fusiform gyrus is necessary for normal face processing. , 2003, Brain : a journal of neurology.

[82]  Rainer Goebel,et al.  Information-based functional brain mapping. , 2006, Proceedings of the National Academy of Sciences of the United States of America.

[83]  E. Thoma Nebraska Symposium on Motivation , 1963 .

[84]  Karl J. Friston,et al.  Dynamic discrimination analysis: A spatial–temporal SVM , 2007, NeuroImage.