An objective signature for visual binding of face parts in the human brain.

Whether and how the parts of a visual object are grouped together to form an integrated ("holistic") representation is a central question in cognitive neuroscience. Although the face is considered to be the quintessential example of holistic representation, this issue has been the subject of much debate in face perception research. The implication of holistic processing is that the response to the whole cannot be predicted from the sum of responses to the parts. Here we apply techniques from nonlinear systems analysis to provide an objective measure of the nonlinear integration of parts into a whole, using the left and right halves of a face stimulus as the parts. High-density electroencephalogram (EEG) was recorded in 15 human participants presented with two halves of a face stimulus, flickering at different frequencies (5.88 vs. 7.14 Hz). Besides specific responses at these fundamental frequencies, reflecting part-based responses, we found intermodulation components (e.g., 7.14 - 5.88 = 1.26 Hz) over the right occipito-temporal hemisphere, reflecting nonlinear integration of the face halves. Part-based responses did not depend on the relative alignment of the two face halves, their spatial separation, or whether the face was presented upright or inverted. By contrast, intermodulations were virtually absent when the two halves were spatially misaligned and separated. Inversion of the whole face configuration also reduced specifically the intermodulation components over the right occipito-temporal cortex. These observations indicate that the intermodulation components constitute an objective, configuration-specific signature of an emergent neural representation of the whole face that is distinct from that generated by the parts themselves.

[1]  M. Wertheimer Untersuchungen zur Lehre von der Gestalt. II , 1923 .

[2]  H. Hécaen,et al.  Agnosia for faces (prosopagnosia). , 1962, Archives of neurology.

[3]  D. Regan Some characteristics of average steady-state and transient responses evoked by modulated light. , 1966, Electroencephalography and clinical neurophysiology.

[4]  D Regan,et al.  Clinical investigation of lesions of the visual pathway: a new objective technique. , 1969, Journal of neurology, neurosurgery, and psychiatry.

[5]  R. Yin Looking at Upside-down Faces , 1969 .

[6]  D. Regan,et al.  A method of measuring the potentials evoked by simultaneous stimulation of different retinal regions. , 1970, Electroencephalography and clinical neurophysiology.

[7]  D. B. Bender,et al.  Visual properties of neurons in inferotemporal cortex of the Macaque. , 1972, Journal of neurophysiology.

[8]  H B Barlow,et al.  Single units and sensation: a neuron doctrine for perceptual psychology? , 1972, Perception.

[9]  P Bakan,et al.  Visual asymmetry in perception of faces. , 1973, Neuropsychologia.

[10]  B Milner,et al.  Perception of faces by patients with localized cortical excisions. , 1983, Canadian journal of psychology.

[11]  R. Desimone,et al.  Stimulus-selective properties of inferior temporal neurons in the macaque , 1984, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[12]  J. Sergent An investigation into component and configural processes underlying face perception. , 1984, British journal of psychology.

[13]  L. Kaufman,et al.  Handbook of perception and human performance , 1986 .

[14]  A. Young,et al.  Configurational Information in Face Perception , 1987, Perception.

[15]  D. Levi,et al.  Evidence for nonlinear binocular interactions in human visual cortex , 1988, Vision Research.

[16]  David Regan,et al.  A frequency domain technique for characterizing nonlinearities in biological systems , 1988 .

[17]  D. Regan Human brain electrophysiology: Evoked potentials and evoked magnetic fields in science and medicine , 1989 .

[18]  O. Koenig,et al.  Separable Mechanisms in Face Processing: Evidence from Hemispheric Specialization , 1991, Journal of Cognitive Neuroscience.

[19]  Minami Ito,et al.  Columns for visual features of objects in monkey inferotemporal cortex , 1992, Nature.

[20]  J. Sergent,et al.  Functional neuroanatomy of face and object processing. A positron emission tomography study. , 1992, Brain : a journal of neurology.

[21]  D Mumford,et al.  On the computational architecture of the neocortex. II. The role of cortico-cortical loops. , 1992, Biological cybernetics.

[22]  M. Farah,et al.  Parts and Wholes in Face Recognition , 1993, The Quarterly journal of experimental psychology. A, Human experimental psychology.

[23]  G. Rhodes,et al.  What's lost in inverted faces? , 1993, Cognition.

[24]  Keiji Tanaka,et al.  Neuronal selectivities to complex object features in the ventral visual pathway of the macaque cerebral cortex. , 1994, Journal of neurophysiology.

[25]  W Singer,et al.  Visual feature integration and the temporal correlation hypothesis. , 1995, Annual review of neuroscience.

[26]  T. Allison,et al.  Electrophysiological Studies of Face Perception in Humans , 1996, Journal of Cognitive Neuroscience.

[27]  S. Hillyard,et al.  Selective attention to stimulus location modulates the steady-state visual evoked potential. , 1996, Proceedings of the National Academy of Sciences of the United States of America.

[28]  L. Spillmann,et al.  From Neuron to Gestalt: Mechanisms of Visual Perception , 1996 .

[29]  Keiji Tanaka,et al.  Inferotemporal cortex and object vision. , 1996, Annual review of neuroscience.

[30]  I Biederman,et al.  Neurocomputational bases of object and face recognition. , 1997, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[31]  T. Allison,et al.  Electrophysiological studies of human face perception. I: Potentials generated in occipitotemporal cortex by face and non-face stimuli. , 1999, Cerebral cortex.

[32]  O. Bertrand,et al.  Oscillatory gamma activity in humans and its role in object representation , 1999, Trends in Cognitive Sciences.

[33]  A. Norcia,et al.  Electrophysiological correlates of vernier and relative motion mechanisms in human visual cortex , 1999, Visual Neuroscience.

[34]  D. P. Russell,et al.  Increased Synchronization of Neuromagnetic Responses during Conscious Perception , 1999, The Journal of Neuroscience.

[35]  J D Victor,et al.  Two-frequency analysis of interactions elicited by Vernier stimuli , 2000, Visual Neuroscience.

[36]  A. Norcia,et al.  Changes in cortical activity during suppression in stereoblindness , 2000, Neuroreport.

[37]  J. Haxby,et al.  The distributed human neural system for face perception , 2000, Trends in Cognitive Sciences.

[38]  T. Hendler,et al.  A hierarchical axis of object processing stages in the human visual cortex. , 2001, Cerebral cortex.

[39]  Y. Yamane,et al.  Complex objects are represented in macaque inferotemporal cortex by the combination of feature columns , 2001, Nature Neuroscience.

[40]  S. Hochstein,et al.  View from the Top Hierarchies and Reverse Hierarchies in the Visual System , 2002, Neuron.

[41]  M. Behrmann,et al.  Impact of learning on representation of parts and wholes in monkey inferotemporal cortex , 2002, Nature Neuroscience.

[42]  J. R. Pomerantz,et al.  Contour grouping inside and outside of facial contexts. , 2003, Acta psychologica.

[43]  Floyd Ratliff,et al.  Intermodulation components of the visual evoked potential: Responses to lateral and superimposed stimuli , 2004, Biological Cybernetics.

[44]  D. Mumford On the computational architecture of the neocortex , 2004, Biological Cybernetics.

[45]  D. Perrett,et al.  Visual neurones responsive to faces in the monkey temporal cortex , 2004, Experimental Brain Research.

[46]  L. Chalupa,et al.  The visual neurosciences , 2004 .

[47]  Charles E Connor,et al.  Underlying principles of visual shape selectivity in posterior inferotemporal cortex , 2004, Nature Neuroscience.

[48]  N. Newman The Visual Neurosciences , 2005 .

[49]  Doris Y. Tsao,et al.  A Cortical Region Consisting Entirely of Face-Selective Cells , 2006, Science.

[50]  Alex R. Wade,et al.  Cue-Invariant Networks for Figure and Background Processing in Human Visual Cortex , 2006, The Journal of Neuroscience.

[51]  Bruno Rossion,et al.  Faces are represented holistically in the human occipito-temporal cortex , 2006, NeuroImage.

[52]  Manabu Tanifuji,et al.  Representation of the spatial relationship among object parts by neurons in macaque inferotemporal cortex. , 2006, Journal of neurophysiology.

[53]  Marie L. Smith,et al.  From a face to its category via a few information processing states in the brain , 2007, NeuroImage.

[54]  Alison Harris,et al.  The Representation of Parts and Wholes in Face-selective Cortex , 2008, Journal of Cognitive Neuroscience.

[55]  B. Rossion Picture-plane inversion leads to qualitative changes of face perception. , 2008, Acta psychologica.

[56]  Søren K. Andersen,et al.  Attentional bias of competitive interactions in neuronal networks of early visual processing in the human brain , 2008, NeuroImage.

[57]  J. Barton,et al.  The effects of face inversion on the perception of long-range and local spatial relations in eye and mouth configuration. , 2008, Journal of experimental psychology. Human perception and performance.

[58]  Alex R. Wade,et al.  Figure-ground interaction in the human visual cortex. , 2008, Journal of vision.

[59]  A Mouraux,et al.  Across-trial averaging of event-related EEG responses and beyond. , 2008, Magnetic resonance imaging.

[60]  C. Jacques,et al.  The initial representation of individual faces in the right occipito-temporal cortex is holistic: electrophysiological evidence from the composite face illusion. , 2009, Journal of vision.

[61]  H. Barlow,et al.  Single Units and Sensation: A Neuron Doctrine for Perceptual Psychology? , 1972, Perception.

[62]  Matthias M. Müller,et al.  Color-selective attention need not be mediated by spatial attention. , 2009, Journal of vision.

[63]  Doris Y. Tsao,et al.  A face feature space in the macaque temporal lobe , 2009, Nature Neuroscience.

[64]  R. Srinivasan,et al.  Nonlinear SSVEP responses are sensitive to the perceptual binding of visual hemifields during conventional ‘eye’ rivalry and interocular ‘percept’ rivalry , 2009, Brain Research.

[65]  Carl R Olson,et al.  Responses to Compound Objects in Monkey Inferotemporal Cortex: The Whole Is Equal to the Sum of the Discrete Parts , 2010, The Journal of Neuroscience.

[66]  K. Verfaillie,et al.  Face inversion impairs holistic perception: evidence from gaze-contingent stimulation. , 2010, Journal of vision.

[67]  R. Goebel,et al.  Holistic perception of individual faces in the right middle fusiform gyrus as evidenced by the composite face illusion. , 2010, Journal of vision.

[68]  Timothy J. Andrews,et al.  Internal and External Features of the Face Are Represented Holistically in Face-Selective Regions of Visual Cortex , 2010, The Journal of Neuroscience.

[69]  J. Lorenceau,et al.  Magnetoencephalographic signatures of visual form and motion binding , 2011, Brain Research.

[70]  James R Pomerantz,et al.  Grouping and emergent features in vision: toward a theory of basic Gestalts. , 2011, Journal of experimental psychology. Human perception and performance.

[71]  R. Goebel,et al.  Holistic Face Categorization in Higher Order Visual Areas of the Normal and Prosopagnosic Brain: Toward a Non-Hierarchical View of Face Perception , 2011, Front. Hum. Neurosci..

[72]  Leslie G. Ungerleider,et al.  Uncovering the visual “alphabet”: Advances in our understanding of object perception , 2011, Vision Research.

[73]  C. Jacques,et al.  The N170 : understanding the time-course of face perception in the human brain , 2011 .

[74]  Johan Wagemans,et al.  Emergence of Perceptual Gestalts in the Human Visual Cortex , 2011, Psychological science.

[75]  B. Rossion,et al.  Robust sensitivity to facial identity in the right human occipito-temporal cortex as revealed by steady-state visual-evoked potentials. , 2011, Journal of vision.

[76]  Alex R. Wade,et al.  Dynamics of Normalization Underlying Masking in Human Visual Cortex , 2012, The Journal of Neuroscience.

[77]  S. Palmer,et al.  A century of Gestalt psychology in visual perception: I. Perceptual grouping and figure-ground organization. , 2012, Psychological bulletin.

[78]  Hubert Preissl,et al.  Steady-state responses in MEG demonstrate information integration within but not across the auditory and visual senses , 2011, NeuroImage.

[79]  Bruno Rossion,et al.  An objective method for measuring face detection thresholds using the sweep steady-state visual evoked response. , 2012, Journal of vision.

[80]  B. Tjan,et al.  The Perception of a Face Is No More Than the Sum of Its Parts , 2012, Psychological science.

[81]  Bruno Rossion,et al.  A steady-state visual evoked potential approach to individual face perception: Effect of inversion, contrast-reversal and temporal dynamics , 2012, NeuroImage.

[82]  Jason M. Gold,et al.  The response of face-selective cortex with single face parts and part combinations , 2012, Neuropsychologia.

[83]  Matthias M. Müller,et al.  Perception of illusory contours forms intermodulation responses of steady state visual evoked potentials as a neural signature of spatial integration , 2013, Biological Psychology.

[84]  B. Rossion The composite face illusion: A whole window into our understanding of holistic face perception , 2013 .

[85]  S. Caharel,et al.  Early holistic face-like processing of Arcimboldo paintings in the right occipito-temporal cortex: evidence from the N170 ERP component. , 2013, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[86]  R. Goebel,et al.  Local Discriminability Determines the Strength of Holistic Processing for Faces in the Fusiform Face Area , 2013, Front. Psychology.

[87]  A. Norcia,et al.  The 6Hz fundamental stimulation frequency rate for individual face discrimination in the right occipito-temporal cortex , 2013, Neuropsychologia.

[88]  K. Grill-Spector,et al.  Neural representations of faces and limbs neighbor in human high-level visual cortex: evidence for a new organization principle , 2011, Psychological Research.