Brain synchronization during perception of facial emotional expressions with natural and unnatural dynamics

Research on the perception of facial emotional expressions (FEEs) often uses static images that do not capture the dynamic character of social coordination in natural settings. Recent behavioral and neuroimaging studies suggest that dynamic FEEs (videos or morphs) enhance emotion perception. To identify mechanisms associated with the perception of FEEs with natural dynamics, the present EEG (Electroencephalography)study compared (i) ecologically valid stimuli of angry and happy FEEs with natural dynamics to (ii) FEEs with unnatural dynamics, and to (iii) static FEEs. FEEs with unnatural dynamics showed faces moving in a biologically possible but unpredictable and atypical manner, generally resulting in ambivalent emotional content. Participants were asked to explicitly recognize FEEs. Using whole power (WP) and phase synchrony (Phase Locking Index, PLI), we found that brain responses discriminated between natural and unnatural FEEs (both static and dynamic). Differences were primarily observed in the timing and brain topographies of delta and theta PLI and WP, and in alpha and beta WP. Our results support the view that biologically plausible, albeit atypical, FEEs are processed by the brain by different mechanisms than natural FEEs. We conclude that natural movement dynamics are essential for the perception of FEEs and the associated brain processes.

[1]  Béatrice de Gelder,et al.  Lateralization for dynamic facial expressions in human superior temporal sulcus , 2015, NeuroImage.

[2]  Karl J. Friston,et al.  Network Interactions Explain Sensitivity to Dynamic Faces in the Superior Temporal Sulcus , 2014, Cerebral cortex.

[3]  Richard Coppola,et al.  Cross-frequency power coupling between hierarchically organized face-selective areas. , 2014, Cerebral cortex.

[4]  David Kleinfeld,et al.  Spectral methods for functional brain imaging. , 2014, Cold Spring Harbor protocols.

[5]  Oliver G. B. Garrod,et al.  Dynamic Facial Expressions of Emotion Transmit an Evolving Hierarchy of Signals over Time , 2014, Current Biology.

[6]  Manfred Herrmann,et al.  The Perception of Dynamic and Static Facial Expressions of Happiness and Disgust Investigated by ERPs and fMRI Constrained Source Analysis , 2013, PloS one.

[7]  N. Cooper,et al.  Beta event-related desynchronization as an index of individual differences in processing human facial expression: further investigations of autistic traits in typically developing adults , 2013, Front. Hum. Neurosci..

[8]  Hame Park,et al.  Differential Beta-Band Event-Related Desynchronization during Categorical Action Sequence Planning , 2013, PloS one.

[9]  A. Manstead,et al.  Effects of Dynamic Aspects of Facial Expressions: A Review , 2013 .

[10]  H. Bülthoff,et al.  What the Human Brain Likes About Facial Motion , 2012, Cerebral cortex.

[11]  Ning Liu,et al.  Dynamic and Static Facial Expressions Decoded from Motion-Sensitive Areas in the Macaque Monkey , 2012, The Journal of Neuroscience.

[12]  H. Bülthoff,et al.  The MPI Facial Expression Database — A Validated Database of Emotional and Conversational Facial Expressions , 2012, PloS one.

[13]  I. Gorodnitsky,et al.  EEG mu component responses to viewing emotional faces , 2012, Behavioural Brain Research.

[14]  Guillermo Recio,et al.  Electrophysiological correlates of perceiving and evaluating static and dynamic facial emotional expressions , 2011, Brain Research.

[15]  Margot J. Taylor,et al.  Converging Evidence for the Advantage of Dynamic Facial Expressions , 2011, Brain Topography.

[16]  A. Leuthold,et al.  Beta-Band Activity during Motor Planning Reflects Response Uncertainty , 2010, The Journal of Neuroscience.

[17]  Karl J. Friston The free-energy principle: a unified brain theory? , 2010, Nature Reviews Neuroscience.

[18]  Karl J. Friston,et al.  Modulation of Perception and Brain Activity by Predictable Trajectories of Facial Expressions , 2009, Cerebral cortex.

[19]  C. Wallraven,et al.  Dynamic information for the recognition of conversational expressions. , 2009, Journal of vision.

[20]  W. Klimesch,et al.  Lifespan differences in cortical dynamics of auditory perception. , 2009, Developmental science.

[21]  P. McGuire,et al.  Functional atlas of emotional faces processing: a voxel-based meta-analysis of 105 functional magnetic resonance imaging studies. , 2009, Journal of psychiatry & neuroscience : JPN.

[22]  Roland Göcke,et al.  Learning AAM fitting through simulation , 2009, Pattern Recognition.

[23]  M. Herrmann,et al.  Emotions in motion: Dynamic compared to static facial expressions of disgust and happiness reveal more widespread emotion-specific activations , 2009, Brain Research.

[24]  R. Silberstein,et al.  Steady State Visually Evoked Potential Correlates of Static and Dynamic Emotional Face Processing , 2009, Brain Topography.

[25]  S. Bentin,et al.  Mirror activity in the human brain while observing hand movements: A comparison between EEG desynchronization in the μ-range and previous fMRI results , 2009, Brain Research.

[26]  J. Schultz,et al.  Natural facial motion enhances cortical responses to faces , 2009, Experimental Brain Research.

[27]  R. Adolphs,et al.  The social brain: neural basis of social knowledge. , 2009, Annual review of psychology.

[28]  P. Enticott,et al.  Mirror neuron activation is associated with facial emotion processing , 2008, Neuropsychologia.

[29]  Nadim Joni Shah,et al.  Gender differences in brain networks supporting empathy , 2008, NeuroImage.

[30]  W. Klimesch,et al.  What does phase information of oscillatory brain activity tell us about cognitive processes? , 2008, Neuroscience & Biobehavioral Reviews.

[31]  D. Heeger,et al.  A Hierarchy of Temporal Receptive Windows in Human Cortex , 2008, The Journal of Neuroscience.

[32]  W. Sato,et al.  Dynamic facial expressions of emotion induce representational momentum , 2008, Cognitive, affective & behavioral neuroscience.

[33]  A. Fuchs,et al.  Coordination: Neural, Behavioral and Social Dynamics , 2008 .

[34]  Michela Balconi,et al.  Event-Related Oscillations (ERO) and Event-Related Potentials (ERP) in Emotional Face Recognition , 2008, The International journal of neuroscience.

[35]  G. Pourtois,et al.  Distributed and interactive brain mechanisms during emotion face perception: Evidence from functional neuroimaging , 2007, Neuropsychologia.

[36]  Sakiko Yoshikawa,et al.  Enhanced Experience of Emotional Arousal in Response to Dynamic Facial Expressions , 2007 .

[37]  C. Frith The social brain? , 2007, Philosophical Transactions of the Royal Society B: Biological Sciences.

[38]  W. Sato,et al.  Spontaneous facial mimicry in response to dynamic facial expressions , 2005, Cognition.

[39]  M. Balconi,et al.  EEG correlates (event-related desynchronization) of emotional face elaboration: A temporal analysis , 2006, Neuroscience Letters.

[40]  A. Mizuno,et al.  A change of the leading player in flow Visualization technique , 2006, J. Vis..

[41]  J. Cohn,et al.  Deciphering the Enigmatic Face , 2005, Psychological science.

[42]  W. Klimesch,et al.  Alpha phase synchronization predicts P1 and N1 latency and amplitude size. , 2005, Cerebral cortex.

[43]  Anthony Randal McIntosh,et al.  Partial least squares analysis of neuroimaging data: applications and advances , 2004, NeuroImage.

[44]  Simon Baker,et al.  Active Appearance Models Revisited , 2004, International Journal of Computer Vision.

[45]  E. Naito,et al.  Enhanced neural activity in response to dynamic facial expressions of emotion: an fMRI study. , 2004, Brain research. Cognitive brain research.

[46]  N. Birbaumer,et al.  Dissociable cortical processing of recognizable and non-recognizable biological movement: analysing gamma MEG activity. , 2004, Cerebral cortex.

[47]  R. Adolphs,et al.  Dissociable neural systems for recognizing emotions , 2003, Brain and Cognition.

[48]  R. Adolphs Cognitive neuroscience: Cognitive neuroscience of human social behaviour , 2003, Nature Reviews Neuroscience.

[49]  Clinton D. Kilts,et al.  Dissociable Neural Pathways Are Involved in the Recognition of Emotion in Static and Dynamic Facial Expressions , 2003, NeuroImage.

[50]  R. Adolphs Recognizing emotion from facial expressions: psychological and neurological mechanisms. , 2002, Behavioral and cognitive neuroscience reviews.

[51]  N. Birbaumer,et al.  Event-related beta desynchronization indicates timing of response selection in a delayed-response paradigm in humans , 2001, Neuroscience Letters.

[52]  J. Martinerie,et al.  The brainweb: Phase synchronization and large-scale integration , 2001, Nature Reviews Neuroscience.

[53]  J. Haxby,et al.  The distributed human neural system for face perception , 2000, Trends in Cognitive Sciences.

[54]  Erkki Oja,et al.  Independent component analysis: algorithms and applications , 2000, Neural Networks.

[55]  T. Sejnowski,et al.  Removing electroencephalographic artifacts by blind source separation. , 2000, Psychophysiology.

[56]  F. L. D. Silva,et al.  Event-related EEG/MEG synchronization and desynchronization: basic principles , 1999, Clinical Neurophysiology.

[57]  W. Klimesch EEG alpha and theta oscillations reflect cognitive and memory performance: a review and analysis , 1999, Brain Research Reviews.

[58]  Timothy F. Cootes,et al.  Active Appearance Models , 1998, ECCV.

[59]  S Makeig,et al.  Blind separation of auditory event-related brain responses into independent components. , 1997, Proceedings of the National Academy of Sciences of the United States of America.

[60]  J. V. Haxby,et al.  Spatial Pattern Analysis of Functional Brain Images Using Partial Least Squares , 1996, NeuroImage.

[61]  H. Maturana,et al.  The Tree of Knowledge: The Biological Roots of Human Understanding , 2007 .

[62]  D. J. Felleman,et al.  Distributed hierarchical processing in the primate cerebral cortex. , 1991, Cerebral cortex.

[63]  J. Tukey,et al.  Variations of Box Plots , 1978 .

[64]  P. Ekman Pictures of Facial Affect , 1976 .