Distinct brain signatures of content and structure violation during action observation

Sentences, musical phrases and goal-directed actions are composed of elements that are linked by specific rules to form meaningful outcomes. In goal-directed actions including a non-canonical element or scrambling the order of the elements alters the action's content and structure, respectively. In the present study we investigated event-related potentials of the electroencephalographic (EEG) activity recorded during observation of both alterations of the action content (obtained by violating the semantic components of an action, e.g. making coffee with cola) and alterations of the action structure (obtained by inverting the order of two temporally adjacent pictures of sequences depicting daily life actions) interfering with the normal flow of the motor acts that compose an action. Action content alterations elicited a bilateral posterior distributed EEG negativity, peaking at around 400 ms after stimulus onset similar to the ERPs evoked by semantic violations in language studies. Alteration of the action structure elicited an early left anterior negativity followed by a late left anterior positivity, which closely resembles the ERP pattern found in language syntax violation studies. Our results suggest a functional dissociation between the processing of action content and structure, reminiscent of a similar dissociation found in the language or music domains. Importantly, this study provides further support to the hypothesis that some basic mechanisms, such as the rule-based structuring of sequential events, are shared between different cognitive domains.

[1]  F. Pulvermüller The syntax of action , 2014, Trends in Cognitive Sciences.

[2]  M. Botvinick,et al.  Abstract Structural Representations of Goal-Directed Behavior , 2010, Psychological science.

[3]  G. Rizzolatti,et al.  Impairment of actions chains in autism and its possible role in intention understanding , 2007, Proceedings of the National Academy of Sciences.

[4]  A. Friederici,et al.  Musical syntax is processed in Broca's area: an MEG study , 2001, Nature Neuroscience.

[5]  M. van Elk,et al.  Semantics in action: An electrophysiological study on the use of semantic knowledge for action , 2008, Journal of Physiology-Paris.

[6]  W. Prinz,et al.  N400-like negativities in action perception reflect the activation of two components of an action representation , 2009, Social neuroscience.

[7]  Virginia Volterra,et al.  Early action and gesture "vocabulary" and its relation with word comprehension and production. , 2012, Child development.

[8]  S. Aglioti,et al.  Neural Underpinnings of Gesture Discrimination in Patients with Limb Apraxia , 2008, The Journal of Neuroscience.

[9]  C. Keysers,et al.  The Observation and Execution of Actions Share Motor and Somatosensory Voxels in all Tested Subjects: Single-Subject Analyses of Unsmoothed fMRI Data , 2008, Cerebral cortex.

[10]  L. Craighero,et al.  Broca's Area in Language, Action, and Music , 2009, Annals of the New York Academy of Sciences.

[11]  F. Manes,et al.  N400 ERPs for actions: building meaning in context , 2013, Front. Hum. Neurosci..

[12]  M. Balconi,et al.  Semantic violation effect on object-related action comprehension. N400-like event-related potentials for unusual and incorrect use , 2011, Neuroscience.

[13]  P. Greenfield,et al.  Language, tools and brain: The ontogeny and phylogeny of hierarchically organized sequential behavior , 1991, Behavioral and Brain Sciences.

[14]  J. Wolfe,et al.  Differential Electrophysiological Signatures of Semantic and Syntactic Scene Processing , 2013, Psychological science.

[15]  Michael F. Bonner,et al.  Hierarchical organization of scripts: converging evidence from FMRI and frontotemporal degeneration. , 2010, Cerebral cortex.

[16]  A. Friederici The brain basis of language processing: from structure to function. , 2011, Physiological reviews.

[17]  W. Fitch,et al.  Annals of the New York Academy of Sciences Hierarchical Processing in Music, Language, and Action: Lashley Revisited , 2022 .

[18]  J. Kraljević,et al.  Gestural development and its relation to a child's early vocabulary. , 2014, Infant behavior & development.

[19]  E. Kaan,et al.  Repair, Revision, and Complexity in Syntactic Analysis: An Electrophysiological Differentiation , 2003, Journal of Cognitive Neuroscience.

[20]  S. Martens,et al.  Hearing Feelings: Affective Categorization of Music and Speech in Alexithymia, an ERP Study , 2011, PloS one.

[21]  G. Dirnberger,et al.  The effects of alteration of effector and side of movement on movement-related cortical potentials , 2002, Clinical Neurophysiology.

[22]  Michela Balconi,et al.  N400 Effect When a Semantic Anomaly is Detected in Action Representation. A Source Localization Analysis , 2014, Journal of clinical neurophysiology : official publication of the American Electroencephalographic Society.

[23]  Sachiko Koyama,et al.  Event-Related Brain Potentials during Memory Search Task , 1993 .

[24]  Ellen F. Lau,et al.  A cortical network for semantics: (de)constructing the N400 , 2008, Nature Reviews Neuroscience.

[25]  Joseph H. Greenberg,et al.  Some Universals of Grammar with Particular Reference to the Order of Meaningful Elements , 1990, On Language.

[26]  Aniruddh D. Patel,et al.  Processing Syntactic Relations in Language and Music: An Event-Related Potential Study , 1998, Journal of Cognitive Neuroscience.

[27]  David Kemmerer,et al.  The Cross-Linguistic Prevalence of SOV and SVO Word Orders Reflects the Sequential and Hierarchical Representation of Action in Broca's Area , 2012, Lang. Linguistics Compass.

[28]  Kara D. Federmeier,et al.  Thirty years and counting: finding meaning in the N400 component of the event-related brain potential (ERP). , 2011, Annual review of psychology.

[29]  Anthony J. Rissling,et al.  Electroencephalography (EEG) and Event‐Related Potentials (ERPs) with Human Participants , 2010, Current protocols in neuroscience.

[30]  Ivan Toni,et al.  Comparable Mechanisms for Action and Language: Neural Systems Behind Intentions, Goals, and Means , 2006, Cortex.

[31]  Kara D. Federmeier,et al.  Electrophysiology reveals semantic memory use in language comprehension , 2000, Trends in Cognitive Sciences.

[32]  Blake W. Johnson,et al.  Comparison of the N300 and N400 ERPs to picture stimuli in congruent and incongruent contexts , 2002, Clinical Neurophysiology.

[33]  P. Holcomb,et al.  Event-related potentials during discourse-level semantic integration of complex pictures. , 2002, Brain research. Cognitive brain research.

[34]  R Core Team,et al.  R: A language and environment for statistical computing. , 2014 .

[35]  Jérôme Daltrozzo,et al.  Conceptual Processing in Music as Revealed by N400 Effects on Words and Musical Targets , 2009, Journal of Cognitive Neuroscience.

[36]  Laurel J Buxbaum,et al.  Critical brain regions for action recognition: lesion symptom mapping in left hemisphere stroke. , 2010, Brain : a journal of neurology.

[37]  A. Friederici,et al.  Electrophysiological Evidence for Two Steps in Syntactic Analysis: Early Automatic and Late Controlled Processes , 1999, Journal of Cognitive Neuroscience.

[38]  Comrie Bernard Language Universals and Linguistic Typology , 1982 .

[39]  S. Koelsch Toward a Neural Basis of Music Perception – A Review and Updated Model , 2011, Front. Psychology.

[40]  Stefan Koelsch,et al.  Shared neural resources between music and language indicate semantic processing of musical tension-resolution patterns. , 2008, Cerebral cortex.

[41]  Jody C. Culham,et al.  Author response: Decoding the neural mechanisms of human tool use , 2013 .

[42]  J. Mazziotta,et al.  Grasping the Intentions of Others with One's Own Mirror Neuron System , 2005, PLoS biology.

[43]  Andrea Moro,et al.  Response to Pulvermüller: the syntax of actions and other metaphors , 2014, Trends in Cognitive Sciences.

[44]  K. Bard,et al.  Gestures and social‐emotional communicative development in chimpanzee infants , 2014, American journal of primatology.

[45]  Alice Mado Proverbio,et al.  RP and N400 ERP components reflect semantic violations in visual processing of human actions , 2009, Neuroscience Letters.

[46]  M. Jeannerod The timing of natural prehension movements. , 1984, Journal of motor behavior.

[47]  Stefan Koelsch,et al.  Interaction between Syntax Processing in Language and in Music: An ERP Study , 2005, Journal of Cognitive Neuroscience.

[48]  Karsten Steinhauer,et al.  On the early left-anterior negativity (ELAN) in syntax studies , 2012, Brain and Language.

[49]  D H Brainard,et al.  The Psychophysics Toolbox. , 1997, Spatial vision.

[50]  R. C. Oldfield The assessment and analysis of handedness: the Edinburgh inventory. , 1971, Neuropsychologia.

[51]  Patric Bach,et al.  Communicating hands: ERPs elicited by meaningful symbolic hand postures , 2004, Neuroscience Letters.

[52]  G. Rizzolatti,et al.  Parietal Lobe: From Action Organization to Intention Understanding , 2005, Science.

[53]  M. Kutas,et al.  Reading senseless sentences: brain potentials reflect semantic incongruity. , 1980, Science.

[54]  Alice C. Roy,et al.  Encoding of human action in Broca's area. , 2009, Brain : a journal of neurology.

[55]  A. Friederici,et al.  Event-related brain potentials during natural speech processing: effects of semantic, morphological and syntactic violations. , 1993, Brain research. Cognitive brain research.

[56]  E. Koechlin,et al.  Broca's Area and the Hierarchical Organization of Human Behavior , 2006, Neuron.

[57]  Scott T. Grafton,et al.  Evidence for a distributed hierarchy of action representation in the brain. , 2007, Human movement science.

[58]  Jody C Culham,et al.  Decoding the neural mechanisms of human tool use , 2013, eLife.

[59]  Arnaud Delorme,et al.  EEGLAB: an open source toolbox for analysis of single-trial EEG dynamics including independent component analysis , 2004, Journal of Neuroscience Methods.

[60]  J. Mazziotta,et al.  Lateralization of the Human Mirror Neuron System , 2006, The Journal of Neuroscience.