The role of prosody in language comprehension: when prosodic breaks and pitch accents come into play

This study addresses the question whether prosodic information can affect the choice for a syntactic analysis in auditory sentence processing. We manipulated the prosody (in the form of a prosodic break (PB)) of locally ambiguous Dutch sentences to favor one of two interpretations. The experimental items contained two different types of so-called control verbs (subjectand object-control) in the matrix clause and were syntactically disambiguated by a transitive or intransitive verb. In Experiment 1 we established the default off-line preference of the items for a transitive or an intransitive disambiguating verb with a visual and auditory fragment completion test. The results suggested that subjectand object-control verbs differently affect the syntactic structure that listeners expect. In Experiment 2 we investigated these two types of verbs separately in an on-line ERP study. Consistent with the literature, the PB elicited a Closure Positive Shift (CPS). Furthermore, in subject-control items, an N400 effect for intransitive relative to transitive disambiguating verbs was found, both for sentences with and for sentences without a PB. This result suggests that the default preference for subject-control verbs goes in the same direction as the effect of the PB. In object-control items, an N400 effect for intransitive relative to transitive disambiguating verbs was found for sentences with a PB, but no effect in the absence of a PB. This indicates that a PB can affect the syntactic analysis that listeners pursue. This chapter has been published as: Bogels, S., Schriefers, H., Vonk, W., Chwilla, D. J., & Kerkhofs, R. (2010). The interplay between prosody and syntax in sentence processing: The case of subjectand object-control verbs. Journal o f Cognitive Neuroscience, 22, 1036-1053.

[1]  A. Friederici,et al.  Late interaction of syntactic and prosodic processes in sentence comprehension as revealed by ERPs. , 2005, Brain research. Cognitive brain research.

[2]  Ulrike Toepel,et al.  Catching the news: Processing strategies in listening to dialogs as measured by ERPs , 2007, Behavioral and Brain Functions.

[3]  Herbert Schriefers,et al.  Sentence processing in the visual and auditory modality: Do comma and prosodic break have parallel functions? , 2008, Brain Research.

[4]  H. V. Dijk,et al.  The state of the brain: How alpha oscillations shape behavior and event-related responses , 2010 .

[5]  M. Kutas,et al.  Brain potentials during reading reflect word expectancy and semantic association , 1984, Nature.

[6]  Magnus Lindgren,et al.  Left-edge boundary tone and main clause verb effects on syntactic processing in embedded clauses – An ERP study , 2009, Journal of Neurolinguistics.

[7]  W. van Donselaar,et al.  The Function of Sentence Accents and Given/New Information in Speech Processing: Different Strategies for Normal-Hearing and Hearing-Impaired Listeners? , 1994, Language and speech.

[8]  J. Trueswell,et al.  Using prosody to avoid ambiguity: Effects of speaker awareness and referential context , 2003 .

[9]  C. Clifton,et al.  Prosodic Boundaries in Adjunct Attachment , 2001 .

[10]  A J Schafer,et al.  Intonational Disambiguation in Sentence Production and Comprehension , 2000, Journal of psycholinguistic research.

[11]  N. Timmer,et al.  The interaction of heparan sulfate proteoglycans with the amyloid β protein. , 2011 .

[12]  E. Grabe,et al.  Prosody, phonology and parsing in closure ambiguities , 1995 .

[13]  H. V. Beek,et al.  The regulation of cerebral perfusion in patients with Alzheimer's disease , 2006 .

[14]  M. J. Grol Parieto-frontal circuitry in visuomotor control , 2008 .

[15]  Peter Hagoort,et al.  The Neural Integration of Speaker and Message , 2008, Journal of Cognitive Neuroscience.

[16]  H. Kolk,et al.  Structure and limited capacity in verbal working memory: A study with event-related potentials , 2003, Brain and Language.

[17]  M. Kjelgaard,et al.  Prosodic Facilitation and Interference in the Resolution of Temporary Syntactic Closure Ambiguity , 1999 .

[18]  A. Friederici The Time Course of Syntactic Activation During Language Processing: A Model Based on Neuropsychological and Neurophysiological Data , 1995, Brain and Language.

[19]  S. van Pelt,et al.  Dynamic neural representations of human visuomotor space , 2003 .

[20]  N. Derks,et al.  The role of the non-preganglionic Edinger-Westphal nucleus in sex-dependent stress adaptation in rodents , 2010 .

[21]  Claudia Männel,et al.  Pauses and Intonational Phrasing: ERP Studies in 5-month-old German Infants and Adults , 2009, Journal of Cognitive Neuroscience.

[22]  J. Lommertzen,et al.  Visuomotor Coupling at Different Levels of Complexity , 2004 .

[23]  Angela D. Friederici,et al.  Effects of Musical Expertise and Boundary Markers on Phrase Perception in Music , 2006, Journal of Cognitive Neuroscience.

[24]  Mark Steedman,et al.  Interaction with context during human sentence processing , 1988, Cognition.

[25]  Anne Cutler,et al.  Introduction. Models and Measurements in the Study of Prosody , 1983 .

[26]  Angela D. Friederici,et al.  Non-musicians’ perception of phrase boundaries in music: A cross-cultural ERP study , 2009, Biological Psychology.

[27]  D. Swinney,et al.  Brain potentials elicited by garden-path sentences: evidence of the application of verb information during parsing. , 1994, Journal of experimental psychology. Learning, memory, and cognition.

[28]  A. Wagman Event-related Brain Potentials in Man , 1981 .

[29]  C. Clifton,et al.  Prosodic phrasing is central to language comprehension , 2006, Trends in Cognitive Sciences.

[30]  A. Friederici,et al.  Word category and verb–argument structure information in the dynamics of parsing , 2004, Cognition.

[31]  Peter Hagoort,et al.  Brain responses to lexical ambiguity resolution and parsing. , 1994 .

[32]  Charles Clifton,et al.  Effects of varying focus and accenting of adjuncts on the comprehension of utterances , 2002 .

[33]  Martijn J. M. Lamers,et al.  Levels of selective attention in action planning , 2004 .

[34]  H. J. Schriefers,et al.  Word order and verb inflection in agrammatic sentence production , 2001 .

[35]  Herbert Schriefers,et al.  The Influence of Animacy on Relative Clause Processing , 2002 .

[36]  R T Knight,et al.  Event-related potentials differentiate the effects of aging on word and nonword repetition in explicit and implicit memory tasks. , 1997, Journal of experimental psychology. Learning, memory, and cognition.

[37]  C. Hooijmans,et al.  Impact of nutritional lipids and vascular factors in Alzheimer's Disease - a study of transgenic Alzheimer mice - , 2008 .

[38]  J. Majdandzic Cerebral mechanisms of processing action goals in self and others , 2003 .

[39]  M. Wyczesany,et al.  Covariation of mood and brain activity: Integration of subjective selfreport data with quantitative EEG measures , 2010 .

[40]  Esther Aarts,et al.  Resisting temptation: the role of the anterior cingulate cortex in adjusting cognitive control , 2004 .

[41]  A D Friederici,et al.  Prosodic Boundaries, Comma Rules, and Brain Responses: The Closure Positive Shift in ERPs as a Universal Marker for Prosodic Phrasing in Listeners and Readers , 2001, Journal of psycholinguistic research.

[42]  Anita Steube,et al.  Misleading dialogues: Human's brain reaction to prosodic information , 2001 .

[43]  Stefan Heim,et al.  Prosodic pitch accents in language comprehension and production: ERP data and acoustic analyses. , 2006, Acta neurobiologiae experimentalis.

[44]  B. Lapatki,et al.  The facial musculature. Characterisation at a motor unit level. , 2010 .

[45]  Stefan Heim,et al.  Focus on focus: The brain’s electrophysiological response to focus particles and accents in German , 2007 .

[46]  Kai Alter,et al.  Prosody in Dialogues and Single Sentences: How Prosody can influence Speech Perception , 2004 .

[47]  Angela D. Friederici,et al.  Processing focus structure and implicit prosody during reading: Differential ERP effects , 2007, Cognition.

[48]  Leo G. M. Noordman,et al.  Reference Management in Instructive Discourse , 2004, Discourse Processes.

[49]  M. Tanenhaus,et al.  Modeling the Influence of Thematic Fit (and Other Constraints) in On-line Sentence Comprehension , 1998 .

[50]  A. Friederici,et al.  Verb Argument Structure Processing: The Role of Verb-Specific and Argument-Specific Information☆☆☆ , 2000 .

[51]  Giosuè Baggio,et al.  Semantics and the electrophysiology of meaning: Tense, aspect, event structure , 2004 .

[52]  I. Nieuwenhuis,et al.  Memory consolidation. A process of integration converging evidence from MEG, fMRI and behavior. , 2010 .

[53]  Herbert Schriefers,et al.  Discourse, Syntax, and Prosody: The Brain Reveals an Immediate Interaction , 2007, Journal of Cognitive Neuroscience.

[54]  A. Snik,et al.  On the role of eye and head position in spatial localisation behaviour , 2011 .

[55]  E. Donchin,et al.  Is the P300 component a manifestation of context updating? , 1988, Behavioral and Brain Sciences.

[56]  Bernard Comrie,et al.  REFLECTIONS ON SUBJECT AND OBJECT CONTROL , 1985 .

[57]  K. Forster,et al.  Event-related brain potential examination of implicit memory processes: masked and unmasked repetition priming. , 1997, Neuropsychology.

[58]  Zhixing Jin,et al.  The pragmatic meanings conveyed by function words in Chinese sentences: An ERP study , 2009, Journal of Neurolinguistics.

[59]  Janet D. Fodor,et al.  The sausage machine: A new two-stage parsing model , 1978, Cognition.

[60]  Dorothee J. Chwilla,et al.  Monitoring in Language Perception , 2009, Lang. Linguistics Compass.

[61]  S. Overeem,et al.  Modulation of human balance reactions. , 2010 .

[62]  Matthias Schlesewsky,et al.  The extended argument dependency model: a neurocognitive approach to sentence comprehension across languages. , 2006, Psychological review.

[63]  M. D. Vrijer Multisensory integration in spatial orientation , 2006 .

[64]  S. I. van Aalderen-Smeets Neural dynamics of visual selection , 2007 .

[65]  Carlos Gussenhoven,et al.  Prosody and information structure: An fMRI study , 2006 .

[66]  J. van der Werf,et al.  Cortical oscillatory activity in human visuomotor integration , 2011 .

[67]  Jonathan Levy,et al.  In cerebro unveiling unconscious mechanisms during reading , 2010 .

[68]  Lesley Stirling,et al.  Does Prosody Support or Direct Sentence Processing , 1996 .

[69]  M. Bauer,et al.  Functional roles of rhythmic neuronal activity in the human visual and somatosensory system , 2008 .

[70]  C. Clifton,et al.  Focus in Relative Clause Construal , 1996 .

[71]  René Scheeringa,et al.  On the relation between oscillatory EEG activity and the BOLD signal , 2011 .

[72]  M. Bakker,et al.  Supraspinal control of walking : lessons from motor imagery , 2001 .

[73]  M. Posner,et al.  Orienting of Attention* , 1980, The Quarterly journal of experimental psychology.

[74]  Cinzia Avesani,et al.  Prosodic marking of information status in Dutch and Italian: a comparative analysis , 2002, J. Phonetics.

[75]  H. H. Clark,et al.  Changing Ideas about Reference , 2004 .

[76]  L. Menenti,et al.  The right language : differential hemispheric contributions to language production and comprehension in context , 2010 .

[77]  J. Knippenberg,et al.  The N150 of the Auditory Evoked Potential from the rat amygdala: In search for its functional significance , 2009 .

[78]  K. Grootens,et al.  Cognitive dysfunction and effects of antipsychotics in schizophrenia and borderline personality disorder. , 2010 .

[79]  J. Pijnacker,et al.  Defeasible inference in autism: a behavioral and electrophysiological approach , 2010 .

[80]  Angela D Friederici,et al.  Prosodic processing at the sentence level in infants , 2006, Neuroreport.

[81]  Carlos Gussenhoven,et al.  Transcription of Dutch intonation , 2005 .

[82]  Christopher T. Kello,et al.  Verb-specific constraints in sentence processing: separating effects of lexical preference from garden-paths. , 1993, Journal of experimental psychology. Learning, memory, and cognition.

[83]  Joël Pynte,et al.  Prosodic Breaks and Attachment Decisions in Sentence Parsing. , 1996 .

[84]  Cheryl M. Beach,et al.  The interpretation of prosodic patterns at points of syntactic structure ambiguity: Evidence for cue trading relations☆ , 1991 .

[85]  M. Kutas,et al.  Who Did What and When? Using Word- and Clause-Level ERPs to Monitor Working Memory Usage in Reading , 1995, Journal of Cognitive Neuroscience.

[86]  J. Kleijnen,et al.  Context Effects , 2003 .

[87]  S. M. Beurze Cortical mechanisms for reach planning , 2004 .

[88]  Kiwako Ito,et al.  Anticipatory effects of intonation: Eye movements during instructed visual search. , 2008, Journal of memory and language.

[89]  M. Tanenhaus,et al.  Accent and reference resolution in spoken-language comprehension , 2002 .

[90]  Andrea Weber,et al.  Finding Referents in Time: Eye-Tracking Evidence for the Role of Contrastive Accents , 2006, Language and speech.

[91]  Angela D. Friederici,et al.  It's Early: Event-related Potential Evidence for Initial Interaction of Syntax and Prosody in Speech Comprehension , 2006, Journal of Cognitive Neuroscience.

[92]  M. van Elk,et al.  Action semantics functional and neural dynamics , 2010 .

[93]  Shaozheng Qin,et al.  Adaptive memory. Imaging medial temporal and prefrontal memory systems , 2010 .

[94]  Mireille Besson,et al.  Brain potentials during semantic and prosodic processing in French. , 2004, Brain research. Cognitive brain research.

[95]  Ulrike Toepel,et al.  Inadequate and infrequent are not alike: ERPs to deviant prosodic patterns in spoken sentence comprehension , 2008, Brain and Language.

[96]  G. A. van Wingen,et al.  Biological determinants of amygdala functioning. , 2009 .

[97]  Angela D. Friederici,et al.  Influence of Prosodic Information on the Processing of Split Particles: ERP Evidence from Spoken German , 2005, Journal of Cognitive Neuroscience.

[98]  Olga Meulenbroek,et al.  Neural correlates of episodic memory in healthy aging and Alzheimer's disease. , 2010 .

[99]  Matthias S. Treder,et al.  Symmetry in (inter)action , 2005 .

[100]  Roel M. Willems,et al.  Neural reflections of meaning in gesture, language, and action , 2009 .

[101]  Sieb G. Nooteboom,et al.  Opposite effects of accentuation and deaccentuation on verification latencies for given and new information , 1987 .

[102]  Matthew W. Crocker,et al.  Ambiguity Resolution in Sentence Processing: Evidence against Frequency-Based Accounts , 2000 .

[103]  Angela D. Friederici,et al.  Prosody-driven Sentence Processing: An Event-related Brain Potential Study , 2005, Journal of Cognitive Neuroscience.

[104]  Harold Bekkering,et al.  Dynamics of cognitive control in task switching: looking beyond the switch cost , 2001 .

[105]  Rolf Schwitter,et al.  Processing Coordinated Structures in PENG Light , 2011, Australasian Conference on Artificial Intelligence.

[106]  C. Crajé,et al.  A)typical motor planning and motor imagery , 2006 .

[107]  R. Ratcliff,et al.  Reliability of prosodic cues for resolving syntactic ambiguity. , 1996, Journal of experimental psychology. Learning, memory, and cognition.

[108]  C. Horlings,et al.  A weak balance; Balance and falls in patients with neuromuscular disorders. , 2010 .

[109]  J. M. Schoffelen,et al.  Neuronal communication through coherence in the human motor system , 2007 .

[110]  B. Poser,et al.  Techniques for BOLD and blood volume weighted fMRI , 2009 .

[111]  K. Rayner,et al.  Making and correcting errors during sentence comprehension: Eye movements in the analysis of structurally ambiguous sentences , 1982, Cognitive Psychology.

[112]  H. Neville,et al.  Language and , 2019, Adventure Diffusion.

[113]  Gina R. Kuperberg,et al.  Neural mechanisms of language comprehension: Challenges to syntax , 2007, Brain Research.

[114]  H. Kolk,et al.  Mediated Priming in the Lexical Decision Task: Evidence from Event-Related Potentials and Reaction Time , 2000 .

[115]  Karl G. D. Bailey,et al.  Good-Enough Representations in Language Comprehension , 2002 .

[116]  Angela D. Friederici,et al.  Brain potentials indicate immediate use of prosodic cues in natural speech processing , 1999, Nature Neuroscience.

[117]  G. Dumont Cognitive and physiological effects of 3,4-methylenedioxymethamphetamine (MDMA or 'ecstasy') in combination with alcohol or cannabis in humans , 2009 .

[118]  M. Rugg,et al.  Lexical contribution to nonword-repetition effects: Evidence from event-related potentials , 1987, Memory & cognition.

[119]  Anne Lacheret,et al.  On-line Processing of Pop-Out Words in Spoken French Dialogues , 2005, Journal of Cognitive Neuroscience.

[120]  L. Nygaard,et al.  Resolution of lexical ambiguity by emotional tone of voice , 2002 .

[121]  O. Witte,et al.  Perception of phrase structure in music , 2005, Human brain mapping.

[122]  J. Thayer,et al.  The continuing problem of false positives in repeated measures ANOVA in psychophysiology: a multivariate solution. , 1987, Psychophysiology.

[123]  Tineke M. Snijders,et al.  More than words: neural and genetic dynamics of syntactic unification , 2004 .

[124]  M. Kutas,et al.  Psycholinguistics Electrified II (1994–2005) , 2006 .

[125]  L. Streeter Acoustic determinants of phrase boundary perception. , 1978, The Journal of the Acoustical Society of America.

[126]  William D. Marslen-Wilson,et al.  THE QUARTERLY JOURNAL OF EXPERIMENTAL PSYCHOLOGY, 1992,45A (1) 73-87 Prosodic Effects in Minimal Attachment , 2022 .

[127]  Susan M. Garnsey,et al.  Semantic Influences On Parsing: Use of Thematic Role Information in Syntactic Ambiguity Resolution , 1994 .