Towards a Computational Model of Actor-Based Language Comprehension

Neurophysiological data from a range of typologically diverse languages provide evidence for a cross-linguistically valid, actor-based strategy of understanding sentence-level meaning. This strategy seeks to identify the participant primarily responsible for the state of affairs (the actor) as quickly and unambiguously as possible, thus resulting in competition for the actor role when there are multiple candidates. Due to its applicability across languages with vastly different characteristics, we have proposed that the actor strategy may derive from more basic cognitive or neurobiological organizational principles, though it is also shaped by distributional properties of the linguistic input (e.g. the morphosyntactic coding strategies for actors in a given language). Here, we describe an initial computational model of the actor strategy and how it interacts with language-specific properties. Specifically, we contrast two distance metrics derived from the output of the computational model (one weighted and one unweighted) as potential measures of the degree of competition for actorhood by testing how well they predict modulations of electrophysiological activity engendered by language processing. To this end, we present an EEG study on word order processing in German and use linear mixed-effects models to assess the effect of the various distance metrics. Our results show that a weighted metric, which takes into account the weighting of an actor-identifying feature in the language under consideration outperforms an unweighted distance measure. We conclude that actor competition effects cannot be reduced to feature overlap between multiple sentence participants and thereby to the notion of similarity-based interference, which is prominent in current memory-based models of language processing. Finally, we argue that, in addition to illuminating the underlying neurocognitive mechanisms of actor competition, the present model can form the basis for a more comprehensive, neurobiologically plausible computational model of constructing sentence-level meaning.

[1]  Hinrich Schütze,et al.  Book Reviews: Foundations of Statistical Natural Language Processing , 1999, CL.

[2]  Ina Bornkessel-Schlesewsky,et al.  Exploring the nature of the ‘subject’-preference: Evidence from the online comprehension of simple sentences in Mandarin Chinese , 2009 .

[3]  Brian McElree,et al.  Accessing Recent Events , 2006 .

[4]  M. Schlesewsky,et al.  Reconciling time, space and function: A new dorsal–ventral stream model of sentence comprehension , 2013, Brain and Language.

[5]  P. Haggard Human volition: towards a neuroscience of will , 2008, Nature Reviews Neuroscience.

[6]  Caitlin M. Fausey,et al.  Who dunnit? Cross-linguistic differences in eye-witness memory , 2011, Psychonomic bulletin & review.

[7]  Kara D. Federmeier Thinking ahead: the role and roots of prediction in language comprehension. , 2007, Psychophysiology.

[8]  G. Kempen,et al.  Syntactic structure assembly in human parsing: a computational model based on competitive inhibition and a lexicalist grammar , 2000, Cognition.

[9]  Matthias Schlesewsky,et al.  An alternative perspective on “semantic P600” effects in language comprehension , 2008, Brain Research Reviews.

[10]  Gustavo Deco,et al.  Stochastic dynamics as a principle of brain function , 2009, Progress in Neurobiology.

[11]  A. Leslie A theory of agency. , 1995 .

[12]  Dorothee J. Chwilla,et al.  Monitoring in Language Perception , 2009, Lang. Linguistics Compass.

[13]  G. Schwarz Estimating the Dimension of a Model , 1978 .

[14]  Gina R. Kuperberg,et al.  Neural mechanisms of language comprehension: Challenges to syntax , 2007, Brain Research.

[15]  E. Rolls,et al.  Brain mechanisms for perceptual and reward-related decision-making , 2013, Progress in Neurobiology.

[16]  Brian MacWhinney,et al.  Functional constraints on sentence processing: A cross-linguistic study , 1982, Cognition.

[17]  B. MacWhinney,et al.  Cue validity and sentence interpretation in English, German, and Italian , 1984 .

[18]  Ina Bornkessel-Schlesewsky,et al.  The role of animacy in the real time comprehension of Mandarin Chinese: Evidence from auditory event-related brain potentials , 2008, Brain and Language.

[19]  H. Kolk,et al.  Monitoring in language perception: The effect of misspellings of words in highly constrained sentences , 2006, Brain Research.

[20]  S L Wolf,et al.  An Alternative Perspective , 1999, Applied psychophysiology and biofeedback.

[21]  Ina Bornkessel-Schlesewsky,et al.  Universal and cross-linguistic influences on the processing of word order and animacy: Neurophysiological evidence from Tamil , 2008 .

[22]  Ina Bornkessel-Schlesewsky,et al.  The Role of Prominence Information in the Real-Time Comprehension of Transitive Constructions: A Cross-Linguistic Approach , 2009, Lang. Linguistics Compass.

[23]  Matthias Schlesewsky,et al.  To Predict or Not to Predict: Influences of Task and Strategy on the Processing of Semantic Relations , 2007, Journal of Cognitive Neuroscience.

[24]  D. J. Felleman,et al.  Distributed hierarchical processing in the primate cerebral cortex. , 1991, Cerebral cortex.

[25]  A. Friederici Towards a neural basis of auditory sentence processing , 2002, Trends in Cognitive Sciences.

[26]  A. Newell Unified Theories of Cognition , 1990 .

[27]  Matthias Schlesewsky,et al.  The Subject Preference in the Processing of Locally Ambiguous WH-Questions in German , 2000 .

[28]  David R. Anderson,et al.  Model selection and multimodel inference : a practical information-theoretic approach , 2003 .

[29]  M. Kutas,et al.  ERPs and Domain Specificity: Beating a Straw Horse , 1998 .

[30]  Angela D. Friederici,et al.  Who did what to whom? The neural basis of argument hierarchies during language comprehension , 2005, NeuroImage.

[31]  A. Friederici,et al.  The status of subject-object reanalyses in the language comprehension architecture , 2008 .

[32]  J. Fodor The Modularity of mind. An essay on faculty psychology , 1986 .

[33]  C. Rorden,et al.  Damage to left anterior temporal cortex predicts impairment of complex syntactic processing: A lesion‐symptom mapping study , 2013, Human brain mapping.

[34]  Russell S. Tomlin,et al.  Basic Word Order : Functional Principles , 2013 .

[35]  M. Schlesewsky,et al.  Competition in argument interpretation: Evidence from the neurobiology of language , 2014 .

[36]  WILLIAM MARSLEN-WILSON,et al.  Linguistic Structure and Speech Shadowing at Very Short Latencies , 1973, Nature.

[37]  H. Akaike A new look at the statistical model identification , 1974 .

[38]  George Kingsley Zipf,et al.  Human behavior and the principle of least effort , 1949 .

[39]  R. C. Oldfield The assessment and analysis of handedness: the Edinburgh inventory. , 1971, Neuropsychologia.

[40]  M. Schlesewsky,et al.  On the universality of language comprehension strategies: Evidence from Turkish , 2008, Cognition.

[41]  M. Silverstein 7. Hierarchy of Features and Ergativity , 1986 .

[42]  Ina Bornkessel,et al.  The Argument Dependency Model: A neurocognitive approach to incremental interpretation , 2002 .

[43]  Matthias Schlesewsky,et al.  Linguistic prominence and Broca's area: The influence of animacy as a linearization principle , 2006, NeuroImage.

[44]  L. Cosmides,et al.  Category-specific attention for animals reflects ancestral priorities, not expertise , 2007, Proceedings of the National Academy of Sciences.

[45]  C. Frith,et al.  Rapid Response Subject Collections , 2022 .

[46]  Franziska Kretzschmar,et al.  The electrophysiological reality of parafoveal processing: On the validity of language-related ERPs in natural reading , 2010 .

[47]  Lars Konieczny,et al.  German Sentence Processing , 1999 .

[48]  D. Barr,et al.  Random effects structure for confirmatory hypothesis testing: Keep it maximal. , 2013, Journal of memory and language.

[49]  Richard L. Lewis Specifying Architectures for Language Processing: Process, Control, and Memory in Parsing and Interpretation , 1997 .

[50]  A. Friederici,et al.  Differential task effects on semantic and syntactic processes as revealed by ERPs. , 2002, Brain research. Cognitive brain research.

[51]  C. Phillips,et al.  Examining the evidence for an independent semantic analyzer: An ERP study in Spanish , 2012, Brain and Language.

[52]  Katherine A. DeLong,et al.  Probabilistic word pre-activation during language comprehension inferred from electrical brain activity , 2005, Nature Neuroscience.

[53]  Christian J Fiebach,et al.  On the cost of syntactic ambiguity in human language comprehension: an individual differences approach. , 2004, Brain research. Cognitive brain research.

[54]  J. Rauschecker Cortical processing of complex sounds , 1998, Current Opinion in Neurobiology.

[55]  Brian MacWhinney,et al.  Competition all the way down: How children learn word order cues to sentence meaning , 2014 .

[56]  Nicolas J. Bourguignon,et al.  Decomposing animacy reversals between agents and experiencers: An ERP study , 2012, Brain and Language.

[57]  G. Zipf,et al.  The Psycho-Biology of Language , 1936 .

[58]  E. Bates,et al.  Psycholinguistics: a cross-language perspective. , 2001, Annual review of psychology.

[59]  Marica De Vincenzi,et al.  Filler-gap dependencies in a null subject language: Referential and nonreferential WHs , 1991 .

[60]  Stephani Foraker,et al.  Memory structures that subserve sentence comprehension , 2003 .

[61]  Matthias Schlesewsky,et al.  The extended argument dependency model: a neurocognitive approach to sentence comprehension across languages. , 2006, Psychological review.

[62]  Östen Dahl,et al.  Animacy and egophoricity: Grammar, ontology and phylogeny , 2008 .

[63]  Brian MacWhinney,et al.  Processing of Morphological and Semantic Cues in Russian and German. , 1999 .

[64]  D. Bates,et al.  Linear Mixed-Effects Models using 'Eigen' and S4 , 2015 .

[65]  Ellen F. Lau,et al.  A cortical network for semantics: (de)constructing the N400 , 2008, Nature Reviews Neuroscience.

[66]  Richard L. Lewis,et al.  The mind and brain of short-term memory. , 2008, Annual review of psychology.

[67]  HERBERT A. SIMON,et al.  The Architecture of Complexity , 1991 .

[68]  Ina Bornkessel-Schlesewsky,et al.  Conflicts in language processing: A new perspective on the N400–P600 distinction , 2011, Neuropsychologia.

[69]  J. Aitchinson The crosslinguistic study of sentence processing , 1993, Journal of Child Language.

[70]  Ina Bornkessel-Schlesewsky,et al.  Meaningful physical changes mediate lexical–semantic integration: Top-down and form-based bottom-up information sources interact in the N400 , 2011, Neuropsychologia.

[71]  P. Hagoort On Broca, brain, and binding: a new framework , 2005, Trends in Cognitive Sciences.

[72]  Lyn Frazier,et al.  Perspectives on sentence processing , 2015 .

[73]  J. Rauschecker,et al.  Phoneme and word recognition in the auditory ventral stream , 2012, Proceedings of the National Academy of Sciences.

[74]  S. S. Stevens Mathematics, measurement, and psychophysics. , 1951 .

[75]  Alessandro Angrilli,et al.  Processing of temporary syntactic ambiguity in Italian “who”-questions: a study with event-related potentials , 2005, Neuroscience Letters.

[76]  J. Rauschecker,et al.  Maps and streams in the auditory cortex: nonhuman primates illuminate human speech processing , 2009, Nature Neuroscience.

[77]  Tilo Kircher,et al.  Neural correlates of narrative shifts during auditory story comprehension , 2009, NeuroImage.

[78]  Julian M. Pine,et al.  Constructing a Language: A Usage-Based Theory of Language Acquisition. , 2004 .

[79]  Matthias Schlesewsky,et al.  On incremental interpretation: Degrees of meaning accessed during sentence comprehension , 2004 .

[80]  Jonathan D. Cohen,et al.  Decision making, the P3, and the locus coeruleus-norepinephrine system. , 2005, Psychological bulletin.

[81]  L. Osterhout,et al.  The independence of combinatory semantic processing: Evidence from event-related potentials , 2005 .

[82]  J. Fodor The Modularity of mind. An essay on faculty psychology , 1986 .

[83]  Kara D. Federmeier,et al.  Electrophysiology reveals semantic memory use in language comprehension , 2000, Trends in Cognitive Sciences.

[84]  V. Reus,et al.  Mind and brain. , 1993, Science.

[85]  S. S. Stevens,et al.  Handbook of experimental psychology , 1951 .

[86]  R. Oostenveld,et al.  Nonparametric statistical testing of EEG- and MEG-data , 2007, Journal of Neuroscience Methods.

[87]  William Croft,et al.  Radical Construction Grammar: Syntactic Theory in Typological Perspective , 2001 .

[88]  Angela D. Friederici,et al.  The Processing of Locally Ambiguous Relative Clauses in German , 1995 .

[89]  B. McElree,et al.  A content-addressable pointer mechanism underlies comprehension of verb-phrase ellipsis , 2008 .

[90]  Hartmut Fitz,et al.  Getting real about Semantic Illusions: Rethinking the functional role of the P600 in language comprehension , 2012, Brain Research.

[91]  M. Schlesewsky,et al.  Think globally: Cross-linguistic variation in electrophysiological activity during sentence comprehension , 2011, Brain and Language.

[92]  B. McElree,et al.  Multi-dimensional contributions to garden path strength: Dissociating phrase structure from case marking , 2004 .

[93]  Richard L. Lewis,et al.  Computational principles of working memory in sentence comprehension , 2006, Trends in Cognitive Sciences.

[94]  Robert Dixon,et al.  Grammatical Categories in Australian Languages , 1980 .

[95]  M. Schlesewsky,et al.  The neural mechanisms of word order processing revisited: Electrophysiological evidence from Japanese , 2008, Brain and Language.

[96]  Peter Hagoort,et al.  How the brain solves the binding problem for language: a neurocomputational model of syntactic processing , 2003, NeuroImage.

[97]  Elisabeth Rieken Radical construction grammar: Syntactic theory in typological perspective , 2009 .

[98]  M. Kutas,et al.  An electrophysiological analysis of animacy effects in the processing of object relative sentences. , 1999, Psychophysiology.

[99]  F. Christian How the Brain Integrates Costs and Benefits During Decision Making , 2010 .

[100]  H. Huynh,et al.  Conditions under Which Mean Square Ratios in Repeated Measurements Designs Have Exact F-Distributions , 1970 .

[101]  Matthias Schlesewsky,et al.  Fractionating language comprehension via frequency characteristics of the human EEG , 2004, Neuroreport.

[102]  Dorothee J. Chwilla,et al.  Monitoring in Language Perception: Mild and Strong Conflicts Elicit Different ERP Patterns , 2010, Journal of Cognitive Neuroscience.

[103]  R. Baayen,et al.  Mixed-effects modeling with crossed random effects for subjects and items , 2008 .

[104]  Michael A. Arbib,et al.  Action and Language Mechanisms in the Brain: Data, Models and Neuroinformatics , 2013, Neuroinformatics.

[105]  Lyn Frazier,et al.  Syntactic processing: Evidence from dutch , 1987 .

[106]  M. Kutas,et al.  Reading senseless sentences: brain potentials reflect semantic incongruity. , 1980, Science.

[107]  Gerard Kempen,et al.  Parsing Verb-Final Clauses in German: Garden-path and ERP Effects Modeled by a Parallel Dynamic Parser , 2008 .

[108]  H. Kolk,et al.  Structure and limited capacity in verbal working memory: A study with event-related potentials , 2003, Brain and Language.

[109]  Gerard Kempen,et al.  The Unification Space implemented as a localist neural net: predictions and error-tolerance in a constraint-based parser , 2009, Cognitive Neurodynamics.

[110]  J. Fodor,et al.  The Modularity of Mind: An Essay on Faculty Psychology , 1984 .

[111]  Caitlin M. Fausey,et al.  Constructing Agency: The Role of Language , 2010, Front. Psychology.

[112]  S. Frisch,et al.  The N400 reflects problems of thematic hierarchizing , 2001, Neuroreport.

[113]  Eric Maris,et al.  Randomization tests for ERP topographies and whole spatiotemporal data matrices. , 2004, Psychophysiology.

[114]  Richard L. Lewis,et al.  An Activation-Based Model of Sentence Processing as Skilled Memory Retrieval , 2005, Cogn. Sci..

[115]  Leslie G. Ungerleider,et al.  A general mechanism for perceptual decision-making in the human brain , 2004, Nature.