Bayesian Models of Human Sentence Processing

Human language processing relies on many kinds of linguisti c knowledge, and is sensitive to their frequency, including l exical frequencies (Tyler, 1984; Salasoo & Pisoni, 1985; Marsl enWilson, 1990; Zwitserlood, 1989; Simpson & Burgess, 1985), idiom frequencies (d’Arcais, 1993), phonological neighbo rhood frequencies (Luce, Pisoni, & Goldfinger, 1990), subcat egorization frequencies (Trueswell, Tanenhaus, & Kello, 19 93), and thematic role frequencies (Trueswell, Tanenhaus, & Gar nsey, 1994; Garnsey, Pearlmutter, Myers, & Lotocky, 1997). But while we know that each of these knowledge sources must be probabilistic, we know very little about exactly how thes e probabilistic knowledge sources are combined. This paper p roposes the use of Bayesian decision trees in modeling the prob abilistic, evidential nature of human sentence processing . Our method reifies conditional independence assertions implic it in sign-based linguistic theories and describes interaction s among features without requiring additional assumptions about m odularity. We show that our Bayesian approach successfully mod els psycholinguistic results on evidence combination in hu man lexical, idiomatic, and syntactic/semantic processing.

[1]  John C. Trueswell,et al.  Tense, Temporal Context, and Syntactic Ambiguity Resolution. , 1991 .

[2]  Lalit R. Bahl,et al.  A Maximum Likelihood Approach to Continuous Speech Recognition , 1983, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[3]  Jerry R. Hobbs,et al.  Two Principles of Parse Preference , 1990, COLING.

[4]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[5]  D. Swinney.  Lexical access during sentence comprehension: (Re)consideration of context effects , 1979 .

[6]  Bradley L. Pritchett.  Garden Path Phenomena and the Grammatical Basis of Language Processing , 1988 .

[7]  M. Tanenhaus,et al.  Context effects in lexical processing , 1987, Cognition.

[8]  P. Tabossi,et al.  The comprehension of idioms. , 1988 .

[9]  M. Tanenhaus,et al.  Context effects in syntactic ambiguity resolution: discourse and semantic influences in parsing reduced relative clauses. , 1993, Canadian journal of experimental psychology = Revue canadienne de psychologie experimentale.

[10]  Beatrice Santorini,et al.  Building a Large Annotated Corpus of English: The Penn Treebank , 1993, CL.

[11]  Susan M. Garnsey,et al.  Semantic Influences On Parsing: Use of Thematic Role Information in Syntactic Ambiguity Resolution , 1994 .

[12]  Curt Burgess,et al.  Activation and selection processes in the recognition of ambiguous words. , 1985 .

[13]  Ivan A. Sag,et al.  Information-Based Syntax and Semantics: Volume 1, Fundamentals , 1987 .

[14]  Christopher T. Kello,et al.  Verb-specific constraints in sentence processing: separating effects of lexical preference from garden-paths. , 1993, Journal of experimental psychology. Learning, memory, and cognition.

[15]  David B. Pisoni,et al.  Similarity neighborhoods of spoken words , 1991 .

[16]  M. MacDonald.  The interaction of lexical and syntactic ambiguity , 1993 .

[17]  Robert P. Goldman,et al.  A Logic for Semantic Interpretation , 1988, ACL.

[18]  Charles J. Fillmore,et al.  The Mechanisms of “Construction Grammar” , 1988 .

[19]  Paola Merlo,et al.  A corpus-based analysis of verb continuation frequencies for syntactic processing , 1994 .

[20]  Daniel Jurafsky,et al.  A Probabilistic Model of Lexical and Syntactic Access and Disambiguation , 1996, Cogn. Sci..

[21]  Philip Resnik,et al.  Probabilistic Tree-Adjoining Grammar as a Framework for Statistical Natural Language Processing , 1992, COLING.

[22]  Paul Gorrell,et al.  Establishing the loci of serial and parallel effects in syntactic processing , 1989 .

[23]  Julie C. Sedivy,et al.  Resolving attachment ambiguities with multiple constraints , 1995, Cognition.

[24]  K. Rayner,et al.  Resolution of syntactic category ambiguities: Eye movements in parsing lexically ambiguous sentences☆ , 1987 .

[25]  C. Clifton,et al.  The independence of syntactic processing , 1986 .

[26]  Lorraine K. Tyler,et al.  The role of lexical representations in language comprehension , 1989 .

[27]  Judea Pearl,et al.  Bayesian Networks , 1998, Encyclopedia of Social Network Analysis and Mining. 2nd Ed..

[28]  Steven P. Abney,et al.  Parsing arguments: Phrase structure and argument structure as determinants of initial parsing decisions. , 1991 .

[29]  Daniel Jurafsky,et al.  How Verb Subcategorization Frequencies Are Affected By Corpus Choice , 1998, COLING.

[30]  P. Resnik.  Selection and information: a class-based approach to lexical relationships , 1993 .

[31]  John Cocke,et al.  Probabilistic Parsing Method for Sentence Disambiguation , 1989, IWPT.

[32]  Maryellen C. MacDonald,et al.  Probabilistic constraints and syntactic ambiguity resolution , 1994 .

[33]  P. Zwitserlood.  The locus of the effects of sentential-semantic context in spoken-word processing , 1989, Cognition.

[34]  L. Tyler.  The structure of the initial cohort: Evidence from gating , 1984, Perception & Psychophysics.

[35]  Edward Gibson,et al.  A computational theory of human linguistic processing: memory limitations and processing breakdown , 1991 .

[36]  Ivan A. Sag,et al.  Information-based syntax and semantics , 1987 .

[37]  A. Salasoo,et al.  Interaction of Knowledge Sources in Spoken Word Identification. , 1985, Journal of memory and language.

[38]  William D. Marslen-Wilson,et al.  Activation, competition, and frequency in lexical access , 1991 .

[39]  Susan M. Garnsey,et al.  The Contributions of Verb Bias and Plausibility to the Comprehension of Temporarily Ambiguous Sentences , 1997 .