A computational theory of human linguistic processing: memory limitations and processing breakdown

This thesis gives a theory of sentence comprehension that attempts to explain a number of linguistic performance effects, including garden-path effects, preferred readings for ambiguous input and processing overload effects. It is hypothesized that the human parser heuristically determines its options based upon evaluation of possible representations with respect to lexical, syntactic, semantic and pragmatic properties, each of which is associated with a weight. Processing overload effects are explained by the assumption of the existence of a maximum load corresponding to the limited capacity of short term memory: a structure becomes unacceptable at a particular parse state if the combination of the processing weights associated with its properties at that state is greater than the available capacity. Furthermore, it is assumed that the language processor is an automatic device that maintains only the best of the set of all compatible representations for an input string. This thesis assumes a formulation of representational evaluation within a parallel framework: one structure is preferred over another if the processing load associated with the first structure is markedly lower than the processing load associated with the second. Thus a garden path effect results if the unpreferred structure is necessary for a successful parse of the input. Four properties of linguistic representations are presented within this framework. The first two--the Properties of Thematic Reception and Transmission--derivable from the $\theta$-Criterion from Government-Binding (GB) Theory (Chomsky (1981)); the third--the Property of Lexical Requirement--derivable from the Projection Principle of GB Theory; and the fourth--the Property of Recency Preference--prefers local attachments over more distant attachments (cf. Kimball (1973), Frazier (1979)). This thesis shows how these properties interact to give a partially unified theory of many performance effects.

[1]  G. A. Miller THE PSYCHOLOGICAL REVIEW THE MAGICAL NUMBER SEVEN, PLUS OR MINUS TWO: SOME LIMITS ON OUR CAPACITY FOR PROCESSING INFORMATION 1 , 1956 .

[2]  Noam Chomsky,et al.  A Note on Phrase Structure Grammars , 1959, Inf. Control..

[3]  Noam Chomsky,et al.  On Certain Formal Properties of Grammars , 1959, Inf. Control..

[4]  Victor H. Yngve,et al.  A model and an hypothesis for language structure , 1960 .

[5]  G. A. Miller,et al.  Some perceptual consequences of linguistic rules , 1963 .

[6]  G. A. Miller,et al.  Finitary models of language users , 1963 .

[7]  J. Fodor,et al.  The structure of a semantic theory , 1963 .

[8]  J. Katz,et al.  An integrated theory of linguistic descriptions , 1964 .

[9]  Noam Chomsky,et al.  वाक्यविन्यास का सैद्धान्तिक पक्ष = Aspects of the theory of syntax , 1965 .

[10]  Donald E. Knuth,et al.  On the Translation of Languages from Left to Right , 1965, Inf. Control..

[11]  Arthur L. Blumenthal,et al.  Observations with self-embedded sentences , 1966 .

[12]  Walter S. Stolz,et al.  A study of the ability to decode grammatically novel sentences , 1967 .

[13]  D. Davidson Truth and Meaning , 1967 .

[14]  J. Fodor,et al.  Some syntactic determinants of sentential complexity , 1967 .

[15]  George A. Miller,et al.  Introduction to the Formal Analysis of Natural Languages , 1968 .

[16]  G. Seth Psychology of Language , 1968, Nature.

[17]  Noam Chomsky,et al.  Remarks on Nominalization , 2020, Nominalization.

[18]  I. M. Schlesinger Sentence structure and the reading process. , 1968 .

[19]  Noam Chomsky,et al.  Conditions on transformations , 1971 .

[20]  Ronald M. Kaplan,et al.  Augmented Transition Networks as Psychological Models of Sentence Comprehension , 1971, IJCAI.

[21]  Steven Abney,et al.  The English Noun Phrase in its Sentential Aspect , 1972 .

[22]  Alfred V. Aho,et al.  The Theory of Parsing, Translation, and Compiling , 1972 .

[23]  Terry Winograd,et al.  Understanding natural language , 1974 .

[24]  T. E. Moore,et al.  Speeded recognition of ungrammaticality , 1972 .

[25]  Jozef Gruska Generalized context-free grammars , 1973, Acta Cybern..

[26]  J. Kimball Seven principles of surface structure parsing in natural language , 1973 .

[27]  Barry K. Rosen,et al.  Syntactic Complexity , 1974, Inf. Control..

[28]  J. Fodor,et al.  The Psychology of Language , 1974 .

[29]  J. L. Mistler-Lachman Queer sentences, ambiguity, and levels of processing , 1975, Memory & cognition.

[30]  J. W. Kimball,et al.  Predictive analysis and over-the-top parsing , 1975 .

[31]  Elizabeth Cowper,et al.  Constraints on sentence complexity : a model for syntactic processing , 1976 .

[32]  William D. Marslen-Wilson,et al.  The On-Line Effects of Semantic Context on Syntactic Processing , 1977 .

[33]  Howard Lasnik,et al.  A Restrictive Theory of Transformational Grammar , 1977 .

[34]  Pamela A. Downing On the Creation and Use of English Compound Nouns. , 1977 .

[35]  Judith N. Levi,et al.  The syntax and semantics of complex nominals , 1978 .

[36]  Janet D. Fodor,et al.  The sausage machine: A new two-stage parsing model , 1978, Cognition.

[37]  Mitchell P. Marcus,et al.  A theory of syntactic recognition for natural language , 1979 .

[38]  Frank Nuessel,et al.  X̄ syntax: a study of phrase structure. Linguistic Inquiry Monograph Two: Ray Jackendoff, MIT Press, Cambridge, Mass., 1977. xii, 249 pp. $11.50 (subscribers to Linguistic Inquiry pay $9.50). , 1979 .

[39]  D. Swinney Lexical access during sentence comprehension: (Re)consideration of context effects , 1979 .

[40]  I. Biederman,et al.  Speeded recognition of ungrammaticality: Double violations , 1979, Cognition.

[41]  Lyn Frazier,et al.  ON COMPREHENDING SENTENCES: SYNTACTIC PARSING STRATEGIES. , 1979 .

[42]  E. Wanner The ATN and the sausage machine: Which one is baloney? , 1980, Cognition.

[43]  W. Cooper,et al.  Sentence Processing: Psycholinguistic Studies Presented to Merrill Garrett. , 1980 .

[44]  Lyn Frazier,et al.  Is the human sentence parsing mechanism an ATN? , 1980, Cognition.

[45]  Timothy W. Finin The Semantic Interpretation of Nominal Compounds , 1980, AAAI.

[46]  W. Marslen-Wilson,et al.  The temporal structure of spoken language understanding , 1980, Cognition.

[47]  Noam Chomsky,et al.  Lectures on Government and Binding , 1981 .

[48]  Mark Steedman,et al.  The use of context by the psychological parser , 1981 .

[49]  P. Carpenter,et al.  Lexical retrieval and error recovery in reading: A model based on eye fixations , 1981 .

[50]  Tim Stowell,et al.  Origins of phrase structure , 1981 .

[51]  Bryan J. Hubbell,et al.  On memory limitations in natural language processing , 1982 .

[52]  K. Rayner,et al.  Making and correcting errors during sentence comprehension: Eye movements in the analysis of structurally ambiguous sentences , 1982, Cognitive Psychology.

[53]  Stanley Peters,et al.  Cross-Serial Dependencies in Dutch , 1982 .

[54]  David Pesetsky,et al.  Paths and categories , 1982 .

[55]  T. Bever,et al.  Natural units of representation interact during sentence comprehension , 1982 .

[56]  Robert William Milne,et al.  Predicting Garden Path Sentences , 1982, Cogn. Sci..

[57]  Richard Shillcock,et al.  The On-Line Resolution of Pronominal Anaphora , 1982 .

[58]  Noam Chomsky Some Concepts and Consequences of the Theory of Government and Binding , 1982 .

[59]  G. Milsark On length and structure in sentence parsing , 1983, Cognition.

[60]  S. Carroll Noam Chomsky. Some Concepts and Consequences of the Theory of Government and Binding. Linguistic Inquiry Monograph No. 6. Cambridge, Mass.: MIT Press. Pp. viii + 110 , 1983, Canadian Journal of Linguistics/Revue canadienne de linguistique.

[61]  Lyn Frazier,et al.  The interaction of syntax and semantics during sentence processing: eye movements in the analysis of semantically biased sentences , 1983 .

[62]  J. Fodor The Modularity of mind. An essay on faculty psychology , 1986 .

[63]  Stuart M. Shieber,et al.  Sentence Disambiguation by a Shift-Reduce Parsing Technique , 1983, ACL.

[64]  C. L. Baker,et al.  The Logical problem of language acquisition , 1984 .

[65]  Richard S. Kayne Connectedness and binary branching , 1984 .

[66]  Lyn Frazier,et al.  Lexical Expectations in Sentence Comprehension. , 1984 .

[67]  Mark S. Seidenberg,et al.  Pre- and postlexical loci of contextual effects on word recognition , 1984, Memory & cognition.

[68]  Lenhart K. Schubert On Parsing Preferences , 1984, COLING.

[69]  Howard Steven Kurtzman,et al.  Studies in syntactic ambiguity resolution , 1985 .

[70]  Dan Flickinger,et al.  Structure-Sharing in Lexical Representation , 1985, ACL.

[71]  Fernando C. N. Pereira Natural language parsing: A new characterization of attachment preferences , 1985 .

[72]  Michael K. Tanenhaus,et al.  Natural language parsing: Do listeners compute linguistic representations? , 1985 .

[73]  Carl Pollard,et al.  Parsing Head-Driven Phrase Structure Grammar , 1985, ACL.

[74]  V. M. Holmes,et al.  The role of specific information about the verb in parsing sentences with local structural ambiguity , 1985 .

[75]  Yorick Wilks,et al.  Syntax, Preference, and Right Attachment , 1985, IJCAI.

[76]  Janet Dean Fodor,et al.  Natural language parsing: How can grammars help parsers? , 1985 .

[77]  Simon Garrod,et al.  On the real-time character of interpretation during reading , 1985 .

[78]  Janet Dean Fodor,et al.  Deterministic parsing and subjacency , 1985 .

[79]  M. Baltin,et al.  The Mental representation of grammatical relations , 1985 .

[80]  Alan Garnham,et al.  Psycholinguistics: Central Topics , 1985 .

[81]  Jordan B. Pollack,et al.  Massively Parallel Parsing: A Strongly Interactive Model of Natural Language Interpretation , 1988, Cogn. Sci..

[82]  K. Forster,et al.  The psychological status of overgenerated sentences , 1985, Cognition.

[83]  Noam Chomsky Knowledge of Language , 1986 .

[84]  MilneRobert,et al.  Resolving Lexical Ambiguity in a Deterministc Parser , 1986, Comput. Linguistics.

[85]  C. Clifton,et al.  The independence of syntactic processing , 1986 .

[86]  William D. Marslen-Wilson,et al.  Crossed and nested dependencies in German and Dutch , 1986 .

[87]  S Kuno,et al.  Multiple-path syntactic analyzer , 1986 .

[88]  Stuart M. Shieber,et al.  An Introduction to Unification-Based Approaches to Grammar , 1986, CSLI Lecture Notes.

[89]  Paul Griffith Gorrell,et al.  STUDIES OF HUMAN SYNTACTIC PROCESSING: RANKED-PARALLEL VERSUS SERIAL MODELS , 1987 .

[90]  Michael K. Tanenhaus,et al.  Thematic roles and language comprehension , 1988 .

[91]  K. Rayner,et al.  Parsing Temporarily Ambiguous Complements , 1987 .

[92]  Wayne S. Murray,et al.  Syntactic structure and the garden path , 1987 .

[93]  Masaru Tomita,et al.  An Efficient Augmented-Context-Free Parsing Algorithm , 1987, Comput. Linguistics.

[94]  R. Jackendoff The Status of Thematic Relations in Linguistic Theory , 1987 .

[95]  K. Rayner,et al.  Resolution of syntactic category ambiguities: Eye movements in parsing lexically ambiguous sentences☆ , 1987 .

[96]  Ivan A. Sag,et al.  Information-based syntax and semantics , 1987 .

[97]  F. Cuetos,et al.  Cross-linguistic differences in parsing: Restrictions on the use of the Late Closure strategy in Spanish , 1988, Cognition.

[98]  Mark Steedman,et al.  Interaction with context during human sentence processing , 1988, Cognition.

[99]  James L. McClelland,et al.  Constituent Attachment and Thematic Role Assignment in Sentence Processing: Influences of Content-Based Expectations , 1988 .

[100]  A. Weinberg Locality principles in syntax and in parsing , 1988 .

[101]  Peter Norvig Multiple Simultaneous Interpretations of Ambiguous Sentences , 1988 .

[102]  A. Belletti,et al.  Psych-verbs and θ-theory , 1988 .

[103]  Bradley L. Pritchett Garden Path Phenomena and the Grammatical Basis of Language Processing , 1988 .

[104]  Gerry T. M. Altmann,et al.  Ambiguity, parsing strategies, and computational models , 1988 .

[105]  Don C. Mitchell,et al.  Verb guidance and other lexical effects in parsing , 1989 .

[106]  Edward Gibson Parsing with Principles: Predicting a Phrasal Node Before Its Head Appears , 1989, IWPT.

[107]  C. Reid,et al.  Parsing Complements: Comments on the Generality of the Principle of Minimal Attachment , 1989 .

[108]  Steven Abney,et al.  A computational model of human parsing , 1989 .

[109]  Jean-Yves Pollock Verb movement, universal grammar and the structure of IP , 1989 .

[110]  Mark Steedman,et al.  Ambiguity in context: A reply , 1989 .

[111]  Stephen Crain,et al.  How to Resolve Structural Ambiguities , 1990 .

[112]  Edward Gibson,et al.  Memory Capacity and Sentence Processing , 1990, ACL.

[113]  Aravind K. Joshi,et al.  Processing crossed and nested dependencies: An automation perspective on the psycholinguistic results , 1990 .

[114]  Lyn Frazier,et al.  Parsing modifiers: Special purpose routines in the human sentence processing mechanism? , 1990 .

[115]  Don C. Mitchell,et al.  Reading in different languages: Is there a universal mechanism for parsing sentences? , 1990 .

[116]  Edward Gibson,et al.  A Computational Theory of Processing Overload and Garden-Path Effects , 1990, COLING.

[117]  Graeme Hirst,et al.  Race-Based Parsing and Syntactic Disambiguation , 1990, Cogn. Sci..

[118]  M. Just,et al.  Individual differences in syntactic processing: The role of working memory , 1991 .

[119]  Steven P. Abney,et al.  In Principle Based Parsing , 1991 .

[120]  M. Just,et al.  Working memory constraints on the processing of syntactic ambiguity , 1992, Cognitive Psychology.

[121]  Graeme Hirst,et al.  Semantic Interpretation and the Resolution of Ambiguity , 1987, Studies in natural language processing.

[122]  Jerry R. Hobbs,et al.  Interpretation as Abduction , 1993, Artif. Intell..

[123]  Yukio Oba,et al.  ON THE DOUBLE OBJECT CONSTRUCTION , 1993 .