Dynamical models of sentence processing

We suggest that the theory of dynamical systems provides a revealing general framework for modeling the representations and mechanism underlying syntactic processing. We show how a particular dynamical model, the Visitation Set Gravitation model of Tabor, Juliano, and Tanenhaus (1997), develops syntactic representations and models a set of contingent frequency effects in parsing that are problematic for other models. We also present new simulations showing how the model accounts for semantic effects in parsing, and propose a new account of the distinction between syntactic and semantic incongruity. The results show how symbolic structures useful in parsing arise as emergent properties of connectionist dynamical systems.

[1]  Morten H. Christiansen.  Innite Languages, Finite Minds Connectionism, Learning and Linguistic Structure , 1994 .

[2]  Jing Peng,et al.  An Efficient Gradient-Based Algorithm for On-Line Training of Recurrent Network Trajectories , 1990, Neural Computation.

[3]  Yves Chauvin,et al.  Backpropagation: the basic theory , 1995 .

[4]  Fernando J. Pineda.  Recurrent backpropagation networks , 1995 .

[5]  Nick Chater,et al.  Connectionist natural language processing: the state of the art , 1999, Cogn. Sci..

[6]  Michael K. Tanenhaus,et al.  Parsing in a Dynamical System: An Attractor-based Account of the Interaction of Lexical and Structural Constraints in Sentence Processing , 1997 .

[7]  Paul Rodríguez,et al.  A Recurrent Neural Network that Learns to Count , 1999, Connect. Sci..

[8]  Garrison W. Cottrell,et al.  Viewing parsing as word sense discrimination: a connectionist approach , 1984 .

[9]  James L. McClelland,et al.  Understanding normal and impaired word reading: computational principles in quasi-regular domains. , 1996, Psychological review.

[10]  James L. McClelland,et al.  Parallel distributed processing: explorations in the microstructure of cognition, vol. 1: foundations , 1986 .

[11]  Jordan B. Pollack,et al.  Massively Parallel Parsing: A Strongly Interactive Model of Natural Language Interpretation , 1988, Cogn. Sci..

[12]  B. McElree,et al.  Syntactic and Thematic Processing in Sentence Comprehension: Evidence for a Temporal Dissociation , 1995 .

[13]  R. J. Nelson,et al.  Introduction to Automata , 1968 .

[14]  Nick Chater,et al.  Connectionist natural language processing: the state of the art , 1999 .

[15]  守屋 悦朗,et al.  J.E.Hopcroft, J.D. Ullman 著, "Introduction to Automata Theory, Languages, and Computation", Addison-Wesley, A5変形版, X+418, \6,670, 1979 , 1980 .

[16]  Ma Conway,et al.  Handbook of perception and cognition , 1996 .

[17]  Jeffrey D. Ullman,et al.  Introduction to Automata Theory, Languages and Computation , 1979 .

[18]  W. Tabor.  Dynamical Automata , 1998 .

[19]  G. Kane.  Parallel Distributed Processing: Explorations in the Microstructure of Cognition, vol 1: Foundations, vol 2: Psychological and Biological Models , 1994 .

[20]  Luís B. Almeida,et al.  A learning rule for asynchronous perceptrons with feedback in a combinatorial environment , 1990 .

[21]  Johanna D. Moore.  Proceedings of the fifteenth annual meeting of the Cognitive Science Society , 1993 .

[22]  B. Kendall.  Nonlinear Dynamics and Chaos , 2001 .

[23]  J. Crutchfield.  The calculi of emergence: computation, dynamics and induction , 1994 .

[24]  Curt Burgess,et al.  Modelling Parsing Constraints with High-dimensional Context Space , 1997 .

[25]  Eugene Charniak,et al.  Statistical language learning , 1997 .

[26]  James L. McClelland,et al.  An interactive activation model of context effects in letter perception: I. An account of basic findings. , 1981 .

[27]  T. Landauer,et al.  A Solution to Plato's Problem: The Latent Semantic Analysis Theory of Acquisition, Induction, and Representation of Knowledge. , 1997 .

[28]  Maryellen C. MacDonald,et al.  The lexical nature of syntactic ambiguity resolution , 1994 .

[29]  Colin M. Brown,et al.  The syntactic positive shift (sps) as an erp measure of syntactic processing , 1993 .

[30]  Kim Ainsworth-Darnell,et al.  Dissociating Brain Responses to Syntactic and Semantic Anomalies: Evidence from Event-Related Potentials , 1998 .

[31]  Giovanni Guida,et al.  Computational models of natural language processing , 1984 .

[32]  Daniel Jurafsky,et al.  A Probabilistic Model of Lexical and Syntactic Access and Disambiguation , 1996, Cogn. Sci..

[33]  Jordan B. Pollack,et al.  Recursive Distributed Representations , 1990, Artif. Intell..

[34]  Geoffrey E. Hinton,et al.  Learning internal representations by error propagation , 1986 .

[35]  Jeffrey L. Elman,et al.  A PDP Approach to Processing Center-Embedded Sentences , 1992 .

[36]  R. Abraham,et al.  Dynamics--the geometry of behavior , 1983 .

[37]  M. Tanenhaus,et al.  Modeling the Influence of Thematic Fit (and Other Constraints) in On-line Sentence Comprehension , 1998 .

[38]  Lee Osterhout,et al.  Event-related potentials and syntactic anomaly: Evidence of anomaly detection during the perception of continuous speech , 1993 .

[39]  Susan M. Garnsey.  Event-related brain potentials in the study of language: An introduction , 1993 .

[40]  Whitney Tabor,et al.  Syntactic innovation : a connectionist model , 1994 .

[41]  Jeffrey L. Elman,et al.  Finding Structure in Time , 1990, Cogn. Sci..

[42]  E. Gibson.  Linguistic complexity: locality of syntactic dependencies , 1998, Cognition.

[43]  J. Trueswell.  THE ROLE OF LEXICAL FREQUENCY IN SYNTACTIC AMBIGUITY RESOLUTION , 1996 .

[44]  James L. McClelland,et al.  Learning and Applying Contextual Constraints in Sentence Comprehension , 1990, Artif. Intell..

[45]  Nick Chater,et al.  Toward a connectionist model of recursion in human linguistic performance , 1999, Cogn. Sci..