Description-based parsing in a connectionist network

Recent developments in connectionist architectures for symbolic computation have made it possible to investigate parsing in a connectionist network while still taking advantage of the large body of work on parsing in symbolic frameworks. This dissertation investigates syntactic parsing in the temporal synchrony variable binding model of symbolic computation in a connectionist network. This computational architecture solves the basic problem with previous connectionist architectures, while keeping their advantages. However, the architecture does have some limitations, which impose computational constraints on parsing in this architecture. This dissertation argues that, despite these constraints, the architecture is computationally adequate for syntactic parsing, and that these constraints make significant linguistic predictions. To make these arguments, the nature of the architecture's limitations are first characterized as a set of constraints on symbolic computation. This allows the investigation of the feasibility and implications of parsing in the architecture to be investigated at the same level of abstraction as virtually all other investigations of syntactic parsing. Then a specific parsing model is developed and implemented in the architecture. The extensive use of partial descriptions of phrase structure trees is crucial to the ability of this model to recover the syntactic structure of sentences within the constraints. Finally, this parsing model is tested on those phenomena which are of particular concern given the constraints, and on an approximately unbiased sample of sentences to check for unforeseen difficulties. The results show that this connectionist architecture is powerful enough for syntactic parsing. They also show that some linguistic phenomena are predicted by the limitations of this architecture. In particular, explanations are given for many cases of unacceptable center embedding, and for several significant constraints on long distance dependencies. These results give evidence for the cognitive significance of this computational architecture and parsing model. This work also shows how the advantages of both connectionist and symbolic techniques can be unified in natural language processing applications. By analyzing how low level biological and computational considerations influence higher level processing, this work has furthered our understanding of the nature of language and how it can be efficiently and effectively processed.

[1]  Edward Gibson,et al.  A computational theory of human linguistic processing: memory limitations and processing breakdown , 1991 .

[2]  Barbara J. Grosz,et al.  Natural-Language Processing , 1982, Artificial Intelligence.

[3]  Mark Steedman,et al.  Interaction with context during human sentence processing , 1988, Cognition.

[4]  Ivan A. Sag,et al.  Information-based syntax and semantics , 1987 .

[5]  J. Fodor,et al.  Connectionism and cognitive architecture: A critical analysis , 1988, Cognition.

[6]  Jordan B. Pollack,et al.  Recursive Distributed Representations , 1990, Artif. Intell..

[7]  J. Kimball Seven principles of surface structure parsing in natural language , 1973 .

[8]  Mitchell P. Marcus,et al.  A theory of syntactic recognition for natural language , 1979 .

[9]  Bryan J. Hubbell,et al.  On memory limitations in natural language processing , 1982 .

[10]  William C. Rounds,et al.  A Logical Version of Functional Grammar , 1987, ACL.

[11]  XTAG Research Group,et al.  A Lexicalized Tree Adjoining Grammar for English , 1998, ArXiv.

[12]  J. Hayes Cognition and the development of language , 1970 .

[13]  Geoffrey K. Pullum,et al.  Generalized Phrase Structure Grammar , 1985 .

[14]  John A. Barnden,et al.  Encoding techniques for complex information structures in connectionist systems , 1991 .

[15]  Michael G. Dyer,et al.  High-level Inferencing in a Connectionist Network , 1989 .

[16]  Jeffrey L. Elman,et al.  Finding Structure in Time , 1990, Cogn. Sci..

[17]  R. Sun On Variable Binding in Connectionist Networks , 1992 .

[18]  L. Shastri,et al.  From simple associations to systematic reasoning: A connectionist representation of rules, variables and dynamic bindings using temporal synchrony , 1993, Behavioral and Brain Sciences.

[19]  G. A. Miller The magical number seven plus or minus two: some limits on our capacity for processing information. , 1956, Psychological review.

[20]  Mitchell P. Marcus,et al.  D-Theory: Talking about Talking about Trees , 1983, ACL.

[21]  Mark Johnson Expressing Disjunctive and Negative Feature Constraints with Classical First-Order Logic , 1990, ACL.

[22]  Aravind K. Joshi,et al.  An Introduction to Tree Adjoining Grammar , 1987 .

[23]  David Pesetsky,et al.  Paths and categories , 1982 .

[24]  Ajay Naresh Jain,et al.  Parsec: a connectionist learning architecture for parsing spoken language , 1992 .

[25]  Aravind K. Joshi,et al.  Word-Order Variation in Natural Language Generation , 1987, AAAI.

[26]  Gerard Kempen,et al.  Incremental syntactic tree formation in human sentence processing: A cognitive architecture based on activation decay and simulated annealing , 1989 .

[27]  K. Vijay-Shanker,et al.  Using Descriptions of Trees in a Tree Adjoining Grammar , 1992, Comput. Linguistics.

[28]  Elizabeth Cowper,et al.  Constraints on sentence complexity : a model for syntactic processing , 1976 .

[29]  C. Fillmore,et al.  Regularity and Idiomaticity in Grammatical Constructions: The Case of Let Alone , 1988 .

[30]  William C. Rounds,et al.  A Complete Logical Calculus for Record Structures Representing Linguistic Information , 1986, LICS.

[31]  Joachim Lambek,et al.  On the Calculus of Syntactic Types , 1961 .