Grammar-based connectionist approaches to language

This article describes an approach to connectionist language research that relies on the development of grammar formalisms rather than computer models. From formulations of the fundamental theoretical commitments of connectionism and of generative grammar, it is argued that these two paradigms are mutually compatible. Integrating the basic assumptions of the paradigms results in formal theories of grammar that centrally incorporate a certain degree of connectionist computation. Two such grammar formalisms—Harmonic Grammar (Legendre, Miyata, & Smolensky, 1990a,b) and Optimality Theory (Prince & Smolensky, 1991, 1993)—are briefly introduced to illustrate grammar-based approaches to connectionist language research. The strengths and weaknesses of grammar-based research and more traditional model-based research are argued to be complementary, suggesting a significant role for both strategies in the spectrum of connectionist language research.

[1]  Michael C. Mozer,et al.  Perception of multiple objects - a connectionist approach , 1991, Neural network modeling and connectionism.

[2]  Michael C. Mozer,et al.  Mathematical Perspectives on Neural Networks , 1996 .

[3]  R. Langacker.  Foundations of cognitive grammar , 1983 .

[4]  James L. McClelland.  Toward a theory of information processing in graded, random, and interactive networks , 1993 .

[5]  J J Hopfield,et al.  Neurons with graded response have collective computational properties like those of two-state neurons. , 1984, Proceedings of the National Academy of Sciences of the United States of America.

[6]  L. Shastri,et al.  From simple associations to systematic reasoning: A connectionist representation of rules, variables and dynamic bindings using temporal synchrony , 1993, Behavioral and Brain Sciences.

[7]  Alan Prince,et al.  Prosodic morphology : constraint interaction and satisfaction , 1993 .

[8]  Caroline Heycock,et al.  Language Acquisition: Knowledge Representation and Processing , 1999 .

[9]  I. Biederman,et al.  Dynamic binding in a neural network for shape recognition. , 1992, Psychological review.

[10]  Cheryl Cydney Zoll,et al.  Parsing Below the Segment in a Constraint-Based Framework , 1998 .

[11]  A Prince,et al.  Optimality: From Neural Networks to Universal Grammar , 1997, Science.

[12]  G. O. Stone,et al.  An analysis of the delta rule and the learning of statistical associations , 1986 .

[13]  George Lakoff,et al.  Women, Fire, and Dangerous Things , 1987 .

[14]  Geoffrey E. Hinton,et al.  Learning distributed representations of concepts. , 1989 .

[15]  Yves Chauvin,et al.  Backpropagation: the basic theory , 1995 .

[16]  P. Smolensky.  On the proper treatment of connectionism , 1988, Behavioral and Brain Sciences.

[17]  Y. Miyata,et al.  Harmonic grammar: A formal multi-level connectionist theory of linguistic well-formedness: Theoretic , 1990 .

[18]  Géraldine Legendre,et al.  Principles for an Integrated Connectionist/Symbolic Theory of Higher Cognition ; CU-CS-600-92 , 1992 .

[19]  David S. Touretzky,et al.  A Computational Basis for Phonology , 1989, NIPS.

[20]  David Zipser,et al.  Feature Discovery by Competive Learning , 1986, Cogn. Sci..

[21]  Teuvo Kohonen,et al.  Associative memory. A system-theoretical approach , 1977 .

[22]  Paul Smolensky,et al.  Information processing in dynamical systems: foundations of harmony theory , 1986 .

[23]  Geoffrey E. Hinton,et al.  OPTIMAL PERCEPTUAL INFERENCE , 1983 .

[24]  Geoffrey E. Hinton,et al.  A Distributed Connectionist Production System , 1988, Cogn. Sci..

[25]  Nick Chater,et al.  Connectionist natural language processing: the state of the art , 1999 .

[26]  John J. Hopfield,et al.  Neural networks and physical systems with emergent collective computational abilities , 1999 .

[27]  P. Smolensky,et al.  When is less more? Faithfulness and minimal links in wh-chains , 1998 .

[28]  Katya Zubritskaya,et al.  Mechanism of sound change in Optimality Theory , 1997, Language Variation and Change.

[29]  Tony Plate,et al.  Holographic Reduced Representations: Convolution Algebra for Compositional Distributed Representations , 1991, IJCAI.

[30]  Stephen A. Ritz,et al.  Distinctive features, categorical perception, and probability learning: some applications of a neural model , 1977 .

[31]  S. Pinker,et al.  On language and connectionism: Analysis of a parallel distributed processing model of language acquisition , 1988, Cognition.

[32]  T. Bever,et al.  The relation between linguistic structure and associative theories of language learning—A constructive critique of some connectionist learning models , 1988, Cognition.

[33]  M. McCloskey.  Networks and Theories: The Place of Connectionism in Cognitive Science , 1991 .

[34]  D. Gary Miller,et al.  1. Theoretical Prerequisites , 1994 .

[35]  Paul Smolensky,et al.  Tensor Product Variable Binding and the Representation of Symbolic Structures in Connectionist Systems , 1990, Artif. Intell..

[36]  W. Freeman.  Second Commentary: On the proper treatment of connectionism by Paul Smolensky (1988) - Neuromachismo Rekindled , 1989 .

[37]  Bruce Tesar,et al.  Learning optimality-theoretic grammars☆ , 1998 .

[38]  Michael C. Mozer,et al.  Rule Induction through Integrated Symbolic and Subsymbolic Processing , 1991, NIPS.

[39]  Jordan B. Pollack,et al.  Recursive Distributed Representations , 1990, Artif. Intell..

[40]  James L. McClelland,et al.  On learning the past-tenses of English verbs: implicit rules or parallel distributed processing , 1986 .

[41]  Leonard Talmy,et al.  Force Dynamics in Language and Cognition , 1987, Cogn. Sci..

[42]  C. P. Dolan,et al.  Tensor manipulation networks: connectionist and symbolic approaches to comprehension, learning, and planning , 1989 .

[43]  P. Smolensky,et al.  Harmonic Grammar -- A Formal Multi-Level Connectionist Theory of Linguistic Well-Formedness: An Application ; CU-CS-464-90 , 1990 .

[44]  J. Blevins.  The Syllable in Phonological Theory , 1995 .

[46]  P. Smolensky.  On the comprehension/production dilemma in child language , 1996 .

[47]  Mitsuhiko Ota,et al.  Optimality Theory: an overview , 2000 .

[48]  Terrence J. Sejnowski,et al.  Parallel Networks that Learn to Pronounce English Text , 1987, Complex Syst..

[49]  Jeffrey L. Elman,et al.  Finding Structure in Time , 1990, Cogn. Sci..

[50]  Michael I. Jordan.  Attractor dynamics and parallelism in a connectionist sequential machine , 1990 .

[51]  Jerome A. Feldman,et al.  Connectionist Models and Their Properties , 1982, Cogn. Sci..

[52]  Paul Smolensky,et al.  Lexical and postlexical processes in spoken word production , 1999 .

[53]  Geoffrey E. Hinton,et al.  Learning and relearning in Boltzmann machines , 1986 .

[54]  James L. McClelland,et al.  Parallel distributed processing: explorations in the microstructure of cognition, vol. 1: foundations , 1986 .

[55]  Pilar Barbosa,et al.  Is the best good enough? : optimality and competition in syntax , 1998 .

[56]  James L. McClelland,et al.  A distributed, developmental model of word recognition and naming. , 1989, Psychological review.

[57]  T. Shallice,et al.  Connectionist Modelling in Cognitive Neuropsychology: A Case Study , 1994 .

[58]  Geoffrey E. Hinton,et al.  Adaptive Mixtures of Local Experts , 1991, Neural Computation.

[59]  S. Pinker,et al.  Connections and symbols , 1988 .

[60]  R. Jakobson.  Child Language, Aphasia and Phonological Universals , 1980 .

[61]  GrossbergS..  Adaptive pattern classification and universal recoding , 1976 .

[62]  R. Jakobson,et al.  Selected Writings: I. Phonological Studies , 1965 .

[63]  Paul Smolensky,et al.  Schema Selection and Stochastic Inference in Modular Environments , 1983, AAAI.