Connectionist natural language processing: the state of the art

This Special Issue on Connectionist Models of Human Language Processing provides an opportunity for an appraisal both of specific connectionist models and of the status and utility of connectionist models of language in general. This introduction provides the background for the papers in the Special Issue. The development of connectionist models of language is traced, from their intellectual origins, to the state of current research. Key themes that arise throughout different areas of connectionist psycholinguistics are highlighted, and recent developments in speech processing, morphology, sentence processing, language production, and reading are described. We argue that connectionist psycholinguistics has already had a significant impact on the psychology of language, and that connectionist models are likely to have an important influence on future research.

[1]  U. Hahn,et al.  German Inflection: Single Route or Dual Route? , 2000, Cognitive Psychology.

[2]  Steven Pinker,et al.  Generalisation of regular and irregular morphological patterns , 1993 .

[3]  Jeffrey L. Elman,et al.  Interactive processes in speech perception: the TRACE model , 1986 .

[4]  James L. McClelland,et al.  More Words but Still No Lexicon: Reply to Besner et al. (1990) , 1990 .

[5]  Gary F. Marcus,et al.  German Inflection: The Exception That Proves the Rule , 1995, Cognitive Psychology.

[6]  James L. McClelland,et al.  Generalization with Componential Attractors: Word and Nonword Reading in an Attractor Network , 1993 .

[7]  S. Pinker,et al.  On language and connectionism: Analysis of a parallel distributed processing model of language acquisition , 1988, Cognition.

[8]  David C. Plaut,et al.  Structure and Function in the Lexical System: Insights from Distributed Models of Word Reading and Lexical Decision , 1997 .

[9]  Gary S. Dell,et al.  Connectionist models of language production: lexical access and grammatical encoding , 1999, Cogn. Sci..

[10]  Johanna D. Moore.  Proceedings of the fifteenth annual meeting of the Cognitive Science Society , 1993 .

[11]  G. Dell,et al.  Language production and serial order: a functional analysis and a model. , 1997, Psychological review.

[12]  N. Chater,et al.  Autonomy, implementation and cognitive architecture: A reply to Fodor and Pylyshyn , 1990, Cognition.

[13]  Gary S. Dell,et al.  Structure and Content in Language Production: A Theory of Frame Constraints in Phonological Speech Errors , 1993, Cogn. Sci..

[14]  Mark S. Seidenberg,et al.  Phonology, reading acquisition, and dyslexia: insights from connectionist models. , 1999 .

[15]  R. Taraban,et al.  Language learning: Cues or rules? , 1989 .

[16]  A. Samuel.  Lexical Activation Produces Potent Phonemic Percepts , 1997, Cognitive Psychology.

[17]  Nick Chater,et al.  Connectionist modelling: Implications for cognitive neuropsychology , 1995 .

[18]  Paul W. B. Atkins,et al.  Models of reading aloud: Dual-route and parallel-distributed-processing approaches. , 1993 .

[19]  Morten H. Christiansen,et al.  Generalization and connectionist language learning , 1994 .

[20]  Gary S. Dell,et al.  The retrieval of phonological forms in production: tests of predictions from a connectionist model , 1988 .

[21]  Jeffrey L. Elman,et al.  Finding Structure in Time , 1990, Cogn. Sci..

[22]  Allen Newell,et al.  Human Problem Solving. , 1973 .

[23]  Ja Bullinaria,et al.  Connectionist modelling: Implications for neuropsychology , 1995 .

[24]  Jerome A. Feldman,et al.  Connectionist Models and Their Properties , 1982, Cogn. Sci..

[25]  Paul Smolensky.  Grammar-based connectionist approaches to language , 1999 .

[26]  Geoffrey E. Hinton,et al.  Learning internal representations by error propagation , 1986 .

[27]  Noam Chomsky,et al.  वाक्यविन्यास का सैद्धान्तिक पक्ष = Aspects of the theory of syntax , 1965 .

[28]  Ashwin Ram,et al.  Proceedings of the Sixteenth Annual Conference of the Cognitive Science Society , 2019 .

[29]  James L. McClelland,et al.  Parallel distributed processing: explorations in the microstructure of cognition, vol. 1: foundations , 1986 .

[30]  Mark A. Fanty,et al.  Context-free parsing with connectionist networks , 1987 .

[31]  Ken N. Seergobin,et al.  On the association between connectionism and data: Are a few words necessary? , 1990 .

[32]  M. Pitt,et al.  Is Compensation for Coarticulation Mediated by the Lexicon , 1998 .

[33]  Mark S. Seidenberg,et al.  Using Connectionist Networks to Examine the Role of Prior Constraints in Human Learning , 1994 .

[34]  Michael K. Tanenhaus,et al.  Parsing in a Dynamical System: An Attractor-based Account of the Interaction of Lexical and Structural Constraints in Sentence Processing , 1997 .

[35]  Joan L. Bybee,et al.  Regular morphology and the lexicon. , 1995 .

[36]  V. Marchman,et al.  U-shaped learning and frequency effects in a multi-layered perception: Implications for child language acquisition , 1991, Cognition.

[37]  James L. McClelland,et al.  Cognitive penetration of the mechanisms of perception: Compensation for coarticulation of lexically restored phonemes , 1988 .

[38]  M. Gareth Gaskell,et al.  Ambiguity, competition, and blending in spoken word recognition , 1999, Cogn. Sci..

[39]  G. Marcus,et al.  Regular and irregular inflection in the acquisition of German noun plurals , 1992, Cognition.

[40]  W. Pitts,et al.  A Logical Calculus of the Ideas Immanent in Nervous Activity (1943) , 2021, Ideas That Created the Future.

[41]  Eugene Charniak,et al.  Statistical language learning , 1997 .

[42]  Nick Chater,et al.  Distributional Information: A Powerful Cue for Acquiring Syntactic Categories , 1998, Cogn. Sci..

[43]  G. Dell,et al.  Origins of Paraphasias in Deep Dysphasia: Testing the Consequences of a Decay Impairment to an Interactive Spreading Activation Model of Lexical Retrieval , 1994, Brain and Language.

[44]  Idan Segev,et al.  Methods in neuronal modeling: From synapses to networks , 1989 .

[45]  P. Smolensky.  On the proper treatment of connectionism , 1988, Behavioral and Brain Sciences.

[46]  Noam Chomsky.  Knowledge of Language , 1986 .

[47]  James L. McClelland,et al.  Connections and disconnections: Acquired dyslexia in a computational model of reading processes. , 1989 .

[48]  James L. McClelland,et al.  An interactive activation model of context effects in letter perception: I. An account of basic findings. , 1981 .

[49]  J. Elman,et al.  Learning and morphological change , 1995, Cognition.

[50]  S Pinker,et al.  Rules of language. , 1991, Science.

[51]  David C. Plaut.  A Connectionist Approach to Word Reading and Acquired Dyslexia: Extension to Sequential Processing , 1999 .

[52]  James L. McClelland,et al.  A distributed, developmental model of word recognition and naming. , 1989, Psychological review.

[53]  William D. Marslen-Wilson,et al.  A connectionist model of phonological representation in speech perception , 1995 .

[54]  Trevor A. Harley,et al.  Phonological activation of semantic competitors during lexical access in speech production , 1993 .

[55]  J. Fodor,et al.  Connectionism and cognitive architecture: A critical analysis , 1988, Cognition.

[56]  J. Elman.  Learning and development in neural networks: the importance of starting small , 1993, Cognition.

[57]  Michael K. Tanenhaus,et al.  Dynamical models of sentence processing , 1999, Cogn. Sci..

[58]  A. Turing.  On Computable Numbers, with an Application to the Entscheidungsproblem. , 1937 .

[59]  Nick Chater,et al.  A phonologically motivated input representation for the modelling of auditory word perception in continuous speech , 1992 .

[60]  James L. McClelland,et al.  Understanding normal and impaired word reading: computational principles in quasi-regular domains. , 1996, Psychological review.

[61]  Jeffrey L. Elman,et al.  Distributed Representations, Simple Recurrent Networks, and Grammatical Structure , 1991, Mach. Learn..

[62]  William D. Marslen-Wilson,et al.  Integrating Form and Meaning: A Distributed Model of Speech Perception. , 1997 .

[63]  Robert Dale,et al.  Proceedings of the Thirteenth Annual Conference of the Cognitive Science Society , 1991 .

[64]  Garrison W. Cottrell,et al.  Toward Connectionist Parsing , 1982, AAAI.

[65]  Jeffrey L. Elman,et al.  Default Generalisation in Connectionist Networks. , 1995 .

[66]  A. A. Mullin,et al.  Principles of neurodynamics , 1962 .

[67]  James L. McClelland,et al.  An interactive activation model of context effects in letter perception: Part 2. The contextual enhancement effect and some tests and extensions of the model. , 1982, Psychological review.

[68]  G. Dell,et al.  Lexical access in aphasic and nonaphasic speakers. , 1997, Psychological review.

[69]  Nick Chater,et al.  Toward a connectionist model of recursion in human linguistic performance , 1999 .

[70]  Terrence J. Sejnowski,et al.  Parallel Networks that Learn to Pronounce English Text , 1987, Complex Syst..

[71]  G S Dell,et al.  A spreading-activation theory of retrieval in sentence production. , 1986, Psychological review.

[72]  J. Bullinaria.  Modeling Reading, Spelling, and Past Tense Learning with Artificial Neural Networks , 1997, Brain and Language.

[73]  N. Chater,et al.  Proceedings of the fourteenth annual conference of the cognitive science society , 1992 .

[74]  V. Marchman,et al.  From rote learning to system building: acquiring verb morphology in children and connectionist nets , 1993, Cognition.

[75]  W. Freeman.  Second Commentary: On the proper treatment of connectionism by Paul Smolensky (1988) - Neuromachismo Rekindled , 1989 .

[76]  J. Fodor.  The Modularity of mind. An essay on faculty psychology , 1986 .

[77]  Maryellen C. MacDonald,et al.  A probabilistic constraints approach to language acquisition and processing , 1999, Cogn. Sci..

[78]  Mark Steedman.  Connectionist Sentence Processing in Perspective , 1999 .

[79]  Patrick Juola,et al.  A connectionist model of English past tense and plural morphology , 1999 .

[80]  W. Levelt,et al.  Speaking: From Intention to Articulation , 1990 .

[81]  W. Ashby.  Design for a Brain , 1954 .