Quasiregularity and Its Discontents: The Legacy of the Past Tense Debate

Rumelhart and McClelland's chapter about learning the past tense created a degree of controversy extraordinary even in the adversarial culture of modern science. It also stimulated a vast amount of research that advanced the understanding of the past tense, inflectional morphology in English and other languages, the nature of linguistic representations, relations between language and other phenomena such as reading and object recognition, the properties of artificial neural networks, and other topics. We examine the impact of the Rumelhart and McClelland model with the benefit of 25 years of hindsight. It is not clear who "won" the debate. It is clear, however, that the core ideas that the model instantiated have been assimilated into many areas in the study of language, changing the focus of research from abstract characterizations of linguistic competence to an emphasis on the role of the statistical structure of language in acquisition and processing.

[1]  Marvin Minsky,et al.  Perceptrons: An Introduction to Computational Geometry , 1969 .

[2]  Joan L. Bybee,et al.  Rules and schemas in the development and use of the English past tense , 1982 .

[3]  Karalyn Patterson,et al.  PAST-TENSE GENERATION FROM FORM VERSUS MEANING: BEHAVIOURAL DATA AND SIMULATION EVIDENCE. , 2009, Journal of memory and language.

[4]  Steven Pinker,et al.  Lexical semantics and irregular inflection , 2010, Language and cognitive processes.

[5]  Mark Johnson,et al.  How the Statistical Revolution Changes (Computational) Linguistics , 2009 .

[6]  James L. McClelland,et al.  U-Shaped Curves in Development: A PDP Approach , 2004, Journal of Cognition and Development.

[7]  W D Marslen-Wilson,et al.  Sentence Perception as an Interactive Parallel Process , 1975, Science.

[8]  S. Pinker,et al.  Default nominal inflection in Hebrew: evidence for mental variables , 1999, Cognition.

[9]  Maryellen C. MacDonald,et al.  Constraint Satisfaction Accounts of Lexical and Sentence Comprehension , 2006 .

[10]  R. Holloway The broth in my brother ’ s brothel : Morpho-orthographic segmentation in visual word recognition , 2005 .

[11]  James L. McClelland,et al.  On learning the past-tenses of English verbs: implicit rules or parallel distributed processing , 1986 .

[12]  Jon Andoni Duñabeitia,et al.  Is morpho-orthographic decomposition purely orthographic? Evidence from masked priming in the same–different task , 2011 .

[13]  M. Halle Prolegomena to a theory of word formation , 1973 .

[14]  V. Marchman,et al.  U-shaped learning and frequency effects in a multi-layered perception: Implications for child language acquisition , 1991, Cognition.

[15]  Maryellen C. MacDonald,et al.  A probabilistic constraints approach to language acquisition and processing , 1999, Cogn. Sci..

[16]  Michele Miozzo,et al.  On the processing of regular and irregular forms of verbs and nouns: evidence from neuropsychology , 2003, Cognition.

[17]  Michael Ramscar The role of meaning in inflection: Why the past tense does not require a rule , 2002, Cognitive Psychology.

[18]  S. Pinker,et al.  The past and future of the past tense , 2002, Trends in Cognitive Sciences.

[19]  P. Smolensky,et al.  Optimality Theory: Constraint Interaction in Generative Grammar , 2004 .

[20]  Noam Chomsky,et al.  वाक्यविन्यास का सैद्धान्तिक पक्ष = Aspects of the theory of syntax , 1965 .

[21]  K. Wexler Very early parameter setting and the unique checking constraint: a new explanation of the optional infinitive stage: a new explanation of the optional infinitive stage , 1998 .

[22]  Maria Teresa Guasti,et al.  Language acquisition : the growth of grammar , 2002 .

[23]  R. Gómez,et al.  Infant artificial language learning and language acquisition , 2000, Trends in Cognitive Sciences.

[24]  Luigi Burzio Missing players: Phonology and the past-tense debate , 2002 .

[25]  Robert M. Gonyea,et al.  Learning at a Distance : , 2009 .

[26]  Noam Chomsky,et al.  A Review of B. F. Skinner's Verbal Behavior , 1980 .

[27]  James L. McClelland The Place of Modeling in Cognitive Science , 2009, Top. Cogn. Sci..

[28]  J. Elman,et al.  Rethinking Innateness: A Connectionist Perspective on Development , 1996 .

[29]  Steven Pinker,et al.  Generalizations of regular and irregular morphology , 1993 .

[30]  K. Plunkett,et al.  A cross-linguistic comparison of single and dual-route models of inflectional morphology , 2000 .

[31]  Jenny R. Saffran,et al.  Verbs are LookING good in early language acquisition - eScholarship , 2009 .

[32]  Mark S. Seidenberg,et al.  Reading in different writing systems: One architecture, multiple solutions. , 2011 .

[33]  Jeffrey L. Elman,et al.  Finding Structure in Time , 1990, Cogn. Sci..

[34]  Steven Pinker,et al.  Why No Mere Mortal Has Ever Flown Out to Center Field , 1991, Cogn. Sci..

[35]  D. Plaut,et al.  SD-squared: on the association between semantic dementia and surface dyslexia. , 2007, Psychological review.

[36]  J. Fodor,et al.  Connectionism and cognitive architecture: A critical analysis , 1988, Cognition.

[37]  M. Tanenhaus,et al.  Modeling the Influence of Thematic Fit (and Other Constraints) in On-line Sentence Comprehension , 1998 .

[38]  J. Marshall,et al.  Patterns of paralexia: a psycholinguistic approach. , 1995, Journal of psycholinguistic research.

[39]  S. Pinker,et al.  On language and connectionism: Analysis of a parallel distributed processing model of language acquisition , 1988, Cognition.

[40]  James L. McClelland,et al.  Why there are complementary learning systems in the hippocampus and neocortex: insights from the successes and failures of connectionist models of learning and memory. , 1995, Psychological review.

[41]  Elissa L. Newport,et al.  Plus or Minus 30 Years in the Language Sciences , 2010, Top. Cogn. Sci..

[42]  James L. McClelland,et al.  Rules or connections in past-tense inflections: what does the evidence rule out? , 2002, Trends in Cognitive Sciences.

[43]  Max Coltheart,et al.  Access to the internal lexicon , 1977 .

[44]  James L. McClelland,et al.  Sentence comprehension: A parallel distributed processing approach , 1989, Language and Cognitive Processes.

[45]  J. Stroop Studies of interference in serial verbal reactions. , 1992 .

[46]  Dorothy Irene Marquart,et al.  The Language Lottery: Toward a Biology of Grammars , 1982 .

[47]  C. L. Baker,et al.  The Logical problem of language acquisition , 1984 .

[48]  Mark S. Seidenberg,et al.  Beyond Rules and Exceptions: A Connectionist Approach to Inflectional Morphology , 1994 .

[49]  D. Bickerton The language bioprogram hypothesis , 1984, Behavioral and Brain Sciences.

[50]  Mark S. Seidenberg,et al.  Cognitive neuropsychology and language: The state of the art , 1988 .

[51]  James L. McClelland,et al.  An interactive activation model of context effects in letter perception: I. An account of basic findings. , 1981 .

[52]  Michael McCloskey,et al.  Catastrophic Interference in Connectionist Networks: The Sequential Learning Problem , 1989 .

[53]  Matthew H. Davis,et al.  Morphological and semantic effects in visual word recognition: A time-course study , 2000 .

[54]  Matthew H. Davis,et al.  The broth in my brother’s brothel: Morpho-orthographic segmentation in visual word recognition , 2004, Psychonomic bulletin & review.

[55]  Elizabeth Jefferies,et al.  Presemantic Cognition in Semantic Dementia: Six Deficits in Search of an Explanation , 2006, Journal of Cognitive Neuroscience.

[56]  J. Berko The Child's Learning of English Morphology , 1958 .

[57]  Rebecca Treiman,et al.  Influence of consonantal context on the pronunciation of vowels: A comparison of human readers and computational models , 2003, Cognition.

[58]  James L. McClelland,et al.  Deficits in phonology and past-tense morphology: What's the connection? , 2003 .

[59]  Noam Chomsky Knowledge of language: its nature, origin, and use , 1988 .

[60]  R N Aslin,et al.  Statistical Learning by 8-Month-Old Infants , 1996, Science.

[61]  S Pinker,et al.  Overregularization in language acquisition. , 1992, Monographs of the Society for Research in Child Development.

[62]  E. Lenneberg Biological Foundations of Language , 1967 .

[63]  S Pinker,et al.  Rules of language. , 1991, Science.

[64]  Maryellen C. MacDonald,et al.  The lexical nature of syntactic ambiguity resolution , 1994 .

[65]  Melody Dye,et al.  Learning language from the input: Why innate constraints can’t explain noun compounding , 2011, Cognitive Psychology.

[66]  David C. Plaut,et al.  Are non-semantic morphological effects incompatible with a distributed connectionist approach to lexical processing? , 2000 .

[67]  Melody Dye,et al.  When the fly flied and when the fly flew: How semantics affect the processing of inflected verbs , 2013, Language and Cognitive Processes.

[68]  G. Lakoff Women, fire, and dangerous things : what categories reveal about the mind , 1989 .

[69]  R. Levy Expectation-based syntactic comprehension , 2008, Cognition.

[70]  James L. McClelland,et al.  A distributed, developmental model of word recognition and naming. , 1989, Psychological review.

[71]  Lorraine K Tyler,et al.  Morphology, language and the brain: the decompositional substrate for language comprehension , 2007, Philosophical Transactions of the Royal Society B: Biological Sciences.

[72]  References , 1971 .

[73]  Richard N. Aslin,et al.  From shared contexts to syntactic categories: The role of distributional information in learning linguistic form-classes , 2013, Cognitive Psychology.

[74]  T. Bever,et al.  The relation between linguistic structure and associative theories of language learning—A constructive critique of some connectionist learning models , 1988, Cognition.

[75]  Refractor Vision , 2000, The Lancet.

[76]  A. Jacobs,et al.  Localist connectionist approaches to human cognition , 1998 .

[77]  James L. McClelland,et al.  Understanding normal and impaired word reading: computational principles in quasi-regular domains. , 1996, Psychological review.

[78]  B. Hayes,et al.  Rules vs. analogy in English past tenses: a computational/experimental study , 2003, Cognition.

[79]  Maryellen C. MacDonald,et al.  Language learning and innateness: Some implications of Compounds Research , 2003, Cognitive Psychology.

[80]  B. MacWhinney,et al.  Implementations are not conceptualizations: Revising the verb learning model , 1991, Cognition.

[81]  Mark S. Seidenberg,et al.  Explaining derivational morphology as the convergence of codes , 2000, Trends in Cognitive Sciences.

[82]  Steven Frisson,et al.  Phonological and Orthographic Overlap Effects in Fast and Masked Priming , 2014, Quarterly journal of experimental psychology.

[83]  S. Pinker,et al.  A Neural Dissociation within Language: Evidence that the Mental Dictionary Is Part of Declarative Memory, and that Grammatical Rules Are Processed by the Procedural System , 1997, Journal of Cognitive Neuroscience.

[84]  David E. Rumelhart,et al.  Toward an interactive model of reading. , 1994 .

[85]  S. Pinker Words and rules in the human brain , 1997, Nature.

[86]  J. Tenenbaum,et al.  The learnability of abstract syntactic principles , 2011, Cognition.

[87]  Carol Lynn Moder,et al.  Morphological Classes as Natural Categories , 1983 .

[88]  Brian MacWhinney,et al.  The emergence of language. , 1999 .

[89]  James L. McClelland,et al.  Letting structure emerge: connectionist and dynamical systems approaches to cognition , 2010, Trends in Cognitive Sciences.

[90]  Martina Penke,et al.  German Noun Plurals: A Challenge to the Dual-Mechanism Model , 2002, Brain and Language.

[91]  G. Kane Parallel Distributed Processing: Explorations in the Microstructure of Cognition, vol 1: Foundations, vol 2: Psychological and Biological Models , 1994 .

[92]  James L. McClelland,et al.  Parallel distributed processing: explorations in the microstructure of cognition, vol. 1: foundations , 1986 .

[93]  Maryellen C MacDonald,et al.  Where does gender come from? Evidence from a complex inflectional system , 2005, Language and cognitive processes.

[94]  V. Marchman,et al.  From rote learning to system building: acquiring verb morphology in children and connectionist nets , 1993, Cognition.

[95]  S. Crain Language acquisition in the absence of experience , 1991, Behavioral and Brain Sciences.

[96]  D. Siegel Topics in English morphology , 1979 .

[97]  Maryellen C. MacDonald,et al.  Semantics and phonology constrain compound formation , 2007 .

[98]  C. Habel,et al.  Language , 1931, NeuroImage.

[99]  M. Coltheart,et al.  Surface dyslexia. , 1983, The Quarterly journal of experimental psychology. A, Human experimental psychology.

[100]  T. Landauer,et al.  A Solution to Plato's Problem: The Latent Semantic Analysis Theory of Acquisition, Induction, and Representation of Knowledge. , 1997 .

[101]  Jeffrey R. Binder,et al.  fMRI of Past Tense Processing: The Effects of Phonological Complexity and Task Difficulty , 2006, Journal of Cognitive Neuroscience.

[102]  Steven Pinker,et al.  The Dislike of Regular Plurals in Compounds: Phonological Familiarity or Morphological Constraint? , 2007 .

[103]  E. Mark Gold,et al.  Language Identification in the Limit , 1967, Inf. Control..

[104]  Noam Chomsky,et al.  Rules and representations , 1980, Behavioral and Brain Sciences.

[105]  Mark S. Seidenberg,et al.  2 Progress in understanding word reading : Data fi tting versus theory building , 2006 .

[106]  George Sperling,et al.  The information available in brief visual presentations. , 1960 .

[107]  Steven Pinker,et al.  The Nature of Regularity and Irregularity: Evidence from Hebrew Nominal Inflection , 2002, Journal of psycholinguistic research.

[108]  E. Newport,et al.  Learning at a distance I. Statistical learning of non-adjacent dependencies , 2004, Cognitive Psychology.

[109]  Mark S. Seidenberg,et al.  Computing the meanings of words in reading: cooperative division of labor between visual and phonological processes. , 2004, Psychological review.

[110]  Xiaojin Zhu,et al.  Can semi-supervised learning explain incorrect beliefs about categories? , 2011, Cognition.

[111]  John McCarthy,et al.  Chess as the Drosophila of AI , 1990 .

[112]  James L. McClelland,et al.  Connectionist models of development , 2003 .

[113]  Chien Yu-Chin,et al.  Children's Knowledge of Locality Conditions in Binding as Evidence for the Modularity of Syntax and Pragmatics , 1990 .

[114]  L. Gerken,et al.  Infants can use distributional cues to form syntactic categories , 2005, Journal of Child Language.

[115]  Mark S. Seidenberg,et al.  Simulating consistency effects and individual differences in nonword naming: A comparison of current models , 2006 .

[116]  Sally Andrews,et al.  From inkmarks to ideas : current issues in lexical processing , 2006 .

[117]  J. Shonkoff,et al.  Development of infants with disabilities and their families: implications for theory and service delivery. , 1992, Monographs of the Society for Research in Child Development.

[118]  Carson T. Schütze The empirical base of linguistics: Grammaticality judgments and linguistic methodology , 1998 .

[119]  Noam Chomsky,et al.  Lectures on Government and Binding , 1981 .

[120]  James L. McClelland Running Head : Letting Structure Emerge Letting Structure Emerge : Connectionist and Dynamical Systems Approaches to Understanding Cognition , 2009 .

[121]  James L. McClelland,et al.  Parallel Distributed Processing: Explorations in the Microstructure of Cognition : Psychological and Biological Models , 1986 .

[122]  James L. McClelland,et al.  An interactive activation model of context effects in letter perception: part 1.: an account of basic findings , 1988 .

[123]  G. Miller,et al.  Cognitive science. , 1981, Science.

[124]  Yvonne Freeh,et al.  Words And Rules The Ingredients Of Language , 2016 .

[125]  L. Gleitman The Structural Sources of Verb Meanings , 2020, Sentence First, Arguments Afterward.

[126]  Mark S. Seidenberg Connectionist Models and Cognitive Theory , 1993 .

[127]  Jenny R. Saffran,et al.  Distributional structure in language: Contributions to noun–verb difficulty differences in infant word recognition , 2014, Cognition.

[128]  Mark S. Seidenberg,et al.  Language Acquisition and Use: Learning and Applying Probabilistic Constraints , 1997, Science.

[129]  M. McCloskey Networks and Theories: The Place of Connectionism in Cognitive Science , 1991 .

[130]  Massimo Piattelli-Palmarini,et al.  Ever since language and learning: afterthoughts on the Piaget-Chomsky debate , 1994, Cognition.

[131]  Mark S. Seidenberg,et al.  Impairments in verb morphology after brain injury: a connectionist model. , 1999, Proceedings of the National Academy of Sciences of the United States of America.

[132]  Paul Boersma,et al.  Modeling Productivity with the Gradual Learning Algorithm: The Problem of Accidentally Exceptionless Generalizations , 2005 .

[133]  S. Pinker The Language Instinct , 1994 .

[134]  Peter Gordon,et al.  Red rats eater exposes recursion in children's word formation , 1996, Cognition.

[135]  James L. McClelland,et al.  Alternatives to the combinatorial paradigm of linguistic theory based on domain general principles of human cognition , 2005 .

[136]  U. Hahn,et al.  German Inflection: Single Route or Dual Route? , 2000, Cognitive Psychology.

[137]  Gary F. Marcus,et al.  German Inflection: The Exception That Proves the Rule , 1995, Cognitive Psychology.

[138]  D Zipser,et al.  Learning the hidden structure of speech. , 1988, The Journal of the Acoustical Society of America.

[139]  J. Tenenbaum,et al.  Variability, negative evidence, and the acquisition of verb argument constructions. , 2010, Journal of child language.

[140]  P. Kiparsky Explanation In Phonology , 1982 .

[141]  Mark S. Seidenberg,et al.  Connecting cues: overlapping regularities support cue discovery in infancy. , 2010, Child development.

[142]  R. Mccall,et al.  The Genetic and Environmental Origins of Learning Abilities and Disabilities in the Early School , 2007, Monographs of the Society for Research in Child Development.

[143]  Jelena Mirkovic,et al.  Rules Versus Statistics: Insights From a Highly Inflected Language , 2011, Cogn. Sci..

[144]  P. Gordon Level-ordering in lexical development , 1985, Cognition.

[145]  G. Lakoff,et al.  Women, Fire, and Dangerous Things: What Categories Reveal about the Mind , 1988 .

[146]  Julie C. Sedivy,et al.  Subject Terms: Linguistics Language Eyes & eyesight Cognition & reasoning , 1995 .

[147]  R Ratcliff,et al.  Connectionist models of recognition memory: constraints imposed by learning and forgetting functions. , 1990, Psychological review.