Learning the Past Tense of English Verbs: The Symbolic Pattern Associator vs. Connectionist Models

Learning the past tense of English verbs - a seemingly minor aspect of language acquisition - has generated heated debates since 1986, and has become a landmark task for testing the adequacy of cognitive modeling. Several artificial neural networks (ANNs) have been implemented, and a challenge for better symbolic models has been posed. In this paper, we present a general-purpose Symbolic Pattern Associator (SPA) based upon the decision-tree learning algorithm ID3. We conduct extensive head-to-head comparisons on the generalization ability between ANN models and the SPA under different representations. We conclude that the SPA generalizes the past tense of unseen verbs better than ANN models by a wide margin, and we offer insights as to why this should be the case. We also discuss a new default strategy for decision-tree learning algorithms.

[1]  Morris Halle,et al.  The rules of language , 1980, IEEE Transactions on Professional Communication.

[2]  James L. McClelland,et al.  On learning the past-tenses of English verbs: implicit rules or parallel distributed processing , 1986 .

[3]  S. Pinker,et al.  On language and connectionism: Analysis of a parallel distributed processing model of language acquisition , 1988, Cognition.

[4]  J. Fodor,et al.  Connectionism and cognitive architecture: A critical analysis , 1988, Cognition.

[5]  T. Bever,et al.  The relation between linguistic structure and associative theories of language learning—A constructive critique of some connectionist learning models , 1988, Cognition.

[6]  Casimir A. Kulikowski,et al.  Computer Systems That Learn: Classification and Prediction Methods from Statistics, Neural Nets, Machine Learning and Expert Systems , 1990 .

[7]  Thomas G. Dietterich,et al.  A Comparative Study of ID3 and Backpropagation for English Text-to-Speech Mapping , 1990, ML.

[8]  V. Marchman,et al.  U-shaped learning and frequency effects in a multi-layered perception: Implications for child language acquisition , 1991, Cognition.

[9]  Thomas G. Dietterich,et al.  Error-Correcting Output Codes: A General Method for Improving Multiclass Inductive Learning Programs , 1991, AAAI.

[10]  B. MacWhinney.  The CHILDES project: tools for analyzing talk , 1992 .

[11]  B. MacWhinney,et al.  Implementations are not conceptualizations: Revising the verb learning model , 1991, Cognition.

[12]  J. Ross Quinlan,et al.  C4.5: Programs for Machine Learning , 1992 .

[13]  Elie Bienenstock,et al.  Neural Networks and the Bias/Variance Dilemma , 1992, Neural Computation.

[14]  B. MacWhinney.  Connections and symbols: closing the gap , 1993, Cognition.

[15]  Steven Pinker,et al.  Generalisation of regular and irregular morphological patterns , 1993 .

[16]  Marin Marinov,et al.  A Symbolic Model for Learning the Past-Tenses of English Verbs , 1993, IJCAI.

[17]  C. Ling,et al.  Answering the connectionist challenge: a symbolic model of learning the past tenses of English verbs , 1993, Cognition.

[18]  Brian D. Ripley,et al.  Statistical aspects of neural networks , 1993 .

[19]  Mark S. Seidenberg,et al.  Beyond Rules and Exceptions: A Connectionist Approach to Inflectional Morphology , 1994 .