Similarity and Rules United: Similarity- and Rule-Based Processing in a Single Neural Network

A central controversy in cognitive science concerns the roles of rules versus similarity. To gain some leverage on this problem, we propose that rule- versus similarity-based processes can be characterized as extremes in a multidimensional space that is composed of at least two dimensions: the number of features (Pothos, 2005) and the physical presence of features. The transition of similarity- to rule-based processing is conceptualized as a transition in this space. To illustrate this, we show how a neural network model uses input features (and in this sense produces similarity-based responses) when it has a low learning rate or in the early phases of training, but it switches to using self-generated, more abstract features (and in this sense produces rule-based responses) when it has a higher learning rate or is in the later phases of training. Relations with categorization and the psychology of learning are pointed out.

[1]  Mark K. Johansen,et al.  Are there representational shifts during category learning? , 2002, Cognitive Psychology.

[2]  Geoffrey E. Hinton,et al.  Learning internal representations by error propagation , 1986 .

[3]  Jonathan D. Cohen,et al.  Prefrontal cortex and flexible cognitive control: rules without symbols. , 2005, Proceedings of the National Academy of Sciences of the United States of America.

[4]  L. Rips.  Similarity, typicality, and categorization , 1989 .

[5]  T. Beckers,et al.  Further evidence for the role of inferential reasoning in forward blocking , 2005, Memory & cognition.

[6]  Jeffrey L. Elman,et al.  Finding Structure in Time , 1990, Cogn. Sci..

[7]  J. Kruschke,et al.  Rules and exemplars in category learning. , 1998, Journal of experimental psychology. General.

[8]  Edward E. Smith,et al.  Alternative strategies of categorization , 1998, Cognition.

[9]  James L. McClelland,et al.  Learning and Applying Contextual Constraints in Sentence Comprehension , 1990, Artif. Intell..

[10]  Shawn W. Ell,et al.  Procedural learning in perceptual categorization , 2003, Memory & cognition.

[11]  J. D. Smith,et al.  Overall similarity in adults' classification: The child in all of us. , 1984 .

[12]  L. Brooks,et al.  Specializing the operation of an explicit rule , 1991 .

[13]  Joshua B. Tenenbaum,et al.  Rules and Similarity in Concept Learning , 1999, NIPS.

[14]  D. Shanks,et al.  FEATURE- AND RULE-BASED GENERALIZATION IN HUMAN ASSOCIATIVE LEARNING , 1998 .

[15]  R. Nosofsky.  American Psychological Association, Inc. Choice, Similarity, and the Context Theory of Classification , 2022 .

[16]  Peter M. Vishton,et al.  Rule learning by seven-month-old infants. , 1999, Science.

[17]  John R. Anderson,et al.  The Adaptive Nature of Human Categorization. , 1991 .

[18]  S. Pinker,et al.  On language and connectionism: Analysis of a parallel distributed processing model of language acquisition , 1988, Cognition.

[19]  J D Smith,et al.  Depression and category learning. , 1993, Journal of experimental psychology. General.

[20]  Edward E. Smith,et al.  The Case for Rules in Reasoning , 1992, Cogn. Sci..

[21]  S. Sloman.  The empirical case for two systems of reasoning. , 1996 .

[22]  D. Shanks.  Associationism and cognition: Human contingency learning at 25 , 2007, Quarterly journal of experimental psychology.

[23]  Ralph R. Miller,et al.  Reasoning rats: forward blocking in Pavlovian animal conditioning is sensitive to constraints of causal inference. , 2006, Journal of experimental psychology. General.

[24]  James L. McClelland,et al.  A distributed, developmental model of word recognition and naming. , 1989, Psychological review.

[25]  Michael R. Waldmann,et al.  Causal Reasoning in Rats , 2006, Science.

[26]  James L. McClelland,et al.  A distributed model of human learning and memory , 1986 .

[27]  James L. McClelland,et al.  Understanding normal and impaired word reading: computational principles in quasi-regular domains. , 1996, Psychological review.

[28]  Wim Fias,et al.  Lexical and syntactic structures in a connectionist model of reading multi-digit numbers , 2006, Connect. Sci..

[29]  James A. Hampton,et al.  Testing the Prototype Theory of Concepts , 1995 .

[30]  R. Nosofsky,et al.  Rule-plus-exception model of classification learning. , 1994, Psychological review.

[31]  James L. McClelland,et al.  Rules or connections in past-tense inflections: what does the evidence rule out? , 2002, Trends in Cognitive Sciences.

[32]  J. Pearce,et al.  Theories of associative learning in animals. , 2001, Annual review of psychology.

[33]  X. LingCharles.  Learning the past tense of English verbs , 1994 .

[34]  James L. McClelland,et al.  Parallel distributed processing: explorations in the microstructure of cognition, vol. 1: foundations , 1986 .

[35]  E. Pothos.  The rules versus similarity distinction. , 2005, The Behavioral and brain sciences.

[36]  Derek C. Penn,et al.  Causal cognition in human and nonhuman animals: a comparative, critical review. , 2007, Annual review of psychology.

[37]  J. Elman,et al.  Networks are not ‘hidden rules’ , 1999, Trends in Cognitive Sciences.

[38]  D. Medin,et al.  SUSTAIN: a network model of category learning. , 2004, Psychological review.

[39]  K. Kurtz.  The divergent autoencoder (DIVA) model of category learning , 2007 .

[40]  Noah D. Goodman,et al.  Learning Causal Schemata , 2007 .

[41]  Robert L. Goldstone,et al.  How abstract is symbolic thought? , 2007, Journal of experimental psychology. Learning, memory, and cognition.

[42]  D. Garlick.  Understanding the nature of the general factor of intelligence: the role of individual differences in neural plasticity as an explanatory mechanism. , 2002, Psychological review.

[43]  James L. McClelland,et al.  Graded state machines: The representation of temporal contingencies in simple recurrent networks , 1991, Machine Learning.

[44]  Edward E. Smith.  Rule and similarity as prototype concepts , 2005 .