Generalization, similarity, and Bayesian inference.

Shepard has argued that a universal law should govern generalization across different domains of perception and cognition, as well as across organisms from different species or even different planets. Starting with some basic assumptions about natural kinds, he derived an exponential decay function as the form of the universal generalization gradient, which accords strikingly well with a wide range of empirical data. However, his original formulation applied only to the ideal case of generalization from a single encountered stimulus to a single novel stimulus, and for stimuli that can be represented as points in a continuous metric psychological space. Here we recast Shepard's theory in a more general Bayesian framework and show how this naturally extends his approach to the more realistic situation of generalizing from multiple consequential stimuli with arbitrary representational structure. Our framework also subsumes a version of Tversky's set-theoretic model of similarity, which is conventionally thought of as the primary alternative to Shepard's continuous metric space model of similarity and generalization. This unification allows us not only to draw deep parallels between the set-theoretic and spatial approaches, but also to significantly advance the explanatory power of set-theoretic models.

[1]  A. Tversky,et al.  Subjective Probability: A Judgment of Representativeness , 1972 .

[2]  R. Shepard,et al.  The internal representation of numbers , 1975, Cognitive Psychology.

[3]  L. Rips Inductive judgments about natural categories. , 1975 .

[4]  A. Tversky Features of Similarity , 1977 .

[5]  Roger N. Shepard,et al.  Additive clustering: Representation of similarities as combinations of discrete overlapping properties. , 1979 .

[6]  P. Arabie,et al.  Mapclus: A mathematical programming approach to fitting the adclus model , 1980 .

[7]  R N Shepard,et al.  Multidimensional Scaling, Tree-Fitting, and Clustering , 1980, Science.

[8]  K. Holyoak,et al.  Induction of category distributions: a framework for classification learning. , 1984, Journal of experimental psychology. Learning, memory, and cognition.

[9]  R. Nosofsky Attention, similarity, and the identification-categorization relationship. , 1986, Journal of experimental psychology. General.

[10]  R. Shepard,et al.  Toward a universal law of generalization for psychological science. , 1987, Science.

[11]  Eve V. Clark,et al.  The principle of contrast: A constraint on language acquisition. , 1987 .

[12]  Stuart J. Russell,et al.  Analogy by Similarity , 1988 .

[13]  Noam Chomsky,et al.  Language and problems of knowledge : the Managua lectures , 1990 .

[14]  L. Rips Similarity, typicality, and categorization , 1989 .

[15]  Edward E. Smith Concepts and induction , 1989 .

[16]  Roger N. Shepard,et al.  Connectionist Implementation of a Theory of Generalization , 1990, NIPS.

[17]  David Haussler,et al.  Bounds on the sample complexity of Bayesian learning using information theory and the VC dimension , 1991, COLT '91.

[18]  M. Gluck Stimulus Generalization and Representation in Adaptive Network Models of Category Learning , 1991 .

[19]  J. Kruschke,et al.  ALCOVE: an exemplar-based connectionist model of category learning. , 1992, Psychological review.

[20]  D. Gentner,et al.  Respects for similarity , 1993 .

[21]  David R. Shanks,et al.  Tests of an Adaptive Network Model for the Identification and Categorization of Continuous-dimension Stimuli , 1994, Connect. Sci..

[22]  J. Carroll,et al.  An alternating combinatorial optimization approach to fitting the INDCLUS and generalized INDCLUS models , 1994 .

[23]  Robert L. Goldstone The role of similarity in categorization: providing a groundwork , 1994, Cognition.

[24]  Joshua B. Tenenbaum,et al.  Learning the Structure of Similarity , 1995, NIPS.

[25]  F. Ashby,et al.  Categorization as probability density estimation , 1995 .

[26]  Thomas G. Dietterich What is machine learning? , 2020, Archives of Disease in Childhood.

[27]  Stephen Muggleton,et al.  Learning from Positive Data , 1996, Inductive Logic Programming Workshop.

[28]  Nick Chater,et al.  Representational Distortion, Similarity and the Universal Law of Generalization , 1997 .

[29]  C. Sumiyoshi CATEGORY BASED INDUCTION , 1997 .

[30]  Feldman,et al.  The Structure of Perceptual Categories , 1997, Journal of mathematical psychology.

[31]  Joshua B. Tenenbaum,et al.  Bayesian Modeling of Human Concept Learning , 1998, NIPS.

[32]  Robert L. Goldstone,et al.  The development of features in object concepts , 1998, Behavioral and Brain Sciences.

[33]  Bradley C. Love,et al.  SUSTAIN: A Model of Human Category Learning , 1998, AAAI/IAAI.

[34]  Evan Heit,et al.  A Bayesian Analysis of Some Forms of Inductive Reasoning , 1998 .

[35]  Joshua B. Tenenbaum,et al.  Rules and Similarity in Concept Learning , 1999, NIPS.

[36]  J. Tenenbaum A Bayesian framework for concept learning , 1999 .

[37]  Tomaso A. Poggio,et al.  Machine Learning, Machine Vision, and the Brain , 1999, AI Mag..

[38]  Evan Heit,et al.  Features of Similarity and Category-Based Induction , 2000 .

[39]  J. Tenenbaum,et al.  Teacakes, Trains, Taxicabs and Toxins: A Bayesian Account of Predicting the Future , 2000 .

[40]  R. Shepard Perceptual-cognitive universals as reflections of the world. , 2001, The Behavioral and brain sciences.

[41]  Refractor Vision , 2000, The Lancet.