Inductive Reasoning: Theory-Based Bayesian Models of Inductive Reasoning

Philosophers since Hume have struggled with the logical problem of induction, but children solve an even more difficult task — the practical problem of induction. Children somehow manage to learn concepts, categories, and word meanings, and all on the basis of a set of examples that seems hopelessly inadequate. The practical problem of induction does not disappear with adolescence: adults face it every day whenever they make any attempt to predict an uncertain outcome. Inductive inference is a fundamental part of everyday life, and for cognitive scientists, a fundamental phenomenon of human learning and reasoning in need of computational explanation. There are at least two important kinds of questions that we can ask about human inductive capacities. First, what is the knowledge on which a given instance of induction is based? Second, how does that knowledge support generalization beyond the specific data observed: how do we judge the strength of an inductive argument from a given set of premises to new cases, or infer which new entities fall under a concept given a set of examples? We provide a computational approach to answering these questions. Experimental psychologists have studied both the process of induction and the nature of prior knowledge representations in depth, but previous computational models of induction have tended to emphasize process to the exclusion of knowledge representation. The approach we describe here attempts to redress this imbalance, by showing how domain-specific prior knowledge can be formalized as a crucial ingredient in a domain-general framework for rational statistical inference. The value of prior knowledge has been attested by both psychologists and machine learning theorists, but with somewhat different emphases. Formal analyses in machine learning show that meaningful generalization is not possible unless a learner begins with some sort of inductive bias: some set of constraints on the space of hypotheses that will be considered (Mitchell, 1997). However, the best known statistical machine-learning algorithms adopt relatively weak inductive biases and thus require much more data for successful generalization than humans do: tens or hundreds of positive and negative examples, in contrast to the human ability to generalize from just one or few positive examples. These machine algorithms lack ways to represent and exploit the rich forms of prior knowledge that guide people’s inductive inferences, and that have been the focus of much attention in cognitive and developmental psychology under the name of “intuitive theories” (Murphy and Medin, 1985). Murphy (1993) characterizes an intuitive theory as “a set of causal relations that collectively generate or explain the phenomena in a domain.” We think of a theory more generally as any system of abstract principles that generates hypotheses for inductive inference in a domain, such as hypotheses about the meanings of new concepts, the conditions for new rules, or the extensions of new properties in that domain. Carey (1985), Wellman and Gelman (1992), and Gopnik and Meltzoff (1997) emphasize the central role of intuitive theories in cognitive development, both as sources of constraint on children’s inductive reasoning and as the locus of deep conceptual change. Only recently have psychologists begun to consider seriously the roles that these intuitive theories might play in formal models of inductive inference (Gopnik and Schulz, 2004; Tenenbaum,

[1]  J. Kruschke,et al.  ALCOVE: an exemplar-based connectionist model of category learning. , 1992, Psychological review.

[2]  B. Ross,et al.  Predictions From Uncertain Categorizations , 1994, Cognitive Psychology.

[3]  John R. Anderson,et al.  The Adaptive Nature of Human Categorization , 1991 .

[4]  Thomas L. Griffiths,et al.  Semi-Supervised Learning with Trees , 2003, NIPS.

[5]  J. Tenenbaum,et al.  Optimal Predictions in Everyday Cognition , 2006, Psychological science.

[6]  C. Mckenzie,et al.  The psychological side of Hempel’s paradox of confirmation , 2000, Psychonomic bulletin & review.

[7]  S. Sloman Feature-Based Induction , 1993, Cognitive Psychology.

[8]  S. Atran Classifying nature across cultures , 1995 .

[9]  N. Chater,et al.  Rational models of cognition , 1998 .

[10]  H. Wellman,et al.  Cognitive development: foundational theories of core domains. , 1992, Annual review of psychology.

[11]  S. Carey Conceptual Change in Childhood , 1985 .

[12]  Ryszard S. Michalski,et al.  Categories and Concepts: Theoretical Views and Inductive Data Analysis , 1993 .

[13]  F. Ashby,et al.  Categorization as probability density estimation , 1995 .

[14]  C. Mckenzie The Accuracy of Intuitive Judgment Strategies: Covariation Assessment and Bayesian Inference , 1994, Cognitive Psychology.

[15]  Thomas G. Dietterich What is machine learning? , 2020, Archives of Disease in Childhood.

[16]  Patrick Shafto,et al.  Development of categorization and reasoning in the natural world: novices to experts, naive similarity to ecological knowledge. , 2003, Journal of experimental psychology. Learning, memory, and cognition.

[17]  J. Tenenbaum,et al.  Sensitivity to Sampling in Bayesian Word Learning We Thank Members of the Ubc Baby Cognition Lab for Their Help with Data Collection, And , 2022 .

[18]  John R. Anderson The Adaptive Character of Thought , 1990 .

[19]  E. Heit,et al.  Similarity and property effects in inductive reasoning. , 1994, Journal of experimental psychology. Learning, memory, and cognition.

[20]  Joshua B. Tenenbaum,et al.  Rules and Similarity in Concept Learning , 1999, NIPS.

[21]  C. Sumiyoshi CATEGORY BASED INDUCTION , 1997 .

[22]  D. Medin,et al.  The role of theories in conceptual coherence. , 1985, Psychological review.

[23]  Gregory L. Murphy,et al.  Theories and concept formation. , 1993 .

[24]  W.,et al.  Categories and Concepts · Theoretical Views and Inductive Data Analysis , 2009 .

[25]  Joshua B. Tenenbaum,et al.  Inferring causal networks from observations and interventions , 2003, Cogn. Sci..

[26]  J. Tenenbaum,et al.  Structure and strength in causal induction , 2005, Cognitive Psychology.

[27]  A. Gopnik,et al.  Mechanisms of theory formation in young children , 2004, Trends in Cognitive Sciences.

[28]  J. Tenenbaum,et al.  Intuitive theories as grammars for causal inference , 2007 .

[29]  N. Chater,et al.  Rational models of cognition , 1998 .

[30]  Evan Heit,et al.  A Bayesian Analysis of Some Forms of Inductive Reasoning , 1998 .

[31]  D. Kirsh,et al.  Proceedings of the 25th annual conference of the Cognitive Science Society , 2003 .

[32]  Vikash K. Mansinghka,et al.  Learning Cross-cutting Systems of Categories , 2006 .

[33]  R. Shepard,et al.  Toward a universal law of generalization for psychological science. , 1987, Science.

[34]  J. Tenenbaum,et al.  Generalization, similarity, and Bayesian inference. , 2001, The Behavioral and brain sciences.

[35]  Eldar Shafir,et al.  Similarity, plausibility, and judgments of probability , 1993, Cognition.

[36]  J. Tenenbaum,et al.  Word learning as Bayesian inference. , 2007, Psychological review.

[37]  J. Tenenbaum,et al.  Learning Domain Structures , 2004 .

[38]  Joshua B. Tenenbaum,et al.  Context-Sensitive Induction , 2005 .

[39]  E. Markman,et al.  Categories and induction in young children , 1986, Cognition.

[40]  P. J. Green,et al.  Density Estimation for Statistics and Data Analysis , 1987 .

[41]  D. Medin,et al.  A relevance theory of induction , 2003, Psychonomic bulletin & review.

[42]  Eleanor Rosch,et al.  Principles of Categorization , 1978 .

[43]  J. Tenenbaum,et al.  Opinion TRENDS in Cognitive Sciences Vol.10 No.7 July 2006 Special Issue: Probabilistic models of cognition Theory-based Bayesian models of inductive learning and reasoning , 2022 .

[44]  A. Gopnik,et al.  Words, thoughts, and theories , 1997 .

[45]  R. Nosofsky Attention, similarity, and the identification-categorization relationship. , 1986, Journal of experimental psychology. General.

[46]  Nick Chater,et al.  A rational analysis of the selection task as optimal data selection. , 1994 .

[47]  Joshua B. Tenenbaum,et al.  Theory-Based Induction , 2003 .

[48]  A. Caramazza,et al.  Curvilinear motion in the absence of external forces: naive beliefs about the motion of objects. , 1980, Science.

[49]  James L. McClelland,et al.  Semantic Cognition: A Parallel Distributed Processing Approach , 2004 .

[50]  J. Tenenbaum,et al.  From mere coincidences to meaningful discoveries , 2007, Cognition.