Theory-Based Induction

We show how an abstract domain theory can be incorporated into a rational statistical model of induction. In particular, we describe a Bayesian model of category-based induction, and generate the prior distribution for our model using a formal theory of the distribution of biological properties across classes of biological kinds. Our theory-based model is both more principled than previous approaches and better able to account for human ratings of argument strength. Philosophers since Hume have struggled with the logical problem of induction, but children solve an even more difficult task — the practical problem of induction. Children somehow manage to learn concepts, categories, and word meanings, and all on the basis of a set of examples that seems hopelessly inadequate. The practical problem of induction does not disappear with adolescence: adults face it every day whenever they make any attempt to predict an uncertain outcome. Inductive inference is a fundamental part of everyday life, and a fundamental phenomenon in need of a psychological explanation. Two important questions can be asked about inductive generalization: what resources does a person bring to an inductive task, and how are these resources combined to generate a response to the demands of the task? In other words, what is the process of induction, and what is the prior knowledge required by this process? Psychologists have considered both of these questions in depth, but previous computational models of induction have tended to emphasize process to the exclusion of prior knowledge. This paper attempts to redress this imbalance by showing how prior knowledge can be included in a computational model founded on rational statistical inference. The importance of prior knowledge has been attested by psychologists and machine learning theorists alike. Murphy and Medin (1985) have suggested that the acquisition of new concepts is guided by “theories” — networks of explanatory connections between existing concepts. Machine learning theorists have built formal models of learning, and argued that generalization within these models is not possible unless a learner begins with some sort of inductive bias (Mitchell, 1997). The challenge that inspires our work is to develop a model with an inductive bias that is well motivated by a theory of the domain under consideration. Many previous models have taken similarity judgments as their representation of prior knowledge (Nosofsky, 1986; Osherson et al., 1990). This approach has been dominant within the tradition of category-based induction, and Osherson et al.’s (1990) similarity-coverage model will be one standard against which our new model will be compared. Using similarity data to represent prior knowledge is a reasonable first attempt, but similarity judgments are less than ideal as a starting point for a model of inductive inference. As Goodman (1972) has pointed out, similarity is a vague and elusive notion. It is meaningless to say that two objects are similar unless a respect for similarity has been specified. Any model based on similarity alone is therefore a model without a secure foundation. Instead of relying on similarity, the model developed in this paper is founded on a simple theory of a particular domain of reasoning: kinds of animals and their properties. The theory consists of two components: the ‘taxonomic principle,’ which holds that the set of animals naturally forms a treestructured taxonomy, and the ‘distribution principle,’ which specifies how properties are probabilistically distributed over the taxonomy. These two principles are used to generate a prior distribution for a Bayesian model of category-based induction. Our approach is located at the most abstract of Marr’s three levels: the level of computational theory (Marr, 1982). Our goal is not to describe the process by which people make inductive inferences, but rather to explain why people reason the way that they do and how they can reliably come to true beliefs about the world from very limited data. Intriguingly, both the taxonomic principle and the distributional principle resemble analogous principles in evolutionary biology and genetics. People’s remarkable ability to make successful inductive leaps may thus be explained as the result of rational inference mechanisms operating under the guidance of a domain theory that reflects the true structure of the environment. We begin by introducing previous approaches to the problem of category-based induction. We then set out a ‘theory of biological properties’ that can generate the prior distribution for a Bayesian model of induction. Next we turn to experimental data, and show that our new model performs better than previous approaches across a collection of four data sets. We conclude by discussing ways in which Bayesian and traditional similarity-based approaches might be complementary, and directions for future work. Category-Based Induction The tasks to be considered were introduced by Osherson et al. (1990). In the first task (the specific inference task), subjects are asked to rate the strength of arguments of the following form: Horses can get disease X Cows can get disease X Dolphins can get disease X The premises state that one or more specific mammals can catch a certain disease, and the conclusion (to be evaluated) states that another specific species (here dolphins) can also catch the disease. In the second task (the general inference task), subjects are asked to consider a generalization from specific premises to a property of all mammals. For instance: Seals can get disease X Dolphins can get disease X All mammals can get disease X