Abstract knowledge guides search and prediction in novel situations

Abstract knowledge guides search and prediction in novel situations Russell E. Warner 1 , Patrick Shafto 1 , Chris Baker 2 , & Joshua B. Tenenbaum 2 Department of Psychological and Brain Sciences, University of Louisville Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology structured stochastic environments (especially so-called bandit problems) and investigated how people develop strategies in these situations (Steyvers, Lee, & Wagen- makers, 2008; Acu˜ na & Schrater, 2008; Gittins, 1979). However, this work has not addressed how people use abstract knowledge to guide reasoning about novel situ- ations in familiar environments. In fact, Steyvers et al. (2008) specifically suggest the incorporation of a learning element into these problems would be a useful extension of the work. In this paper, we propose and test a frame- work for just such a purpose; showing how people use experience to guide search and prediction. Abstract People combine their abstract knowledge about the world with data they have gathered in order to guide search and prediction in everyday life. We present a Bayesian model that formalizes knowledge transfer. Our model consists of two components: a hierarchical Bayesian model of learning and a Markov Decision Pro- cess modeling planning and search. An experiment tests qualitative predictions of the model, showing a strong fit between human data and model predictions. We con- clude by discussing relations to previous work and future directions. People combine their abstract knowledge about the world with data they have gathered in order to guide search and prediction in everyday life. Rather than sim- ply treat every search as a novel event, individuals ap- ply knowledge from previous experiences to make pre- dictions about new situations. Using knowledge about distribution of items across instances, such as a prizes in a cracker jack boxes, individuals have expectations about how many times you will have to reach in to get the prize. Because cracker jacks boxes have more popped corn and peanuts than prizes, you might use that knowledge to predict that any given hand-full will only contain popped corn and peanuts. Now imagine an alternative scenario in which you got your cracker jacks box from a discount bin at the factory. If we found a few boxes contained almost 50% prizes, we could generate a different expec- tation about the process that created the boxes, and use this knowledge to make predictions about what we will pull out of the box next, and what we might expect from the next box. Here knowledge about boxes, and the pro- cesses that generate boxes allow us to make predictions about novel situations. In general, knowledge at multiple levels of abstraction allows us to generate expectations and predictions for novel situations. Recent research has provided insight into some as- pects of this ability to transfer knowledge. Hierarchi- cal Bayesian models (HBMs) have been used to model how people can combine past experience with sparse data in a novel situation to make confident and accurate predictions. For example, Kemp, Perfors, and Tenen- baum (2007) showed that hierarchical Bayesian models can capture how children’s experience with with nouns causes them to infer that objects with the same name tend to have the same shape, allowing generalization of novel names from only one or two examples. A sepa- rate line of research has focused on prediction in un- A Bayesian model of search and prediction in novel situations Our approach to modeling how people use previous ex- perience to guide inference and search in novel situations combines the previously mentioned hierarchical Bayesian models with a Markov Decision Process (MDP). Hierar- chical Bayesian models provide a representation of how experience can affect beliefs at multiple levels of abstrac- tion, allowing predictions about novel situations (Kemp et al., 2007). Markov decision processes provide a model of search and decision making (Russell & Norvig, 2002). We briefly introduce each approach and then describe how to integrate the two approaches, using a simplified problem to develop intuitions. Instead of cracker jacks boxes, which have predictable mixture proportions, imagine a discount store that sells boxes of cookies that don’t have predictable mixtures. A clerk at the discount store suggests a game in which, if a boy correctly guesses the next cookie drawn from a box, he gets to keep the box. However, the boy’s mother is about to leave, putting the boy at risk of not getting any- thing. What should he do? In the case of discount store boxes, which are not well-mixed, if the boy can draw one example before guessing, he can dramatically increase his chance of guessing the next cookie correctly. But if the boy were at a regular grocery store, where the boxes were well-mixed, asking to see one sample would give him no information – his chances of guessing correctly remain at 50/50. With our model, we can represent the boy’s abstract knowledge about mixtures of boxes, which in Figure 1 is Level 3 knowledge. And we can further show how this abstract knowledge should inform his decision to draw or guess.