Gaze Following as Goal Inference: A Bayesian Model

Gaze Following as Goal Inference: A Bayesian Model Abram L. Friesen and Rajesh P. N. Rao {afriesen, rao}@cs.washington.edu Department of Computer Science and Engineering University of Washington, Box 352350 Seattle, WA 98195 USA Abstract The ability to follow the gaze of another human plays a crit- ical role in cognitive development. Infants as young as 12 months old have been shown to follow the gaze of adults. Re- cent experimental results indicate that gaze following is not merely an imitation of head movement. We propose that chil- dren learn a probabilistic model of the consequences of their movements, and later use this learned model of self as a surro- gate for another human. We introduce a Bayesian model where gaze following occurs as a consequence of goal inference in a learned probabilistic graphical model. Bayesian inference over this learned model provides both an estimate of another’s fix- ation location and the appropriate action to follow their gaze. The model can be regarded as a probabilistic instantiation of Meltzoff’s “Like me” hypothesis. We present simulation re- sults based on a nonparametric Gaussian process implemen- tation of the model, and compare the model’s performance to infant gaze following results. Keywords: cognitive development; machine learning; artifi- cial intelligence; goal inference; Bayesian modeling; gaze fol- lowing. Introduction Gaze following plays an important role in cognitive develop- ment. Following the gaze of an adult, for example, allows a child to jointly attend to an object, learn its name and other properties, as well as learn useful actions to perform on the object through imitation. It has been shown that children as young as 12 months old can follow the gaze of an adult and engage in joint attention (Brooks & Meltzoff, 2002). Recent results have shown that gaze following is not merely an imitation of head movement. For example, 14- and 18-month olds do not follow the gaze of an adult who is wearing a blindfold, although they follow gaze if the adult wears the same band as a headband. This suggests that these children do not follow gaze because they are aware of the consequences of wearing a blindfold (i.e., occlusion) and unlike 12-month olds, make the inference that the adult is not looking at an object. This observation is closely related to Meltzoff’s “Like me” hypothesis (Meltzoff, 2005) which states that self-experience plays an important role in making inferences about the internal states of others. In particular, in the case of the blindfold experiment, self-experience with own eye closure and occluders may influence gaze following behavior. To test this hypothesis, Meltzoff and Brooks provided one group of 12-month olds with self-experience with an opaque blindfold while two other groups either had no self-experience or had self-experience with a windowed blindfold. On seeing an adult with a blindfold turn towards an object, most of the children who had had self-experience with blindfolds did not turn to the object while the other two groups did (Meltzoff & Brooks, 2008). These results suggest that (a) gaze following involves an inference of the underlying intention or goal of the head movement, and (b) self-experience plays a major role in learning the consequences of intentions and related actions. In this paper, we propose a new model for gaze following and joint attention that can be viewed as a probabilistic instan- tiation of the “Like me” hypothesis. The model itself is gen- eral and can be applied to modeling other forms of goal-based imitation, but we focus here on gaze following. In the fol- lowing section, we derive our framework for gaze following based on probabilistic graphical models. We describe how a child could learn a probabilistic model of the consequences of their own head movements, and later use this learned model to interpret the actions of another person. Bayesian inference over the learned graphical model provides both an estimate of another’s fixation location and the appropriate action to move one’s own gaze for joint attention. For the simulations, a model based on Gaussian process regression was used to learn the mapping between goals, actions, and their sensory consequences. We present preliminary results comparing the model to infant gaze following results and discuss the appli- cability of the proposed framework for understanding other forms of goal-based imitation and sensorimotor planning. A Bayesian Model for Gaze-Following In the following section, we develop and explain our model for gaze-following as goal inference. We begin with a sim- plified explanation of our graphical model, and then give an overview of the computational components. Hypothesis Our hypothesis is the following: humans learn a goal-directed mechanism for planning gaze movements. A goal location, provided by either internal or external stimuli, combined with the current state, determines an action. This action, again in conjunction with the current state, determines the final state. We represent this mechanism with the graphical model shown in Figure 1(a) where G is the goal, A is the action, X i is the current state, and X f is the final state. In the context of gaze following, the goal is a desired fixation location, the action is a vector of motor commands, and the state represents head position and orientation. With our proposed model, an artificial agent can both plan future movements given desired fixation points and determine fixation points given observed head poses. Both of these cor- respond to performing inference over the graphical model.