Goal and action inference for helpful robots using self as simulator

New applications are bringing robots into environments where they will have the opportunity to cooperate with humans as capable partners. A crucial element of cooperation is the ability to infer actions and goals of another by observing them. The ability to understand the intention being enacted by another is very important for anticipating the needs of and providing timely assistance to them. This thesis presents an approach to building a robot that is capable of action and goal inference that is based on the concept of Simulation Theory (a dominant theory in philosophy for how people do this). Simulation Theory argues that we exploit our own psychological responses in order to simulate others' minds to infer their mental states. With respect to action recognition and goal inference this implies that the ability to perform an action helps one to recognize when the same action is performed by others. It further implies that a robot could leverage its own action/goal representation to infer goals of others based on their actions. This implementation addresses the task of acquiring perceptual data about the physical motion of another agent and the context in which is it performed and mapping it onto the robot's own perceptual and movement repertoire. This implementation then addresses how to achieve the simulation to recognize the actions and infer the goals of the observed agent based on this movement and perceptual data. I demonstrate this skill by having the robot exhibit behaviors that address the inferred goal being enacted by the human. In principle this approach could be extended to not only infer the intentions of others, but other mental states as well (motivations, emotions, desires, beliefs, etc.). The main contribution of this work is a plausible working model of simulation theory (informed by scientific studies of autism, imitation, and the development of theory of other minds) that is able to infer the intention behind observable action and its effects. This is an important step towards building robots that can begin to understand human behavior in terms of the mental states that generate it, rather than only upon observable surface behavior. This understanding is key for cooperating with robots instead of using them as tools. Thesis Supervisor: Cynthia Breazeal Title: Assistant Professor of Media Arts and Sciences