Modeling the Mistakes of Boundedly Rational Agents Within a Bayesian Theory of Mind

When inferring the goals that others are trying to achieve, people intuitively understand that others might make mistakes along the way. This is crucial for activities such as teaching, offering assistance, and deciding between blame or forgiveness. However, Bayesian models of theory of mind have generally not accounted for these mistakes, instead modeling agents as mostly optimal in achieving their goals. As a result, they are unable to explain phenomena like locking oneself out of one’s house, or losing a game of chess. Here, we extend the Bayesian Theory of Mind framework to model boundedly rational agents who may have mistaken goals, plans, and actions. We formalize this by modeling agents as probabilistic programs, where goals may be confused with semantically similar states, plans may be misguided due to resource-bounded planning, and actions may be unintended due to execution errors. We present experiments eliciting human goal inferences in two domains: (i) a gridworld puzzle with gems locked behind doors, and (ii) a block-stacking domain. Our model better explains human inferences than alternatives, while generalizing across domains. These findings indicate the importance of modeling others as bounded agents, in order to account for the full richness of human intuitive psychology.

[1]  H. Wellman,et al.  Explaining human movements and actions: Children's understanding of the limits of psychological explanation , 1997, Cognition.

[2]  Joshua B. Tenenbaum,et al.  Outcome or Strategy? A Bayesian Model of Intelligence Attribution , 2016, CogSci.

[3]  Joshua Lee,et al.  Getting to Know One Another: Calibrating Intent, Capabilities and Trust for Human-Robot Collaboration , 2020, 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[4]  R. Dechter,et al.  Heuristics, Probability and Causality. A Tribute to Judea Pearl , 2010 .

[5]  David J. Israel,et al.  Plans and resource‐bounded practical reasoning , 1988, Comput. Intell..

[6]  F. Cushman Crime and punishment: Distinguishing the roles of causal and intentional analyses in moral judgment , 2008, Cognition.

[7]  Hector Geffner,et al.  Probabilistic Plan Recognition Using Off-the-Shelf Classical Planners , 2010, AAAI.

[8]  Joshua B. Tenenbaum,et al.  The Naïve Utility Calculus: Computational Principles Underlying Commonsense Psychology , 2016, Trends in Cognitive Sciences.

[9]  R. Saxe,et al.  The neural basis of the interaction between theory of mind and moral judgment , 2007, Proceedings of the National Academy of Sciences.

[10]  Anca D. Dragan,et al.  LESS is More: Rethinking Probabilistic Models of Human Behavior , 2020, 2020 15th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[11]  Henry M. Wellman,et al.  Theory of Mind for Learning and Teaching: The Nature and Role of Explanation. , 2004 .

[12]  H. Wimmer,et al.  Beliefs about beliefs: Representation and constraining function of wrong beliefs in young children's understanding of deception , 1983, Cognition.

[13]  Chris L. Baker,et al.  Action understanding as inverse planning , 2009, Cognition.

[14]  Craig A. Knoblock,et al.  PDDL-the planning domain definition language , 1998 .

[15]  M. Tomasello,et al.  Altruistic Helping in Human Infants and Young Chimpanzees , 2006, Science.

[16]  Noah D. Goodman,et al.  Learning the Preferences of Ignorant, Inconsistent Agents , 2015, AAAI.

[17]  Falk Lieder,et al.  A resource-rational analysis of human planning , 2018, CogSci.

[18]  Owain Evans Learning the Preferences of Bounded Agents , 2015 .

[19]  Joshua B. Tenenbaum,et al.  Ten-month-old infants infer the value of goals from the costs of actions , 2017, Science.

[20]  Anind K. Dey,et al.  Maximum Entropy Inverse Reinforcement Learning , 2008, AAAI.

[21]  James T. Townsend,et al.  Sequential sampling models of choice: Some recent advances , 2008 .

[22]  Bernhard Nebel,et al.  The FF Planning System: Fast Plan Generation Through Heuristic Search , 2011, J. Artif. Intell. Res..

[23]  Eyal Amir,et al.  Bayesian Inverse Reinforcement Learning , 2007, IJCAI.

[24]  F. Cushman,et al.  How We Know What Not To Think , 2019, Trends in Cognitive Sciences.

[25]  Anca D. Dragan,et al.  Where Do You Think You're Going?: Inferring Beliefs about Dynamics from Behavior , 2018, NeurIPS.

[26]  Chris L. Baker,et al.  Rational quantitative attribution of beliefs, desires and percepts in human mentalizing , 2017, Nature Human Behaviour.

[27]  Joshua B. Tenenbaum,et al.  Online Bayesian Goal Inference for Boundedly-Rational Planning Agents , 2020, NeurIPS.