Guidelines for developing explainable cognitive models

Cognitive models can be used to generate the behavior of virtual players in simulation-based training systems. To learn from such training, the virtual players must display realistic human behavior, and trainees need to understand why the other players behave the way they do. This understanding can be achieved by explaining the underlying reasons for the virtual players’ behavior. In this paper, it is discussed how to design cognitive models in such a way that they are able to explain the behavior they generate. Three users studies were carried out to assess what type of explanations are useful for training, and how that relates to cognitive model design. Several guidelines for developing explainable cognitive models are proposed.

[1]  W. Lewis Johnson,et al.  Agents that Learn to Explain Themselves , 1994, AAAI.

[2]  Michael van Lent,et al.  An Explainable Artificial Intelligence System for Small-unit Tactical Behavior , 2004, AAAI.

[3]  D. Dennett.  The Intentional Stance. , 1987 .

[4]  Emma Norling,et al.  Folk psychology for human modelling: extending the BDI paradigm , 2004, Proceedings of the Third International Joint Conference on Autonomous Agents and Multiagent Systems, 2004. AAMAS 2004..

[5]  Jürgen Dix,et al.  Multi-Agent Programming: Languages, Tools and Applications , 2009 .

[6]  H. Kelley.  Attribution theory in social psychology , 1967 .

[7]  B. Malle,et al.  How People Explain Behavior: A New Theoretical Framework , 1999, Personality and social psychology review : an official journal of the Society for Personality and Social Psychology, Inc.

[8]  John-Jules Ch. Meyer,et al.  A Methodology for Developing Self-explaining Agents for Virtual Training , 2009, LADS.

[9]  Maaike Harbers,et al.  Intelligent Agents for Training On-Board Fire Fighting , 2009, HCI.

[10]  F. Heider.  The psychology of interpersonal relations , 1958 .

[11]  F. Keil,et al.  Explanation and understanding , 2015 .

[12]  John-Jules Ch. Meyer,et al.  A Study into Preferred Explanations of Virtual Agent Behavior , 2009, IVA.

[13]  Lin Padgham,et al.  Hierarchical planning in BDI agent programming languages: a formal approach , 2006, AAMAS '06.

[14]  Annerieke Heuvelink.  Cognitive Models for Training Simulations , 2009 .

[15]  Luc Steels,et al.  Second-Generation Expert Systems , 1985, IEEE Expert.

[16]  Valerie L. Shalin,et al.  Cognitive task analysis , 2000 .

[17]  Mehdi Dastani,et al.  2APL: a practical agent programming language , 2008, Autonomous Agents and Multi-Agent Systems.

[18]  H. Chad Lane,et al.  Design recommendations to support automated explanation and tutoring , 2005 .

[19]  H. Chad Lane,et al.  Teaching Negotiation Skills through Practice and Reflection with Virtual Humans , 2006, Simul..