Plan Explanations as Model Reconciliation

Recent work in explanation generation for decision making agents has looked at how unexplained behavior of autonomous systems can be understood in terms of differences in the model of the system and the human's understanding of the same, and how the explanation process as a result of this mismatch can be then seen as a process of reconciliation of these models. Existing algorithms in such settings, while having been built on contrastive, selective and social properties of explanations as studied extensively in the psychology literature, have not, to the best of our knowledge, been evaluated in settings with actual humans in the loop. As such, the applicability of such explanations to human-AI and human-robot interactions remains suspect. In this paper, we set out to evaluate these explanation generation algorithms in a series of studies in a mock search and rescue scenario with an internal semi-autonomous robot and an external human commander. During that process, we hope to demonstrate to what extent the properties of these algorithms hold as they are evaluated by humans.

[1]  Subbarao Kambhampati,et al.  Balancing Explicability and Explanation in Human-Aware Planning , 2017, AAAI Fall Symposia.

[2]  Tim Miller,et al.  Explanation in Artificial Intelligence: Insights from the Social Sciences , 2017, Artif. Intell..

[3]  Yu Zhang,et al.  Plan Explanations as Model Reconciliation: Moving Beyond Explanation as Soliloquy , 2017, IJCAI.

[4]  Yu Zhang,et al.  AI Challenges in Human-Robot Cognitive Teaming , 2017, ArXiv.

[5]  Seth Flaxman,et al.  European Union Regulations on Algorithmic Decision-Making and a "Right to Explanation" , 2016, AI Mag..

[6]  Hadas Kress-Gazit,et al.  Sorry Dave, I'm Afraid I Can't Do That: Explaining Unachievable Robot Tasks Using Natural Language , 2013, Robotics: Science and Systems.

[7]  Anca D. Dragan,et al.  Expressing Robot Incapability , 2018, 2018 13th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[8]  David E. Smith Planning as an Iterative Process , 2012, AAAI.

[9]  Subbarao Kambhampati A Classification of Plan Modification Strategies Based on Coverage and Information Requirements , 1990 .

[10]  Siddhartha S. Srinivasa,et al.  Legibility and predictability of robot motion , 2013, 2013 8th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[11]  Maria Fox,et al.  Explainable Planning , 2017, ArXiv.

[12]  Nancy J. Cooke,et al.  Interactive Team Cognition , 2013, Cogn. Sci..

[13]  Subbarao Kambhampati,et al.  Explicability? Legibility? Predictability? Transparency? Privacy? Security? The Emerging Landscape of Interpretable Agent Behavior , 2018, ICAPS.

[14]  Matthias Scheutz,et al.  "Sorry, I Can't Do That": Developing Mechanisms to Appropriately Reject Directives in Human-Robot Interactions , 2015, AAAI Fall Symposia.

[15]  T. Lombrozo The structure and function of explanations , 2006, Trends in Cognitive Sciences.

[16]  Subbarao Kambhampati,et al.  Handling Model Uncertainty and Multiplicity in Explanations via Model Reconciliation , 2018, ICAPS.

[17]  Cade Earl Bartlett Communication between Teammates in Urban Search and Rescue , 2015 .

[18]  Hadas Kress-Gazit,et al.  Towards minimal explanations of unsynthesizability for high-level robot behaviors , 2013, 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[19]  Stephanie Rosenthal,et al.  Dynamic generation and refinement of robot verbalization , 2016, 2016 25th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN).

[20]  Pat Langley,et al.  Explainable Agency for Intelligent Autonomous Systems , 2017, AAAI.

[21]  Subbarao Kambhampati,et al.  Hierarchical Expertise-Level Modeling for User Specific Robot-Behavior Explanations , 2020, AAAI.

[22]  Yu Zhang,et al.  Plan explicability and predictability for robot task planning , 2015, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[23]  Nancy J. Cooke,et al.  Measuring team situation awareness in decentralized command and control environments , 2006, Ergonomics.

[24]  Subbarao Kambhampati,et al.  Algorithms for the Greater Good! On Mental Modeling and Acceptable Symbiosis in Human-AI Collaboration , 2018, ArXiv.

[25]  J. Mathieu,et al.  The influence of shared mental models on team process and performance. , 2000, The Journal of applied psychology.

[26]  J. Dessalles,et al.  Reasoning as a lie detection device , 2011, Behavioral and Brain Sciences.

[27]  Yu Zhang,et al.  Explicable Robot Planning as Minimizing Distance from Expected Behavior , 2016, ArXiv.

[28]  T. Lombrozo Explanation and Abductive Inference , 2012 .

[29]  E. Salas,et al.  Shared mental models in expert team decision making. , 1993 .

[30]  Bradley Hayes,et al.  Improving Robot Controller Transparency Through Autonomous Policy Explanation , 2017, 2017 12th ACM/IEEE International Conference on Human-Robot Interaction (HRI.

[31]  Agnar Aamodt,et al.  Explanation in Case-Based Reasoning–Perspectives and Goals , 2005, Artificial Intelligence Review.

[32]  Jorge A. Baier,et al.  Preferred Explanations: Theory and Generation via Planning , 2011, AAAI.