A Bayesian Account of Measures of Interpretability in Human-AI Interaction

Existing approaches for the design of interpretable agent behavior consider different measures of interpretability in isolation. In this paper we posit that, in the design and deployment of human-aware agents in the real world, notions of interpretability are just some among many considerations; and the techniques developed in isolation lack two key properties to be useful when considered together: they need to be able to 1) deal with their mutually competing properties; and 2) an open world where the human is not just there to interpret behavior in one specific form. To this end, we consider three well-known instances of interpretable behavior studied in existing literature -- namely, explicability, legibility, and predictability -- and propose a revised model where all these behaviors can be meaningfully modeled together. We will highlight interesting consequences of this unified model and motivate, through results of a user study, why this revision is necessary.

[1]  Chris L. Baker,et al.  Action understanding as inverse planning , 2009, Cognition.

[2]  Subbarao Kambhampati,et al.  Explicability? Legibility? Predictability? Transparency? Privacy? Security? The Emerging Landscape of Interpretable Agent Behavior , 2018, ICAPS.

[3]  Tim Miller,et al.  Explanation in Artificial Intelligence: Insights from the Social Sciences , 2017, Artif. Intell..

[4]  Subbarao Kambhampati,et al.  Explicable Planning as Minimizing Distance from Expected Behavior , 2019, AAMAS.

[5]  Subbarao Kambhampati,et al.  Designing Environments Conducive to Interpretable Robot Behavior , 2020, ArXiv.

[6]  Pat Langley,et al.  Explainable Agency for Intelligent Autonomous Systems , 2017, AAAI.

[7]  Erez Karpas,et al.  Privacy Preserving Plans in Partially Observable Environments , 2016, IJCAI.

[8]  Chris L. Baker,et al.  Goal Inference as Inverse Planning , 2007 .

[9]  Subbarao Kambhampati,et al.  Expectation-Aware Planning: A Unifying Framework for Synthesizing and Executing Self-Explaining Plans for Human-Aware Planning , 2019 .

[10]  Anca D. Dragan,et al.  Probabilistically Safe Robot Planning with Confidence-Based Human Predictions , 2018, Robotics: Science and Systems.

[11]  Tathagata Chakraborti,et al.  Foundations of Human-Aware Planning - A Tale of Three Models , 2018 .

[12]  Yu Zhang,et al.  Plan explicability and predictability for robot task planning , 2015, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[13]  Anca D. Dragan,et al.  Where Do You Think You're Going?: Inferring Beliefs about Dynamics from Behavior , 2018, NeurIPS.

[14]  Siddhartha S. Srinivasa,et al.  Effects of Robot Motion on Human-Robot Collaboration , 2015, 2015 10th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[15]  Sebastian Sardiña,et al.  Deceptive Path-Planning , 2017, IJCAI.

[16]  Subbarao Kambhampati,et al.  Hierarchical Expertise Level Modeling for User Specific Contrastive Explanations , 2018, IJCAI.

[17]  Charles Kemp,et al.  Bayesian models of cognition , 2008 .

[18]  Manuela Veloso,et al.  Generation of Policy-Level Explanations for Reinforcement Learning , 2019, AAAI.

[19]  Miquel Ramírez,et al.  Action Selection for Transparent Planning , 2018, AAMAS.

[20]  David Gunning,et al.  DARPA's explainable artificial intelligence (XAI) program , 2019, IUI.

[21]  Miquel Ramírez,et al.  Model Recognition as Planning , 2019, ICAPS.

[22]  Subbarao Kambhampati,et al.  A Unified Framework for Planning in Adversarial and Cooperative Environments , 2018, AAAI.

[23]  Anca D. Dragan,et al.  Generating Plans that Predict Themselves , 2018, WAFR.

[24]  Siddhartha S. Srinivasa,et al.  Legibility and predictability of robot motion , 2013, 2013 8th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[25]  Anca D. Dragan,et al.  Robot Planning with Mathematical Models of Human State and Action , 2017, ArXiv.

[26]  Yu Zhang,et al.  Plan Explanations as Model Reconciliation: Moving Beyond Explanation as Soliloquy , 2017, IJCAI.

[27]  Subbarao Kambhampati,et al.  A Case Study of XAIP in a Model Acquisition Task for Dialogue Planning , 2020 .

[28]  Tim Miller,et al.  Explainable Reinforcement Learning Through a Causal Lens , 2019, AAAI.

[29]  Ronald Fagin,et al.  Reasoning about knowledge , 1995 .

[30]  Rachel K. E. Bellamy,et al.  Planning and visualization for a smart meeting room assistant , 2019, AI Commun..

[31]  Kevin Crowston,et al.  Amazon Mechanical Turk: A Research Tool for Organizations and Information Systems Scholars , 2012, Shaping the Future of ICT Research.

[32]  Subbarao Kambhampati,et al.  The Emerging Landscape of Explainable Automated Planning & Decision Making , 2020, IJCAI.

[33]  Tim Miller,et al.  Contrastive explanation: a structural-model approach , 2018, The Knowledge Engineering Review.

[34]  Siddhartha S. Srinivasa,et al.  Generating Legible Motion , 2013, Robotics: Science and Systems.

[35]  F. E. H. N. Wijermans.  The Cambridge Handbook of Computational Psychology , 2009 .

[36]  Carlos Guestrin,et al.  "Why Should I Trust You?": Explaining the Predictions of Any Classifier , 2016, ArXiv.

[37]  Shlomo Zilberstein,et al.  Maximizing Plan Legibility in Stochastic Environments , 2020, AAMAS.

[38]  Susanne Biundo-Stephan,et al.  Making Hybrid Plans More Clear to Human Users - A Formal Approach for Generating Sound Explanations , 2012, ICAPS.