Designs for explaining intelligent agents

Explanation is an important capability for usable intelligent systems, including intelligent agents and cognitive models embedded within simulations and other decision support systems. Explanation facilities help users understand how and why an intelligent system possesses a given structure and set of behaviors. Prior research has resulted in a number of approaches to provide explanation capabilities and identified some significant challenges. We describe designs that can be reused to create intelligent agents capable of explaining themselves. The designs include ways to provide ontological, mechanistic, and operational explanations. These designs inscribe lessons learned from prior research and provide guidance for incorporating explanation facilities into intelligent systems. The designs are derived from both prior research on explanation tool design and from the empirical study reported here on the questions users ask when working with an intelligent system. We demonstrate the use of these designs through examples implemented using the Herbal high-level cognitive modeling language. These designs can help build better agents-they support creating more usable and more affordable intelligent agents by encapsulating prior knowledge about how to generate explanations in concise representations that can be instantiated or adapted by agent developers.

[1]  Bernd Bruegge,et al.  Object-Oriented Software Engineering: Using UML, Patterns and Java, Second Edition , 2003 .

[2]  M. Resnik,et al.  Aspects of Scientific Explanation. , 1966 .

[3]  Johanna D. Moore,et al.  Explanations in knowledge systems: design for explainable expert systems , 1991, IEEE Expert.

[4]  Edward H. Shortliffe,et al.  Rule Based Expert Systems: The Mycin Experiments of the Stanford Heuristic Programming Project (The Addison-Wesley series in artificial intelligence) , 1984 .

[5]  K. Suzanne Barber,et al.  Comprehending agent software , 2005, AAMAS '05.

[6]  Frank E. Ritter,et al.  A Tutorial on Herbal: A High-Level Language and Development Environment Based on Protégé for Developing Cognitive Models in Soar , 2005 .

[7]  John M. Carroll,et al.  Reconstructing minimalism , 1997, SIGDOC '97.

[8]  Simon French,et al.  Explaining and justifying the advice of a decision support system: a natural language generation approach , 2003, Expert Syst. Appl..

[9]  Arthur C. Graesser,et al.  Questions and information systems , 1992 .

[10]  A. Coughlan,et al.  International Market Entry and Expansion via Independent or Integrated Channels of Distribution , 1987 .

[11]  Diana E. Forsythe,et al.  Using ethnography in the design of an explanation system , 1995 .

[12]  John M. Carroll.  Minimalism beyond the Nurnberg funnel , 1998 .

[13]  Wendy Grace Lehnert,et al.  The Process of Question Answering , 2022 .

[14]  Christopher Alexander,et al.  The Timeless Way of Building , 1979 .

[15]  Joseph A. Bellizzi,et al.  Purchasing agent's influence in the buying process , 1980 .

[16]  Johanna D. Moore,et al.  Generating explanations in context , 1993, IUI '93.

[17]  C. Hempel,et al.  Aspects of Scientific Explanation and Other Essays in the Philosophy of Science. , 1966 .

[18]  Frank E. Ritter,et al.  Using Multidisciplinary Expert Evaluations to Test and Improve Cognitive Model Interfaces , 2002 .

[19]  Paul J. Feltovich,et al.  Expertise in context: human and machine , 1997 .

[20]  Robert A. Wilson,et al.  Explanation and Cognition , 2000 .

[21]  A. Newell.  Unified Theories of Cognition , 1990 .

[22]  C. Antaki.  Explaining and Arguing: The Social Organization of Accounts , 1994 .

[23]  A. Graesser,et al.  Mechanisms that generate questions , 1992 .

[24]  Cristina Conati,et al.  Providing adaptive support to the understanding of instructional material , 2001, IUI '01.

[25]  Bernd Bruegge,et al.  Object-Oriented Software Engineering Using UML, Patterns, and Java , 2009 .

[26]  Jonathan Grudin,et al.  Why CSCW Applications Fail: Problems in the Design and Evaluation of Organization of Organizational Interfaces. , 1988 .

[27]  R HaynesSteven,et al.  Designs for explaining intelligent agents , 2009 .

[28]  John Yen,et al.  CAST: Collaborative Agents for Simulating Teamwork , 2001, IJCAI.

[29]  Allen Newell,et al.  SOAR: An Architecture for General Intelligence , 1987, Artif. Intell..

[30]  Wujin Chu,et al.  Industrial Purchasing: An Empirical Exploration of the Buyclass Framework , 1987 .

[31]  Lance A. Miller,et al.  Review of The process of question answering: a computer simulation of cognition by Wendy G. Lehnert. Lawrence Erlbaum Associates 1978. , 1980 .

[32]  William J. Clancey,et al.  The Epistemology of a Rule-Based Expert System - A Framework for Explanation , 1981, Artif. Intell..

[33]  Frank E. Ritter,et al.  Herbal: A high-level language and development environment for developing cognitive models in Soar , 2005 .

[34]  G.P. Morgan,et al.  Increasing efficiency of the development of user models , 2005, 2005 IEEE Design Symposium, Systems and Information Engineering.

[35]  James A. Hendler,et al.  The Semantic Web" in Scientific American , 2001 .

[36]  Johanna D. Moore,et al.  Planning Text for Advisory Dialogues , 1989, ACL.

[37]  Frederick P. Brooks,et al.  No Silver Bullet: Essence and Accidents of Software Engineering , 1987 .

[38]  Joseph C. Pitt,et al.  Theories of Explanation , 1988 .

[39]  C. Hempel,et al.  Studies in the Logic of Explanation , 1948, Philosophy of Science.

[40]  M. Lombard,et al.  Content Analysis in Mass Communication: Assessment and Reporting of Intercoder Reliability , 2002 .

[41]  R. Sun.  Cognition and Multi-Agent Interactions: From Cognitive Modeling to Social Simulation , 2005 .

[42]  L. Richard Ye,et al.  The Impact of Explanation Facilities in User Acceptance of Expert System Advice , 1995, MIS Q..

[43]  Murray Silverstein,et al.  A Pattern Language , 1977 .

[44]  Izak Benbasat,et al.  Explanations From Intelligent Systems: Theoretical Foundations and Implications for Practice , 1999, MIS Q..

[45]  Deborah L. McGuinness,et al.  Infrastructure for Web Explanations , 2003, SEMWEB.

[46]  Glenn Taylor,et al.  VISTA : A Generic Toolkit for Visualizing Agent Behavior , 2002 .

[47]  Alistair G. Sutcliffe,et al.  The domain theory - patterns for knowledge and software reuse , 2002 .

[48]  G Stix,et al.  The mice that warred. , 2001, Scientific American.

[49]  John Riedl,et al.  Explaining collaborative filtering recommendations , 2000, CSCW '00.

[50]  John R Anderson,et al.  An integrated theory of the mind. , 2004, Psychological review.

[51]  Kristiina Jokinen,et al.  Explanation and Interaction: The computer generation of Explanatory Dialogues , 1996, Machine Translation.

[52]  B. Chandrasekaran,et al.  Explanations in knowledge systems: the role of explicit representation of design knowledge , 1991, IEEE Expert.

[53]  Jon Elster,et al.  Explaining technical change : a case study in the philosophy of science , 1985 .

[54]  Johanna D. Moore,et al.  Planning Text for Advisory Dialogues: Capturing Intentional and Rhetorical Information , 1993, CL.

[55]  Christian Lebiere,et al.  Cognition and Multi-Agent Interaction: From Cognitive Modeling to Social Simulation , 2006 .

[56]  Deborah L. McGuinness,et al.  Explaining answers from the Semantic Web: the Inference Web approach , 2004, J. Web Semant..

[57]  Ian S. Graham,et al.  A pattern language for Web usability , 2003 .

[58]  S. Banbury,et al.  What do patients want to know: An empirical approach to explanation generation and validation , 1995 .

[59]  Michael J. Prietula,et al.  The Turing effect: the nature of trust in expert systems advice , 1997 .

[60]  J. Woodward,et al.  Scientific Explanation and the Causal Structure of the World , 1988 .

[61]  Ronald J. Brachman,et al.  Systems That Know What They're Doing , 2002, IEEE Intell. Syst..

[62]  B. V. Fraassen,et al.  The Pragmatics of Explanation , 1977 .

[63]  W. Lewis Johnson,et al.  Agents that Learn to Explain Themselves , 1994, AAAI.

[64]  Ralph Johnson,et al.  design patterns elements of reusable object oriented software , 2019 .

[65]  Frank E. Ritter,et al.  Cognition and Multi-Agent Interaction: Including Human Variability in a Cognitive Architecture to Improve Team Simulation , 2005 .

[66]  Thomas P. Moran,et al.  Design rationale: the argument behind the artifact , 1989, CHI '89.

[67]  Michael Colclough.  The Process of Question Answering — A Computer Simulation of Cognition , 1979 .

[68]  S. Toulmin,et al.  Philosophy of Science , 1970, The British Journal for the History of Science.

[69]  Izak Benbasat,et al.  The Use and Effects of Knowledge-Based System Explanations: Theoretical Foundations and a Framework for Empirical Evaluation , 1996, Inf. Syst. Res..

[70]  Randolph M. Jones,et al.  Automated Intelligent Pilots for Combat Flight Simulation , 1998, AI Mag..

[71]  Mihhail Matskin,et al.  Implementing explanation ontology for agent system , 2003, Proceedings IEEE/WIC International Conference on Web Intelligence (WI 2003).

[72]  Michael Wolverton,et al.  Presenting Significant Information in Expert System Explanation , 1995, EPIA.

[73]  D. Dennett.  The Intentional Stance. , 1987 .

[74]  James Coplien,et al.  Industrial experience with design patterns , 1996, Proceedings of IEEE 18th International Conference on Software Engineering.

[75]  John M. Carroll.  Choosing a Minimalist Approach for Expert Users , 1998 .

[76]  John M. Carroll,et al.  Minimalism for Complex Tasks , 1998 .

[77]  Yoram Reich,et al.  Measuring the value of knowledge , 1995, Int. J. Hum. Comput. Stud..