Explanation and trust: what to tell the user in security and AI?

There is a common problem in artificial intelligence (AI) and information security. In AI, an expert system needs to be able to justify and explain a decision to the user. In information security, experts need to be able to explain to the public why a system is secure. In both cases, an important goal of explanation is to acquire or maintain the users’ trust. In this paper, I investigate the relation between explanation and trust in the context of computing science. This analysis draws on literature study and concept analysis, using elements from system theory as well as actor-network theory. I apply the conceptual framework to both AI and information security, and show the benefit of the framework for both fields by means of examples. The main focus is on expert systems (AI) and electronic voting systems (security). Finally, I discuss consequences of the analysis for ethics in terms of (un)informed consent and dissent, and the associated division of responsibilities.

[1]  Padraig Cunningham,et al.  A Case-Based Explanation System for Black-Box Systems , 2005, Artificial Intelligence Review.

[2]  Joseph H. Park,et al.  England's Controversy over the Secret Ballot , 1931 .

[3]  Wolter Pieters,et al.  Acceptance of Voting Technology: Between Confidence and Trust , 2006, iTrust.

[4]  L. Richard Ye,et al.  The Impact of Explanation Facilities in User Acceptance of Expert System Advice , 1995, MIS Q..

[5]  England.  May 2007 Election Report Election Observation Mission , .

[6]  Eleni Kosta,et al.  Mobile-Centric Ambient Intelligence in Health- and Homecare—Anticipating Ethical and Legal Challenges , 2010, Sci. Eng. Ethics.

[7]  Pekka Nikander,et al.  Users and Trust in Cyberspace , 2000, Security Protocols Workshop.

[8]  Agnar Aamodt,et al.  Explanation in Case-Based Reasoning–Perspectives and Goals , 2005, Artificial Intelligence Review.

[9]  Peter G. Neumann,et al.  Security by obscurity , 2003, CACM.

[10]  Wolter Pieters,et al.  Reve\{a,i\}ling the risks: a phenomenology of information security , 2010 .

[11]  Mariarosaria Taddeo,et al.  Defining Trust and E-Trust: From Old Theories to New Problems , 2009, Int. J. Technol. Hum. Interact..

[12]  Eugene C. Freuder,et al.  A Case Study in Explanation and Implication , 2008 .

[13]  Wolter Pieters,et al.  Temptations of turnout and modernisation: E-voting discourses in the UK and The Netherlands , 2007, J. Inf. Commun. Ethics Soc..

[14]  R. Golembiewski.  Trust and Power Two Works by Niklas Luhmann . By Niklas Luhmann. (New York: John Wiley, 1980. Pp. xix + 208. $28.95.) , 1981, American Political Science Review.

[15]  Isaiah Berlin,et al.  Vico and Herder. Two Studies in the History of Ideas. , 1977 .

[16]  Thomas Roth-Berghofer,et al.  Mapping Goals and Kinds of Explanations to the Knowledge Containers of Case-Based Reasoning Systems , 2005, ICCBR.

[17]  Dietrich Fahrenholtz,et al.  Towards a Sociological View of Trust in Computer Science , 2001 .

[18]  John-Jules Ch. Meyer,et al.  A Study into Preferred Explanations of Virtual Agent Behavior , 2009, IVA.

[19]  T. Beauchamp,et al.  A History and Theory of Informed Consent , 1986, American Political Science Review.

[20]  Bongshin Lee,et al.  Electronic voting system usability issues , 2003, CHI '03.

[21]  Peter van den Besselaar,et al.  Security as belief user's perceptions on the security of electronic voting systems , 2004 .

[22]  Wolter Pieters,et al.  RIES - Internet Voting in Action , 2005, COMPSAC.

[23]  Carl E. Landwehr,et al.  Basic concepts and taxonomy of dependable and secure computing , 2004, IEEE Transactions on Dependable and Secure Computing.

[24]  Philip Brey,et al.  Freedom and Privacy in Ambient Intelligence , 2005, Ethics and Information Technology.

[25]  P. Verbeek.  What Things Do: Philosophical Reflections on Technology, Agency, and Design , 2005 .

[26]  Brian Randell,et al.  Voting Technologies and Trust , 2006, IEEE Security & Privacy.

[27]  Izak Benbasat,et al.  Explanations From Intelligent Systems: Theoretical Foundations and Implications for Practice , 1999, MIS Q..

[28]  Deborah L. McGuinness,et al.  Toward establishing trust in adaptive agents , 2008, IUI '08.

[29]  R. Parloff.  Will Microsoft verdict miss mark? , 2002, IEEE Spectrum.

[30]  Rebecca T. Mercuri.  A better ballot box , 2002 .

[31]  N. Luhmann.  Familiarity, Confidence, Trust: Problems and Alternatives , 2000 .

[32]  Sjouke Mauw,et al.  Foundations of Attack Trees , 2005, ICISC.

[33]  Herman T. Tavani,et al.  Genomic Research and Data-Mining Technology: Implications for Personal Privacy and Informed Consent , 2004, Ethics and Information Technology.

[34]  Gabrielle Durepos.  Reassembling the Social: An Introduction to Actor‐Network‐Theory , 2008 .

[35]  Li Chen,et al.  Trust building with explanation interfaces , 2006, IUI '06.

[36]  William A. Wallace,et al.  Trust in electronic environments , 2003, 36th Annual Hawaii International Conference on System Sciences, 2003. Proceedings of the.

[37]  Ben Shneiderman,et al.  Designing trust into online experiences , 2000, CACM.

[38]  T. Burns,et al.  Trust and power : two works , 1979 .

[39]  SørmoFrode,et al.  Explanation in Case-Based Reasoning---Perspectives and Goals , 2005 .

[40]  PietersWolter.  Explanation and trust , 2011 .