Solving the Black Box Problem: A Normative Framework for Explainable Artificial Intelligence

Many of the computing systems programmed using Machine Learning are opaque: it is difficult to know why they do what they do or how they work. The Explainable Artificial Intelligence research program aims to develop analytic techniques with which to render opaque computing systems transparent, but lacks a normative framework with which to evaluate these techniques' explanatory success. The aim of the present discussion is to develop such a framework, while paying particular attention to different stakeholders' distinct explanatory requirements. Building on an analysis of 'opacity' from philosophy of science, this framework is modeled after David Marr's influential account of explanation in cognitive science. Thus, the framework distinguishes between the different questions that might be asked about an opaque computing system, and specifies the general way in which these questions should be answered. By applying this normative framework to current techniques such as input heatmapping, feature-detector identification, and diagnostic classification, it will be possible to determine whether and to what extent the Black Box Problem can be solved.

[1]  Cameron Buckner,et al.  Empiricism without magic: transformational abstraction in deep convolutional neural networks , 2018, Synthese.

[2]  Carlos Zednik.  Will Machine Learning Yield Machine Intelligence? , 2017, PT-AI.

[3]  Gary Marcus,et al.  Deep Learning: A Critical Appraisal , 2018, ArXiv.

[4]  D. Dennett.  The Intentional Stance. , 1987 .

[5]  Jeffrey L. Elman,et al.  Finding Structure in Time , 1990, Cogn. Sci..

[6]  Peter Norvig,et al.  Artificial Intelligence: A Modern Approach , 1995 .

[7]  Charles Wallis,et al.  Computation and cognition , 2003, J. Exp. Theor. Artif. Intell..

[8]  T. Shallice,et al.  The Organisation of Mind , 2011, Cortex.

[9]  Jenna Burrell,et al.  How the machine ‘thinks’: Understanding opacity in machine learning algorithms , 2016 .

[10]  Sara Veldhoen,et al.  Visualisation and 'Diagnostic Classifiers' Reveal How Recurrent and Recursive Neural Networks Process Hierarchical Structure , 2018, J. Artif. Intell. Res..

[11]  P. Churchland.  Eliminative materialism and the propositional attitudes , 1993 .

[12]  Marvin Minsky,et al.  Semantic Information Processing , 1968 .

[13]  Paul Humphreys,et al.  The philosophical novelty of computer simulation methods , 2009, Synthese.

[14]  Luciano Floridi,et al.  Why a Right to Explanation of Automated Decision-Making Does Not Exist in the General Data Protection Regulation , 2017 .

[15]  John Bickle,et al.  Reducing mind to molecular pathways: explicating the reductionism implicit in current cellular and molecular neuroscience , 2006, Synthese.

[16]  Seth Flaxman,et al.  European Union Regulations on Algorithmic Decision-Making and a "Right to Explanation" , 2016, AI Mag..

[17]  W. Ramsey.  Do Connectionist Representations Earn Their Explanatory Keep , 1997 .

[18]  Alun D. Preece,et al.  Interpretable to Whom? A Role-based Model for Analyzing Interpretable Machine Learning Systems , 2018, ArXiv.

[19]  Wojciech Samek,et al.  Methods for interpreting and understanding deep neural networks , 2017, Digit. Signal Process..

[20]  Bolei Zhou,et al.  GAN Dissection: Visualizing and Understanding Generative Adversarial Networks , 2018, ICLR.

[21]  Gualtiero Piccinini,et al.  Integrating psychology and neuroscience: functional analyses as mechanism sketches , 2011, Synthese.

[22]  Alistair Knott,et al.  Transparency in Algorithmic and Human Decision-Making: Is There a Double Standard? , 2018, Philosophy & Technology.

[23]  Gernot Rieder,et al.  Big Data: A New Empiricism and its Epistemic and Socio-Political Consequences , 2017 .

[24]  William Bechtel,et al.  Discovering Complexity: Decomposition and Localization as Strategies in Scientific Research , 2010 .

[25]  Zachary Chase Lipton.  The mythos of model interpretability , 2016, ACM Queue.

[26]  Juan M. Durán,et al.  Grounds for Trust: Essential Epistemic Opacity and Computational Reliabilism , 2018, Minds and Machines.

[27]  Ron McClamrock,et al.  Marr's three levels: A re-evaluation , 1991, Minds and Machines.

[28]  Carlos Zednik.  Mechanisms in Cognitive Science , 2017 .

[29]  Catherine Stinson,et al.  Mechanisms in psychology: ripping nature at its seams , 2015, Synthese.

[30]  Joan Bruna,et al.  Intriguing properties of neural networks , 2013, ICLR.

[31]  Carlos Guestrin,et al.  "Why Should I Trust You?": Explaining the Predictions of Any Classifier , 2016, ArXiv.

[32]  Michael Pfeiffer,et al.  Deep Learning With Spiking Neurons: Opportunities and Challenges , 2018, Front. Neurosci..

[33]  W. Freeman.  Second Commentary: On the proper treatment of connectionism by Paul Smolensky (1988) - Neuromachismo Rekindled , 1989 .

[34]  David Marr,et al.  VISION A Computational Investigation into the Human Representation and Processing of Visual Information , 2009 .

[35]  Anthony Chemero,et al.  Anti-Representationalism and the Dynamical Stance , 2000, Philosophy of Science.

[36]  Franco Turini,et al.  Local Rule-Based Explanations of Black Box Decision Systems , 2018, ArXiv.

[37]  Andy Clark,et al.  Associative Engines: Connectionism, Concepts, and Representational Change , 1993 .

[38]  Marcel van Gerven,et al.  Explanation Methods in Deep Learning: Users, Values, Concerns and Challenges , 2018, ArXiv.

[39]  Joshua B. Tenenbaum,et al.  Building machines that learn and think like people , 2016, Behavioral and Brain Sciences.

[40]  Derek Doran,et al.  What Does Explainable AI Really Mean? A New Conceptualization of Perspectives , 2017, CEx@AI*IA.

[41]  Minsuk Kahng,et al.  Visual Analytics in Deep Learning: An Interrogative Survey for the Next Frontiers , 2018, IEEE Transactions on Visualization and Computer Graphics.

[42]  Oron Shagrir,et al.  Marr on Computational-Level Theories* , 2010, Philosophy of Science.

[43]  P. Smolensky.  On the proper treatment of connectionism , 1988, Behavioral and Brain Sciences.