Outcome-Explorer: A Causality Guided Interactive Visual Interface for Interpretable Algorithmic Decision Making

The widespread adoption of algorithmic decision-making systems has brought about the necessity to interpret the reasoning behind these decisions. The majority of these systems are complex black box models, and auxiliary models are often used to approximate and then explain their behavior. However, recent research suggests that such explanations are not overly accessible to non-expert users and can lead to incorrect interpretation of the underlying model. In this paper, we show that a predictive and interactive model based on causality is inherently interpretable, does not require any auxiliary model, and allows both expert and non-expert users to understand the model comprehensively. To demonstrate our method we developed Outcome Explorer, a causality guided interactive interface, and evaluated it by conducting think-aloud sessions with three expert users and a user study with 18 non-expert users. All three expert users found our tool to be comprehensive in supporting their explanation needs while the non-expert users were able to understand the inner workings of a model easily.

[1]  Andrea Vedaldi,et al.  Interpretable Explanations of Black Boxes by Meaningful Perturbation , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[2]  Paul Voigt,et al.  The EU General Data Protection Regulation (GDPR) , 2017 .

[3]  Paul Voigt,et al.  The Eu General Data Protection Regulation (Gdpr): A Practical Guide , 2017 .

[4]  Klaus Mueller,et al.  A network-based interface for the exploration of high-dimensional data spaces , 2012, 2012 IEEE Pacific Visualization Symposium.

[5]  Suresh Venkatasubramanian,et al.  Problems with Shapley-value-based explanations as feature importance measures , 2020, ICML.

[6]  Toniann Pitassi,et al.  Fairness through Causal Awareness: Learning Causal Latent-Variable Models for Biased Data , 2018, FAT.

[7]  Cynthia Rudin,et al.  Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead , 2018, Nature Machine Intelligence.

[8]  Jun Wang,et al.  Visual Causality Analysis Made Practical , 2017, 2017 IEEE Conference on Visual Analytics Science and Technology (VAST).

[9]  Yingcai Wu,et al.  A Visual Analytics Approach for Exploratory Causal Analysis: Exploration, Validation, and Applications , 2020, IEEE Transactions on Visualization and Computer Graphics.

[10]  Krzysztof Z. Gajos,et al.  Proxy tasks and subjective measures can be misleading in evaluating explainable AI systems , 2020, IUI.

[11]  Alexandra Chouldechova,et al.  A case study of algorithm-assisted decision making in child maltreatment hotline screening decisions , 2018, FAT.

[12]  Richard S. Johannes,et al.  Using the ADAP Learning Algorithm to Forecast the Onset of Diabetes Mellitus , 1988 .

[13]  D. Rubinfeld,et al.  Hedonic housing prices and the demand for clean air , 1978 .

[14]  Matt J. Kusner,et al.  Causal Interventions for Fairness , 2018, ArXiv.

[15]  Ruocheng Guo,et al.  Causal Interpretability for Machine Learning - Problems, Methods and Evaluation , 2020, SIGKDD Explor..

[16]  Cynthia Rudin,et al.  Why Are We Using Black Box Models in AI When We Don’t Need To? A Lesson From An Explainable AI Competition , 2019, 1.2.

[17]  Jonathan Herington,et al.  Measuring the Biases that Matter: The Ethical and Casual Foundations for Measures of Fairness in Algorithms , 2019, FAT.

[18]  Jun Wang,et al.  The Visual Causality Analyst: An Interactive Interface for Causal Reasoning , 2016, IEEE Transactions on Visualization and Computer Graphics.

[19]  Cynthia Rudin,et al.  The Bayesian Case Model: A Generative Approach for Case-Based Reasoning and Prototype Classification , 2014, NIPS.

[20]  Tim Miller,et al.  Explanation in Artificial Intelligence: Insights from the Social Sciences , 2017, Artif. Intell..

[21]  Finale Doshi-Velez,et al.  A Roadmap for a Rigorous Science of Interpretability , 2017, ArXiv.

[22]  Haiyi Zhu,et al.  Explaining Decision-Making Algorithms through UI: Strategies to Help Non-Expert Stakeholders , 2019, CHI.

[23]  Minsuk Kahng,et al.  Visual Analytics in Deep Learning: An Interrogative Survey for the Next Frontiers , 2018, IEEE Transactions on Visualization and Computer Graphics.

[24]  Joseph P. Simmons,et al.  Overcoming Algorithm Aversion: People Will Use Imperfect Algorithms If They Can (Even Slightly) Modify Them , 2016, Manag. Sci..

[25]  Martin Wattenberg,et al.  The What-If Tool: Interactive Probing of Machine Learning Models , 2019, IEEE Transactions on Visualization and Computer Graphics.

[26]  Norbert Schuff,et al.  Challenges and Opportunities with Causal Discovery Algorithms: Application to Alzheimer’s Pathophysiology , 2020, Scientific Reports.

[27]  Emden R. Gansner,et al.  Graphviz - Open Source Graph Drawing Tools , 2001, GD.

[28]  Duen Horng Chau,et al.  Summit: Scaling Deep Learning Interpretability by Visualizing Activation and Attribution Summarizations , 2019, IEEE Transactions on Visualization and Computer Graphics.

[29]  Sendhil Mullainathan,et al.  Dissecting Racial Bias in an Algorithm that Guides Health Decisions for 70 Million People , 2019, FAT.

[30]  Matt J. Kusner,et al.  Causal Reasoning for Algorithmic Fairness , 2018, ArXiv.

[31]  R. Tibshirani,et al.  Generalized Additive Models , 1986 .

[32]  Murray D. Smith,et al.  Structural Equation Modeling: Concepts, Issues, and Applications , 1996 .

[33]  Carlos Guestrin,et al.  "Why Should I Trust You?": Explaining the Predictions of Any Classifier , 2016, ArXiv.

[34]  Shruti Tople,et al.  Alleviating Privacy Attacks via Causal Learning , 2020, ICML.

[35]  Mélanie Frappier,et al.  The Book of Why: The New Science of Cause and Effect , 2018, Science.

[36]  T. Brennan,et al.  Evaluating the Predictive Validity of the Compas Risk and Needs Assessment System , 2009 .

[37]  Vasant Honavar,et al.  Fairness in Algorithmic Decision Making: An Excursion Through the Lens of Causality , 2019, WWW.

[38]  HeerJeffrey,et al.  D3 Data-Driven Documents , 2011 .

[39]  Huamin Qu,et al.  RuleMatrix: Visualizing and Understanding Classifiers with Rules , 2018, IEEE Transactions on Visualization and Computer Graphics.

[40]  D. G. Weeks,et al.  Linear structural equations with latent variables , 1980 .

[41]  Minsuk Kahng,et al.  ActiVis: Visual Exploration of Industry-Scale Deep Neural Network Models , 2017, IEEE Transactions on Visualization and Computer Graphics.

[42]  Steven M. Drucker,et al.  Gamut: A Design Probe to Understand How Data Scientists Understand Machine Learning Models , 2019, CHI.

[43]  Bernhard Schölkopf,et al.  Avoiding Discrimination through Causal Reasoning , 2017, NIPS.

[44]  Peter Spirtes,et al.  Introduction to Causal Inference , 2010, J. Mach. Learn. Res..

[45]  Hubert Lin,et al.  Silva: Interactively Assessing Machine Learning Fairness Using Causality , 2020, CHI.

[46]  Illtyd Trethowan.  Causality , 1938 .

[47]  Hanghang Tong,et al.  PC-Fairness: A Unified Framework for Measuring Causality-based Fairness , 2019, NeurIPS.

[48]  Mohan S. Kankanhalli,et al.  Trends and Trajectories for Explainable, Accountable and Intelligible Systems: An HCI Research Agenda , 2018, CHI.

[49]  Kenney Ng,et al.  Interacting with Predictions: Visual Inspection of Black-box Machine Learning Models , 2016, CHI.

[50]  Jesse Schell,et al.  The Art of Game Design: A book of lenses , 2019 .

[51]  LiuHuan,et al.  Causal Interpretability for Machine Learning - Problems, Methods and Evaluation , 2020 .

[52]  Enrico Bertini,et al.  ViCE: visual counterfactual explanations for machine learning models , 2020, IUI.

[53]  David Maxwell Chickering,et al.  ModelTracker: Redesigning Performance Analysis Tools for Machine Learning , 2015, CHI.

[54]  Edward L. Deci,et al.  Self-determination theory and the role of basic psychological needs in personality and the organization of behavior. , 2008 .

[55]  Scott Lundberg,et al.  A Unified Approach to Interpreting Model Predictions , 2017, NIPS.

[56]  P. Spirtes,et al.  Review of Causal Discovery Methods Based on Graphical Models , 2019, Front. Genet..

[57]  Elias Bareinboim,et al.  Fairness in Decision-Making - The Causal Explanation Formula , 2018, AAAI.