Forward Composition Propagation for Explainable Neural Reasoning

This paper proposes an algorithm called Forward Composition Propagation (FCP) to explain the predictions of feed-forward neural networks operating on structured pattern recognition problems. In the proposed FCP algorithm, each neuron is described by a composition vector indicating the role of each problem feature in that neuron. Composition vectors are initialized using a given input instance and subsequently propagated through the whole network until we reach the output layer. It is worth mentioning that the algorithm is executed once the network’s training network is done. The sign of each composition value indicates whether the corresponding feature excites or inhibits the neuron, while the absolute value quantifies such an impact. Aiming to validate the FCP algorithm’s correctness, we develop a case study concerning bias detection in a state-of-the-art problem in which the ground truth ∗Corresponding author Email address: i.d.c.grau.garcia@tue.nl (Gonzalo Nápoles) Equal contribution Preprint submitted to TBD December 24, 2021 ar X iv :2 11 2. 12 71 7v 1 [ cs .L G ] 2 3 D ec 2 02 1 is known. The simulation results show that the composition values closely align with the expected behavior of protected features.

[1]  Osbert Bastani,et al.  Interpretability via Model Extraction , 2017, ArXiv.

[2]  Matti Pietikäinen,et al.  Deep Learning for Generic Object Detection: A Survey , 2018, International Journal of Computer Vision.

[3]  Rob Fergus,et al.  Visualizing and Understanding Convolutional Networks , 2013, ECCV.

[4]  Cynthia Rudin,et al.  All Models are Wrong, but Many are Useful: Learning a Variable's Importance by Studying an Entire Class of Prediction Models Simultaneously , 2019, J. Mach. Learn. Res..

[5]  Klaus-Robert Müller,et al.  Layer-Wise Relevance Propagation: An Overview , 2019, Explainable AI.

[6]  Hridesh Rajan,et al.  Do the machine learning models on a crowd sourced platform exhibit bias? an empirical study on model fairness , 2020, ESEC/SIGSOFT FSE.

[7]  Jimeng Sun,et al.  RETAIN: An Interpretable Predictive Model for Healthcare using Reverse Time Attention Mechanism , 2016, NIPS.

[8]  Bolei Zhou,et al.  Network Dissection: Quantifying Interpretability of Deep Visual Representations , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[9]  Maria Pantoja,et al.  Introduction to Deep Learning , 2018, Deep Learning.

[10]  H. Tsukimoto,et al.  Rule extraction from neural networks via decision tree induction , 2001, IJCNN'01. International Joint Conference on Neural Networks. Proceedings (Cat. No.01CH37222).

[11]  Avanti Shrikumar,et al.  Learning Important Features Through Propagating Activation Differences , 2017, ICML.

[12]  Thomas Brox,et al.  Striving for Simplicity: The All Convolutional Net , 2014, ICLR.

[13]  Kristina Lerman,et al.  A Survey on Bias and Fairness in Machine Learning , 2019, ACM Comput. Surv..

[14]  Scott Lundberg,et al.  A Unified Approach to Interpreting Model Predictions , 2017, NIPS.

[15]  Diptikalyan Saha,et al.  Black box fairness testing of machine learning models , 2019, ESEC/SIGSOFT FSE.

[16]  Alexander Binder,et al.  Explaining nonlinear classification decisions with deep Taylor decomposition , 2015, Pattern Recognit..

[17]  Yarin Gal,et al.  Real Time Image Saliency for Black Box Classifiers , 2017, NIPS.

[18]  Carlos Guestrin,et al.  "Why Should I Trust You?": Explaining the Predictions of Any Classifier , 2016, ArXiv.

[19]  Graham W. Taylor,et al.  Adaptive deconvolutional networks for mid and high level feature learning , 2011, 2011 International Conference on Computer Vision.

[20]  Gonzalo Nápoles,et al.  Recommender system using Long-term Cognitive Networks , 2020, Knowl. Based Syst..

[21]  Tameru Hailesilassie,et al.  Rule Extraction Algorithm for Deep Neural Networks: A Review , 2016, ArXiv.

[22]  Eneldo Loza Mencía,et al.  DeepRED - Rule Extraction from Deep Neural Networks , 2016, DS.

[23]  D. K. Mishra,et al.  KDRuleEx: A Novel Approach for Enhancing User Comprehensibility Using Rule Extraction , 2012, 2012 Third International Conference on Intelligent Systems Modelling and Simulation.

[24]  Nicu Sebe,et al.  Explainable deep learning for efficient and robust pattern recognition: A survey of recent developments , 2021, Pattern Recognit..

[25]  Geoffrey E. Hinton,et al.  Distilling a Neural Network Into a Soft Decision Tree , 2017, CEx@AI*IA.

[26]  Jacob Andreas,et al.  Compositional Explanations of Neurons , 2020, NeurIPS.

[27]  Ann Nowé,et al.  Interpretable self-labeling semi-supervised classifier , 2018 .

[28]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[29]  Alexander Binder,et al.  On Pixel-Wise Explanations for Non-Linear Classifier Decisions by Layer-Wise Relevance Propagation , 2015, PloS one.

[30]  Klaus-Robert Müller,et al.  Explaining Recurrent Neural Network Predictions in Sentiment Analysis , 2017, WASSA@EMNLP.

[31]  Lale Özbakir,et al.  A soft computing-based approach for integrated training and rule extraction from artificial neural networks: DIFACONN-miner , 2010, Appl. Soft Comput..

[32]  Alberto Lavelli,et al.  An introduction to Deep Learning in Natural Language Processing: Models, techniques, and tools , 2021, Neurocomputing.

[33]  Andrew Zisserman,et al.  Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps , 2013, ICLR.

[34]  Jacob Cohen.  A Coefficient of Agreement for Nominal Scales , 1960 .

[35]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[36]  Abhishek Das,et al.  Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization , 2016, 2017 IEEE International Conference on Computer Vision (ICCV).

[37]  Francisco Herrera,et al.  Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AI , 2020, Inf. Fusion.

[38]  Suresh Venkatasubramanian,et al.  A comparative study of fairness-enhancing interventions in machine learning , 2018, FAT.