EBBE-Text: Explaining Neural Networks by Exploring Text Classification Decision Boundaries

While neural networks (NN) have been successfully applied to many NLP tasks, the way they function is often difficult to interpret. In this article, we focus on binary text classification via NNs and propose a new tool, which includes a visualization of the decision boundary and the distances of data elements to this boundary. This tool increases the interpretability of NN. Our approach uses two innovative views: (1) an overview of the text representation space and (2) a local view allowing data exploration around the decision boundary for various localities of this representation space. These views are integrated into a visual platform, EBBE-Text, which also contains state-of-the-art visualizations of NN representation spaces and several kinds of information obtained from the classification process. The various views are linked through numerous interactive functionalities that enable easy exploration of texts and classification results via the various complementary views. A user study shows the effectiveness of the visual encoding and a case study illustrates the benefits of using our tool for the analysis of the classifications obtained with several recent NNs and two datasets.

[1]  Luca Longo,et al.  Classification of Explainable Artificial Intelligence Methods through Their Output Formats , 2021, Mach. Learn. Knowl. Extr..

[2]  Duen Horng Chau,et al.  Dodrio: Exploring Transformer Models with Interactive Visualization , 2021, ACL.

[3]  Jiayao Wang,et al.  Attention Flows: Analyzing and Comparing Attention Mechanisms in Language Models , 2020, IEEE Transactions on Visualization and Computer Graphics.

[4]  Ross Maciejewski,et al.  Visual Analysis of Class Separations With Locally Linear Segments , 2020, IEEE Transactions on Visualization and Computer Graphics.

[5]  Sebastian Gehrmann,et al.  exBERT: A Visual Analysis Tool to Explore Learned Representations in Transformer Models , 2020, ACL.

[6]  F. Rossi,et al.  The State of the Art in Enhancing Trust in Machine Learning Models with the Use of Visualizations , 2020, Comput. Graph. Forum.

[7]  Andreas Kerren,et al.  A survey of surveys on the use of visualization for interpreting machine learning models , 2020, Inf. Vis..

[8]  James R. Eagan,et al.  Flexible and Context-Specific AI Explainability: A Multidisciplinary Approach , 2020, SSRN Electronic Journal.

[9]  Alexandru Telea,et al.  Constructing and Visualizing High-Quality Classifier Decision Boundary Maps , 2019, Inf..

[10]  Marie-Jeanne Lesot,et al.  The Dangers of Post-hoc Interpretability: Unjustified Counterfactual Explanations , 2019, IJCAI.

[11]  Jeffrey Heer,et al.  Local Decision Pitfalls in Interactive Machine Learning , 2019, ACM Trans. Comput. Hum. Interact..

[12]  Jesse Vig,et al.  A Multiscale Visualization of Attention in the Transformer Model , 2019, ACL.

[13]  Omer Levy,et al.  What Does BERT Look at? An Analysis of BERT’s Attention , 2019, BlackboxNLP@ACL.

[14]  Yonatan Belinkov,et al.  Analyzing the Structure of Attention in a Transformer Language Model , 2019, BlackboxNLP@ACL.

[15]  John DeNero,et al.  Adding Interpretable Attention to Neural Translation Models Improves Word Alignment , 2019, ArXiv.

[16]  Alex Endert,et al.  A Heuristic Approach to Value-Driven Evaluation of Visualizations , 2019, IEEE Transactions on Visualization and Computer Graphics.

[17]  Yang Wang,et al.  Manifold: A Model-Agnostic Framework for Interpretation and Diagnosis of Machine Learning Models , 2018, IEEE Transactions on Visualization and Computer Graphics.

[18]  Jimeng Sun,et al.  RAIM: Recurrent Attentive and Intensive Model of Multimodal Patient Monitoring Data , 2018, KDD.

[19]  Lalana Kagal,et al.  Explaining Explanations: An Overview of Interpretability of Machine Learning , 2018, 2018 IEEE 5th International Conference on Data Science and Advanced Analytics (DSAA).

[20]  Kush R. Varshney,et al.  Topological Data Analysis of Decision Boundaries with Application to Model Selection , 2018, ICML.

[21]  Alexander M. Rush,et al.  Seq2seq-Vis: A Visual Debugging Tool for Sequence-to-Sequence Models , 2018, IEEE Transactions on Visualization and Computer Graphics.

[22]  Leland McInnes,et al.  UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction , 2018, ArXiv.

[23]  Franco Turini,et al.  A Survey of Methods for Explaining Black Box Models , 2018, ACM Comput. Surv..

[24]  Minsuk Kahng,et al.  Visual Analytics in Deep Learning: An Interrogative Survey for the Next Frontiers , 2018, IEEE Transactions on Visualization and Computer Graphics.

[25]  Elmar Eisemann,et al.  DeepEyes: Progressive Visual Analytics for Designing Deep Neural Networks , 2018, IEEE Transactions on Visualization and Computer Graphics.

[26]  Issa Traoré,et al.  Detection of Online Fake News Using N-Gram Analysis and Machine Learning Techniques , 2017, ISDDC.

[27]  Silvio Savarese,et al.  Active Learning for Convolutional Neural Networks: A Core-Set Approach , 2017, ICLR.

[28]  Martin Wattenberg,et al.  SmoothGrad: removing noise by adding noise , 2017, ArXiv.

[29]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[30]  Minsuk Kahng,et al.  ActiVis: Visual Exploration of Industry-Scale Deep Neural Network Models , 2017, IEEE Transactions on Visualization and Computer Graphics.

[31]  Bowen Zhou,et al.  A Structured Self-attentive Sentence Embedding , 2017, ICLR.

[32]  Zoubin Ghahramani,et al.  Deep Bayesian Active Learning with Image Data , 2017, ICML.

[33]  Erhardt Barth,et al.  A Hybrid Convolutional Variational Autoencoder for Text Generation , 2017, EMNLP.

[34]  Daniel Jurafsky,et al.  Understanding Neural Networks through Representation Erasure , 2016, ArXiv.

[35]  Martin Wattenberg,et al.  Embedding Projector: Interactive Visualization and Interpretation of Embeddings , 2016, ArXiv.

[36]  Ramprasaath R. Selvaraju,et al.  Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization , 2016, International Journal of Computer Vision.

[37]  Yash Goyal,et al.  Towards Transparent AI Systems: Interpreting Visual Question Answering Models , 2016, 1608.08974.

[38]  Alexander M. Rush,et al.  LSTMVis: A Tool for Visual Analysis of Hidden State Dynamics in Recurrent Neural Networks , 2016, IEEE Transactions on Visualization and Computer Graphics.

[39]  Marco Tulio Ribeiro,et al.  “Why Should I Trust You?”: Explaining the Predictions of Any Classifier , 2016, NAACL.

[40]  Julian J. McAuley,et al.  Ups and Downs: Modeling the Visual Evolution of Fashion Trends with One-Class Collaborative Filtering , 2016, WWW.

[41]  Fei-Fei Li,et al.  Visualizing and Understanding Recurrent Networks , 2015, ArXiv.

[42]  David Maxwell Chickering,et al.  ModelTracker: Redesigning Performance Analysis Tools for Machine Learning , 2015, CHI.

[43]  Martin A. Riedmiller,et al.  Striving for Simplicity: The All Convolutional Net , 2014, ICLR.

[44]  Yoshua Bengio,et al.  Neural Machine Translation by Jointly Learning to Align and Translate , 2014, ICLR.

[45]  Djallel Bouneffouf,et al.  Exponentiated Gradient Exploration for Active Learning , 2014, Comput..

[46]  Kenneth E. Shirley,et al.  LDAvis: A method for visualizing and interpreting topics , 2014 .

[47]  C. Veenman,et al.  Visualizing multi-dimensional decision boundaries in 2D , 2013, Data Mining and Knowledge Discovery.

[48]  Jeffrey Dean,et al.  Distributed Representations of Words and Phrases and their Compositionality , 2013, NIPS.

[49]  Thomas Ertl,et al.  Visual Classifier Training for Text Document Retrieval , 2012, IEEE Transactions on Visualization and Computer Graphics.

[50]  Burr Settles,et al.  Closing the Loop: Fast, Interactive Semi-Supervised Annotation With Queries on Features and Instances , 2011, EMNLP.

[51]  Michael Granitzer,et al.  User-Based Active Learning , 2010, 2010 IEEE International Conference on Data Mining Workshops.

[52]  Karl Pearson F.R.S..  X. On the criterion that a given system of deviations from the probable in the case of a correlated system of variables is such that it can be reasonably supposed to have arisen from random sampling , 2009 .

[53]  Congfu Xu,et al.  Using decision boundary to analyze classifiers , 2008, 2008 3rd International Conference on Intelligent System and Knowledge Engineering.

[54]  Jin-Kao Hao,et al.  An effective two-stage simulated annealing algorithm for the minimum linear arrangement problem , 2008, Comput. Oper. Res..

[55]  Cynthia A. Brewer,et al.  ColorBrewer.org: An Online Tool for Selecting Colour Schemes for Maps , 2003 .

[56]  Ofer Melnik,et al.  Decision Region Connectivity Analysis: A Method for Analyzing High-Dimensional Classifiers , 2002, Machine Learning.

[57]  U. Brandes.  A faster algorithm for betweenness centrality , 2001 .

[58]  Yannis Dimopoulos,et al.  Use of some sensitivity criteria for choosing networks with good generalization ability , 1995, Neural Processing Letters.

[59]  Steven Fortune,et al.  A sweepline algorithm for Voronoi diagrams , 1986, SCG '86.

[60]  B. Matthews.  Comparison of the predicted and observed secondary structure of T4 phage lysozyme. , 1975, Biochimica et biophysica acta.

[61]  S. Shapiro,et al.  An Analysis of Variance Test for Normality (Complete Samples) , 1965 .

[62]  Karl Pearson F.R.S..  LIII. On lines and planes of closest fit to systems of points in space , 1901 .

[63]  Jérôme Azé,et al.  EBBE-Text : Visualisation de la frontière de décision des réseaux de neurones en classification automatique de textes , 2021, EGC.

[64]  Ilya Sutskever,et al.  Language Models are Unsupervised Multitask Learners , 2019 .

[65]  Wojciech Samek,et al.  Explainable AI: Interpreting, Explaining and Visualizing Deep Learning , 2019, Explainable AI.

[66]  Klaus-Robert Müller,et al.  Layer-Wise Relevance Propagation: An Overview , 2019, Explainable AI.

[67]  Jörg Tiedemann,et al.  An Analysis of Encoder Representations in Transformer-Based Machine Translation , 2018, BlackboxNLP@EMNLP.

[68]  Issa Traore,et al.  Detecting opinion spams and fake news using text classification , 2018, Secur. Priv..

[69]  Bernhard Waltl,et al.  Explainable Artificial Intelligence – the New Frontier in Legal Informatics , 2018 .

[70]  G. Tourassi,et al.  Visualization for Classification in Deep Neural Networks , 2017 .

[71]  Burr Settles,et al.  Active Learning Literature Survey , 2009 .

[72]  Geoffrey E. Hinton,et al.  Visualizing Data using t-SNE , 2008 .