Towards better analysis of machine learning models: A visual analytics perspective

Abstract Interactive model analysis, the process of understanding, diagnosing, and refining a machine learning model with the help of interactive visualization, is very important for users to efficiently solve real-world artificial intelligence and data mining problems. Dramatic advances in big data analytics have led to a wide variety of interactive model analysis tasks. In this paper, we present a comprehensive analysis and interpretation of this rapidly developing area. Specifically, we classify the relevant work into three categories: understanding, diagnosis, and refinement. Each category is exemplified by recent influential work. Possible future research opportunities are also explored and discussed.

[1]  Marc Streit,et al.  Opening the Black Box: Strategies for Increased User Involvement in Existing Algorithm Implementations , 2014, IEEE Transactions on Visualization and Computer Graphics.

[2]  Zhen Li,et al.  Towards Better Analysis of Deep Convolutional Neural Networks , 2016, IEEE Transactions on Visualization and Computer Graphics.

[3]  Baining Guo,et al.  Mining evolutionary multi-branch trees from text streams , 2013, KDD.

[4]  Weiwei Cui,et al.  How Hierarchical Topics Evolve in Large Text Corpora , 2014, IEEE Transactions on Visualization and Computer Graphics.

[5]  Shimei Pan,et al.  TIARA: Interactive, Topic-Based Visual Text Summarization and Analysis , 2012, TIST.

[6]  Sergio A. Alvarez,et al.  NVIS: an interactive visualization tool for neural networks , 2001, IS&T/SPIE Electronic Imaging.

[7]  Ching-Yung Lin,et al.  TargetVue: Visual Analysis of Anomalous User Behaviors in Online Communication Systems , 2016, IEEE Transactions on Visualization and Computer Graphics.

[8]  Burr Settles,et al.  Active Learning , 2012, Synthesis Lectures on Artificial Intelligence and Machine Learning.

[9]  Paul Geladi,et al.  Principal Component Analysis , 1987, Comprehensive Chemometrics.

[10]  Bo Zhang,et al.  Scalable Inference for Logistic-Normal Topic Models , 2013, NIPS.

[11]  David Maxwell Chickering,et al.  ModelTracker: Redesigning Performance Analysis Tools for Machine Learning , 2015, CHI.

[12]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[13]  Paulo E. Rauber,et al.  Visualizing the Hidden Activity of Artificial Neural Networks , 2017, IEEE Transactions on Visualization and Computer Graphics.

[14]  H. Sebastian Seung,et al.  Learning the parts of objects by non-negative matrix factorization , 1999, Nature.

[15]  Furu Wei,et al.  Query-sensitive mutual reinforcement chain and its application in query-oriented multi-document summarization , 2008, SIGIR '08.

[16]  Min Chen,et al.  An Analysis of Machine- and Human-Analytics in Classification , 2017, IEEE Transactions on Visualization and Computer Graphics.

[17]  Jude W. Shavlik,et al.  Visualizing Learning and Computation in Artificial Neural Networks , 1992, Int. J. Artif. Intell. Tools.

[18]  Jian Pei,et al.  Online Visual Analytics of Text Streams , 2015, IEEE Transactions on Visualization and Computer Graphics.

[19]  Joshua B. Tenenbaum,et al.  Human-level concept learning through probabilistic program induction , 2015, Science.

[20]  Jeffrey Heer,et al.  Topic Model Diagnostics: Assessing Domain Relevance via Topical Alignment , 2013, ICML.

[21]  Jaegul Choo,et al.  UTOPIAN: User-Driven Topic Modeling Based on Interactive Nonnegative Matrix Factorization , 2013, IEEE Transactions on Visualization and Computer Graphics.

[22]  Kwan-Liu Ma,et al.  Visualizing Flow of Uncertainty through Analytical Processes , 2012, IEEE Transactions on Visualization and Computer Graphics.

[23]  Shie Mannor,et al.  Graying the black box: Understanding DQNs , 2016, ICML.

[24]  Thomas Ertl,et al.  Visual Classifier Training for Text Document Retrieval , 2012, IEEE Transactions on Visualization and Computer Graphics.

[25]  Jean-Daniel Fekete,et al.  Visual Analytics Infrastructures: From Data Management to Exploration , 2013, Computer.

[26]  Baining Guo,et al.  TopicPanorama: A Full Picture of Relevant Topics , 2014, IEEE Transactions on Visualization and Computer Graphics.

[27]  David Gotz,et al.  Progressive Visual Analytics: User-Driven Visual Exploration of In-Progress Analytics , 2014, IEEE Transactions on Visualization and Computer Graphics.

[29]  Shixia Liu,et al.  Topic- and Time-Oriented Visual Text Analysis , 2016, IEEE Computer Graphics and Applications.

[30]  Andrew McCallum,et al.  Employing EM and Pool-Based Active Learning for Text Classification , 1998, ICML.

[31]  Cynthia Rudin,et al.  Interpretable classifiers using rules and Bayesian analysis: Building a better stroke prediction model , 2015, ArXiv.

[32]  Yale Song,et al.  #FluxFlow: Visual Analysis of Anomalous Information Spreading on Social Media , 2014, IEEE Transactions on Visualization and Computer Graphics.

[33]  Elmar Eisemann,et al.  Approximated and User Steerable tSNE for Progressive Visual Analytics , 2015, IEEE Transactions on Visualization and Computer Graphics.

[34]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[35]  Ryan Turner,et al.  A Model Explanation System: Latest Updates and Extensions , 2016, ArXiv.

[36]  Kwan-Liu Ma,et al.  A framework for uncertainty-aware visual analytics , 2009, 2009 IEEE Symposium on Visual Analytics Science and Technology.

[37]  Hong Zhou,et al.  OpinionSeer: Interactive Visualization of Hotel Customer Feedback , 2010, IEEE Transactions on Visualization and Computer Graphics.

[38]  Rosane Minghim,et al.  An Approach to Supporting Incremental Visual Data Classification , 2015, IEEE Transactions on Visualization and Computer Graphics.

[39]  Kwan-Liu Ma,et al.  Opening the black box - data driven visualization of neural networks , 2005, VIS 05. IEEE Visualization, 2005..

[40]  G. G. Stokes.  "J." , 1890, The New Yale Book of Quotations.

[41]  David A. Cohn,et al.  Improving generalization with active learning , 1994, Machine Learning.

[42]  David A. Cohn,et al.  Active Learning with Statistical Models , 1996, NIPS.

[43]  Rosane Minghim,et al.  Improved Similarity Trees and their Application to Visual Data Classification , 2011, IEEE Transactions on Visualization and Computer Graphics.

[44]  Geoffrey E. Hinton,et al.  Visualizing Data using t-SNE , 2008 .

[45]  Mengchen Liu,et al.  A survey on information visualization: recent advances and challenges , 2014, The Visual Computer.

[46]  Xin Tong,et al.  TextFlow: Towards Better Understanding of Evolving Topics in Text , 2011, IEEE Transactions on Visualization and Computer Graphics.

[47]  Adam W. Harley.  An Interactive Node-Link Visualization of Convolutional Neural Networks , 2015, ISVC.

[48]  Daniel A. Keim,et al.  The Role of Uncertainty, Awareness, and Trust in Visual Analytics , 2016, IEEE Transactions on Visualization and Computer Graphics.

[49]  William Ribarsky,et al.  HierarchicalTopics: Visually Exploring Large Text Collections Using Topic Hierarchies , 2013, IEEE Transactions on Visualization and Computer Graphics.

[50]  Silvia Miksch,et al.  Visual Methods for Analyzing Probabilistic Classification Data , 2014, IEEE Transactions on Visualization and Computer Graphics.

[51]  Kwan-Liu Ma,et al.  A cluster-space visual interface for arbitrary dimensional classification of volume data , 2004, VISSYM'04.

[52]  Bongshin Lee,et al.  Squares: Supporting Interactive Performance Analysis for Multiclass Classifiers , 2017, IEEE Transactions on Visualization and Computer Graphics.

[53]  Qinying Liao,et al.  An Uncertainty-Aware Approach for Exploratory Microblog Retrieval , 2015, IEEE Transactions on Visualization and Computer Graphics.