TopoAct: Exploring the Shape of Activations in Deep Learning

Deep neural networks such as GoogLeNet and ResNet have achieved superhuman performance in tasks like image classification. To understand how such superior performance is achieved, we can probe a trained deep neural network by studying neuron activations, that is, combinations of neuron firings, at any layer of the network in response to a particular input. With a large set of input images, we aim to obtain a global view of what neurons detect by studying their activations. We ask the following questions: What is the shape of the space of activations? That is, what is the organizational principle behind neuron activations, and how are the activations related within a layer and across layers? Applying tools from topological data analysis, we present TopoAct, a visual exploration system used to study topological summaries of activation vectors for a single layer as well as the evolution of such summaries across multiple layers. We present visual exploration scenarios using TopoAct that provide valuable insights towards learned representations of an image classifier.

[1]  Deborah Silver,et al.  Feature Visualization , 1994, Scientific Visualization.

[2]  Andrea Vedaldi,et al.  Understanding deep image representations by inverting them , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[3]  Pascal Vincent,et al.  Visualizing Higher-Layer Features of a Deep Network , 2009 .

[4]  Herbert Edelsbrunner,et al.  Hierarchical Morse—Smale Complexes for Piecewise Linear 2-Manifolds , 2003, Discret. Comput. Geom..

[5]  Bernd Hamann,et al.  Measuring the Distance Between Merge Trees , 2014, Topological Methods in Data Analysis and Visualization.

[6]  Leland McInnes,et al.  UMAP: Uniform Manifold Approximation and Projection , 2018, J. Open Source Softw..

[7]  Fei-Fei Li,et al.  ImageNet: A large-scale hierarchical image database , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[8]  Silvia Biasotti,et al.  An overview on properties and efficacy of topological skeletons in shape modeling , 2003, 2003 Shape Modeling International..

[9]  Daniela Giorgi,et al.  Reeb graphs for shape analysis and applications , 2008, Theor. Comput. Sci..

[10]  Martin Wattenberg,et al.  Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV) , 2017, ICML.

[11]  Facundo Mémoli,et al.  Topological Methods for the Analysis of High Dimensional Data Sets and 3D Object Recognition , 2007, PBG@Eurographics.

[12]  Bei Wang,et al.  Convergence between Categorical Representations of Reeb Space and Mapper , 2015, SoCG.

[13]  Jason Yosinski,et al.  Multifaceted Feature Visualization: Uncovering the Different Types of Features Learned By Each Neuron in Deep Neural Networks , 2016, ArXiv.

[14]  Arvind Satyanarayan,et al.  The Building Blocks of Interpretability , 2018 .

[15]  Minsuk Kahng,et al.  Visual Analytics in Deep Learning: An Interrogative Survey for the Next Frontiers , 2018, IEEE Transactions on Visualization and Computer Graphics.

[16]  Tim Dwyer,et al.  Scalable, Versatile and Simple Constrained Graph Layout , 2009, Comput. Graph. Forum.

[17]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[18]  Jack Snoeyink,et al.  Computing contour trees in all dimensions , 2000, SODA '00.

[19]  Andrea Vedaldi,et al.  Visualizing Deep Convolutional Neural Networks Using Natural Pre-images , 2015, International Journal of Computer Vision.

[20]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[21]  Hans-Peter Kriegel,et al.  A Density-Based Algorithm for Discovering Clusters in Large Spatial Databases with Noise , 1996, KDD.

[22]  Qiang Chen,et al.  Network In Network , 2013, ICLR.

[23]  Samuel Gerber,et al.  Data Analysis with the Morse-Smale Complex: The msr Package for R , 2012 .

[24]  Herbert Edelsbrunner,et al.  Reeb spaces of piecewise linear mappings , 2008, SCG '08.

[25]  Duen Horng Chau,et al.  Summit: Scaling Deep Learning Interpretability by Visualizing Activation and Attribution Summarizations , 2019, IEEE Transactions on Visualization and Computer Graphics.

[26]  Valerio Pascucci,et al.  Local, smooth, and consistent Jacobi set simplification , 2013, Comput. Geom..

[27]  Marc'Aurelio Ranzato,et al.  Building high-level features using large scale unsupervised learning , 2011, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[28]  Rob Fergus,et al.  Visualizing and Understanding Convolutional Networks , 2013, ECCV.

[29]  Geoffrey E. Hinton,et al.  Visualizing Data using t-SNE , 2008 .

[30]  Valerio Pascucci,et al.  Morse-smale complexes for piecewise linear 3-manifolds , 2003, SCG '03.

[31]  P. Alexandroff,et al.  Über den allgemeinen Dimensionsbegriff und seine Beziehungen zur elementaren geometrischen Anschauung , 1928 .

[32]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[33]  Andrew Zisserman,et al.  Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps , 2013, ICLR.