TopoAct: Visually Exploring the Shape of Activations in Deep Learning

Deep neural networks such as GoogLeNet and ResNet have achieved impressive performance in tasks like image classification. To understand how such performance is achieved, we can probe a trained deep neural network by studying neuron activations, i. e., combinations of neuron firings, at any layer of the network in response to a particular input. With a large number of inputs, we aim to obtain a global view of what neurons detect by studying their activations. We ask the following questions: What is the shape of the activation space? What is the organizational principle behind neuron activations? How are the activations related within a layer and across layers? Applying tools from topological data analysis, we present TopoAct, a visual exploration system to study topological summaries of activation vectors for a single layer as well as the evolution of such summaries across multiple layers. We present exploration scenarios using TopoAct that provide valuable insights towards learned representations of an image classifier. We expect TopoAct to give a topological perspective that enriches the current toolbox of neural network analysis, and to provide a basis for network architecture diagnosis and data anomaly detection.