Examining the Causal Structures of Deep Neural Networks Using Information Theory

Deep Neural Networks (DNNs) are often examined at the level of their response to input, such as analyzing the mutual information between nodes and data sets. Yet DNNs can also be examined at the level of causation, exploring "what does what" within the layers of the network itself. Historically, analyzing the causal structure of DNNs has received less attention than understanding their responses to input. Yet definitionally, generalizability must be a function of a DNN's causal structure as it reflects how the DNN responds to unseen or even not-yet-defined future inputs. Here, we introduce a suite of metrics based on information theory to quantify and track changes in the causal structure of DNNs during training. Specifically, we introduce the effective information (EI) of a feedforward DNN, which is the mutual information between layer input and output following a maximum-entropy perturbation. The EI can be used to assess the degree of causal influence nodes and edges have over their downstream targets in each layer. We show that the EI can be further decomposed in order to examine the sensitivity of a layer (measured by how well edges transmit perturbations) and the degeneracy of a layer (measured by how edge overlap interferes with transmission), along with estimates of the amount of integrated information of a layer. Together, these properties define where each layer lies in the "causal plane", which can be used to visualize how layer connectivity becomes more sensitive or degenerate over time, and how integration changes during training, revealing how the layer-by-layer causal structure differentiates. These results may help in understanding the generalization capabilities of DNNs and provide foundational tools for making DNNs both more generalizable and more explainable.

[1]  Zhizheng Wu,et al.  Merlin: An Open Source Neural Network Speech Synthesis System , 2016, SSW.

[2]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[3]  Brian E. Ruttenberg,et al.  Causal Learning and Explanation of Deep Neural Networks via Autoencoded Activations , 2018, ArXiv.

[4]  Randall D. Beer,et al.  Nonnegative Decomposition of Multivariate Information , 2010, ArXiv.

[5]  Quoc V. Le,et al.  Sequence to Sequence Learning with Neural Networks , 2014, NIPS.

[6]  Rajat Raina,et al.  Large-scale deep unsupervised learning using graphics processors , 2009, ICML '09.

[7]  Yang Jin,et al.  Capsule Network Performance on Complex Data , 2017, ArXiv.

[8]  Sang Joon Kim,et al.  A Mathematical Theory of Communication , 2006 .

[9]  J. I.  The Design of Experiments , 1936, Nature.

[10]  G. E. Berrios,et al.  . New York: Cambridge , 2000 .

[11]  Giulio Tononi,et al.  Integrated Information in Discrete Dynamical Systems: Motivation and Theoretical Framework , 2008, PLoS Comput. Biol..

[12]  Shun-ichi Amari,et al.  Unified framework for information integration based on information geometry , 2015, Proceedings of the National Academy of Sciences.

[13]  David D. Cox,et al.  On the information bottleneck theory of deep learning , 2018, ICLR.

[14]  Max Tegmark,et al.  Improved Measures of Integrated Information , 2016, PLoS Comput. Biol..

[15]  Terrence J. Sejnowski,et al.  Slow Feature Analysis: Unsupervised Learning of Invariances , 2002, Neural Computation.

[16]  Naftali Tishby,et al.  The information bottleneck method , 2000, ArXiv.

[17]  Senthil Mani,et al.  Explaining Deep Learning Models using Causal Inference , 2018, ArXiv.

[18]  Larissa Albantakis,et al.  From the Phenomenology to the Mechanisms of Consciousness: Integrated Information Theory 3.0 , 2014, PLoS Comput. Biol..

[19]  Geoffrey E. Hinton,et al.  Rectified Linear Units Improve Restricted Boltzmann Machines , 2010, ICML.

[20]  Larissa Albantakis,et al.  What Caused What? A Quantitative Account of Actual Causation Using Dynamical Causal Networks , 2017, Entropy.

[21]  G. Tononi.  Consciousness as Integrated Information: a Provisional Manifesto , 2008, The Biological Bulletin.

[22]  Larissa Albantakis,et al.  How causal analysis can reveal autonomy in models of biological systems , 2017, Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[23]  Ronald M. Summers,et al.  Deep Convolutional Neural Networks for Computer-Aided Detection: CNN Architectures, Dataset Characteristics and Transfer Learning , 2016, IEEE Transactions on Medical Imaging.

[24]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[25]  Naftali Tishby,et al.  Opening the Black Box of Deep Neural Networks via Information , 2017, ArXiv.

[26]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[27]  Erik P. Hoel,et al.  Can the macro beat the micro? Integrated information across spatiotemporal scales. , 2016, Neuroscience of consciousness.

[28]  Pedro A. M. Mediano,et al.  Measuring Integrated Information: Comparison of Candidate Measures in Theory and Simulation , 2018, Entropy.

[29]  Kurt Hornik,et al.  Multilayer feedforward networks are universal approximators , 1989, Neural Networks.

[30]  Marco Broccardo,et al.  One neuron is more informative than a deep neural network for aftershock pattern forecasting , 2019, 1904.01983.

[31]  Rory A. Fisher,et al.  The Design of Experiments. , 1936 .

[32]  Robert J. Wood,et al.  Science, technology and the future of small autonomous drones , 2015, Nature.

[33]  Nathan Srebro,et al.  Exploring Generalization in Deep Learning , 2017, NIPS.

[34]  Andrew Zisserman,et al.  Turning a Blind Eye: Explicit Removal of Biases and Variation from Deep Neural Network Embeddings , 2018, ECCV Workshops.

[35]  Erik P. Hoel.  When the Map Is Better Than the Territory , 2016, Entropy.

[36]  Michael Kampffmeyer,et al.  Information Plane Analysis of Deep Neural Networks via Matrix-Based Renyi's Entropy and Tensor Kernels , 2019, ArXiv.

[37]  Erik P. Hoel,et al.  Quantifying causal emergence shows that macro can beat micro , 2013, Proceedings of the National Academy of Sciences.

[38]  Olaf Sporns,et al.  Measuring information integration , 2003, BMC Neuroscience.

[39]  Xin Zhang,et al.  End to End Learning for Self-Driving Cars , 2016, ArXiv.

[40]  A. V. Olgac,et al.  Performance Analysis of Various Activation Functions in Generalized MLP Architectures of Neural Networks , 2011 .

[41]  Samy Bengio,et al.  Understanding deep learning requires rethinking generalization , 2016, ICLR.

[42]  Naftali Tishby,et al.  Deep learning and the information bottleneck principle , 2015, 2015 IEEE Information Theory Workshop (ITW).

[43]  David Balduzzi,et al.  Information, learning and falsification , 2011, NIPS 2011.