A general deep learning framework for network reconstruction and dynamics learning

Many complex processes can be viewed as dynamical systems on networks. However, in real cases, only the performances of the system are known, the network structure and the dynamical rules are not observed. Therefore, recovering latent network structure and dynamics from observed time series data are important tasks because it may help us to open the black box, and even to build up the model of a complex system automatically. Although this problem hosts a wealth of potential applications in biology, earth science, and epidemics etc., conventional methods have limitations. In this work, we introduce a new framework, Gumbel Graph Network (GGN), which is a model-free, data-driven deep learning framework to accomplish the reconstruction of both network connections and the dynamics on it. Our model consists of two jointly trained parts: a network generator that generating a discrete network with the Gumbel Softmax technique; and a dynamics learner that utilizing the generated network and one-step trajectory value to predict the states in future steps. We exhibit the universality of our framework on different kinds of time-series data: with the same structure, our model can be trained to accurately recover the network structure and predict future states on continuous, discrete, and binary dynamics, and outperforms competing network reconstruction methods.

[1]  Jieping Ye,et al.  Network Reconstruction Based on Evolutionary-Game Data via Compressive Sensing , 2011, Physical Review X.

[2]  Isabelle Guyon,et al.  Taking Human out of Learning Applications: A Survey on Automated Machine Learning , 2018, 1810.13306.

[3]  V. Latora,et al.  Complex networks: Structure and dynamics , 2006 .

[4]  Cyrus Shahabi,et al.  Diffusion Convolutional Recurrent Neural Network: Data-Driven Traffic Forecasting , 2017, ICLR.

[5]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[6]  Wen-Xu Wang,et al.  Reconstructing propagation networks with natural diversity and identifying hidden sources , 2014, Nature Communications.

[7]  Duncan J. Watts,et al.  Collective dynamics of ‘small-world’ networks , 1998, Nature.

[8]  Zhanxing Zhu,et al.  Spatio-temporal Graph Convolutional Neural Network: A Deep Learning Framework for Traffic Forecasting , 2017, IJCAI.

[9]  Razvan Pascanu,et al.  Relational inductive biases, deep learning, and graph networks , 2018, ArXiv.

[10]  R. Zemel,et al.  Neural Relational Inference for Interacting Systems , 2018, ICML.

[11]  K. Kaneko Overview of coupled map lattices. , 1992, Chaos.

[12]  Wen-Xu Wang,et al.  Reconstructing complex networks with binary-state dynamics , 2015, ArXiv.

[13]  K. Kaneko Pattern dynamics in spatiotemporal chaos: Pattern selection, diffusion of defect and pattern competition intermettency , 1989 .

[14]  Muriel Médard,et al.  Network deconvolution as a general method to distinguish direct dependencies in networks , 2013, Nature Biotechnology.

[15]  Razvan Pascanu,et al.  Visual Interaction Networks: Learning a Physics Simulator from Video , 2017, NIPS.

[16]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[17]  Dahua Lin,et al.  Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognition , 2018, AAAI.

[18]  Aaron Klein,et al.  Efficient and Robust Automated Machine Learning , 2015, NIPS.

[19]  Max Welling,et al.  Semi-Supervised Classification with Graph Convolutional Networks , 2016, ICLR.

[20]  E. Ott,et al.  Onset of synchronization in large networks of coupled oscillators. , 2004, Physical review. E, Statistical, nonlinear, and soft matter physics.

[21]  Todd P. Coleman,et al.  Estimating the directed information to infer causal relationships in ensemble neural spike train recordings , 2010, Journal of Computational Neuroscience.

[22]  Nathaniel Virgo,et al.  Permutation-equivariant neural networks applied to dynamics prediction , 2016, ArXiv.

[23]  Tara N. Sainath,et al.  Deep Neural Networks for Acoustic Modeling in Speech Recognition , 2012 .

[24]  Mark E. J. Newman,et al.  The Structure and Function of Complex Networks , 2003, SIAM Rev..

[25]  Hao Ma,et al.  GaAN: Gated Attention Networks for Learning on Large and Spatiotemporal Graphs , 2018, UAI.

[26]  G. Cecchi,et al.  Scale-free brain functional networks. , 2003, Physical review letters.

[27]  Jure Leskovec,et al.  GraphRNN: Generating Realistic Graphs with Deep Auto-regressive Models , 2018, ICML.

[28]  S. Kauffman Metabolic stability and epigenesis in randomly constructed genetic nets. , 1969, Journal of theoretical biology.

[29]  Tara N. Sainath,et al.  Deep Neural Networks for Acoustic Modeling in Speech Recognition: The Shared Views of Four Research Groups , 2012, IEEE Signal Processing Magazine.

[30]  J. Collins,et al.  Inferring Genetic Networks and Identifying Compound Mode of Action via Expression Profiling , 2003, Science.

[31]  Yoshiki Kuramoto,et al.  Self-entrainment of a population of coupled non-linear oscillators , 1975 .

[32]  Karl J. Friston,et al.  Structural and Functional Brain Networks: From Connections to Cognition , 2013, Science.

[33]  Razvan Pascanu,et al.  Learning Deep Generative Models of Graphs , 2018, ICLR 2018.

[34]  Philip S. Yu,et al.  A Comprehensive Survey on Graph Neural Networks , 2019, IEEE Transactions on Neural Networks and Learning Systems.

[35]  Silvio Savarese,et al.  Structural-RNN: Deep Learning on Spatio-Temporal Graphs , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[36]  Marc Timme,et al.  Revealing network connectivity from response dynamics. , 2006, Physical review letters.

[37]  Pietro Liò,et al.  Graph Attention Networks , 2017, ICLR.

[38]  Wen-Xu Wang,et al.  Predicting catastrophes in nonlinear dynamical systems by compressive sensing. , 2011, Physical review letters.

[39]  A. Barabasi,et al.  Network link prediction by global silencing of indirect correlations , 2013, Nature Biotechnology.

[40]  Joshua M. Stuart,et al.  A Gene-Coexpression Network for Global Discovery of Conserved Genetic Modules , 2003, Science.

[41]  Jure Leskovec,et al.  GraphRNN: A Deep Generative Model for Graphs , 2018, ICML 2018.

[42]  S. Strogatz Exploring complex networks , 2001, Nature.

[43]  Ben Poole,et al.  Categorical Reparameterization with Gumbel-Softmax , 2016, ICLR.

[44]  Razvan Pascanu,et al.  Interaction Networks for Learning about Objects, Relations and Physics , 2016, NIPS.

[45]  S. Bressler,et al.  Beta oscillations in a large-scale sensorimotor cortical network: directional influences revealed by Granger causality. , 2004, Proceedings of the National Academy of Sciences of the United States of America.

[46]  S. Nadarajah,et al.  The beta Gumbel distribution , 2004 .

[47]  Ying-Cheng Lai,et al.  Universal data-based method for reconstructing complex networks with binary-state dynamics. , 2015, Physical review. E.

[48]  Jürgen Kurths,et al.  Complex networks reveal global pattern of extreme-rainfall teleconnections , 2019, Nature.

[49]  Albert-László Barabási,et al.  Statistical mechanics of complex networks , 2001, ArXiv.

[50]  Nicola De Cao,et al.  MolGAN: An implicit generative model for small molecular graphs , 2018, ArXiv.

[51]  Wen-Xu Wang,et al.  Time-series–based prediction of complex oscillator networks via compressive sensing , 2011 .

[52]  Marc Timme,et al.  Model-free inference of direct network interactions from nonlinear collective dynamics , 2017, Nature Communications.

[53]  Stephan Günnemann,et al.  NetGAN: Generating Graphs via Random Walks , 2018, ICML.