GOPHER: Categorical probabilistic forecasting with graph structure via local continuous-time dynamics

We consider the problem of probabilistic forecasting over categories with graph structure, where the dynamics at a vertex depends on its local connectivity structure. We present GOPHER, a method that combines the inductive bias of graph neural networks with neural ODEs to capture the intrinsic local continuous-time dynamics of our probabilistic forecasts. We study the benefits of these two inductive biases by comparing against baseline models that help disentangle the benefits of each. We find that capturing the graph structure is crucial for accurate in-domain probabilistic predictions and more sample efficient models. Surprisingly, our experiments demonstrate that the continuous time evolution inductive bias brings little to no benefit despite reflecting the true probability dynamics.

[1]  Scott W. Linderman,et al.  Discovering Latent Network Structure in Point Process Data , 2014, ICML.

[2]  Rose Yu,et al.  Bridging Physics-based and Data-driven modeling for Learning Dynamical Systems , 2020, L4DC.

[3]  Ricky T. Q. Chen,et al.  Neural Spatio-Temporal Point Processes , 2020, ICLR.

[4]  Markus Heinonen,et al.  Learning continuous-time PDEs from sparse data with graph neural networks , 2020, ICLR.

[5]  Lada A. Adamic,et al.  The role of social networks in information diffusion , 2012, WWW.

[6]  Maximilian Nickel,et al.  Riemannian Continuous Normalizing Flows , 2020, NeurIPS.

[7]  Atsushi Yamashita,et al.  Continuous-Depth Neural Models for Dynamic Graph Prediction , 2021, ArXiv.

[8]  David Duvenaud,et al.  FFJORD: Free-form Continuous Dynamics for Scalable Reversible Generative Models , 2018, ICLR.

[9]  George Athanasopoulos,et al.  Forecasting: principles and practice , 2013 .

[10]  Matthias W. Seeger,et al.  Deep State Space Models for Time Series Forecasting , 2018, NeurIPS.

[11]  Christopher De Sa,et al.  Neural Manifold Ordinary Differential Equations , 2020, NeurIPS.

[12]  R. Cowan.  An introduction to the theory of point processes , 1978 .

[13]  Zhanxing Zhu,et al.  Spatio-temporal Graph Convolutional Neural Network: A Deep Learning Framework for Traffic Forecasting , 2017, IJCAI.

[14]  David Duvenaud,et al.  Neural Ordinary Differential Equations , 2018, NeurIPS.

[15]  Jure Leskovec,et al.  How Powerful are Graph Neural Networks? , 2018, ICLR.

[16]  Mehran Mesbahi,et al.  Advection on graphs , 2011, IEEE Conference on Decision and Control and European Control Conference.

[17]  Andrew Gordon Wilson,et al.  Simplifying Hamiltonian and Lagrangian Neural Networks via Explicit Constraints , 2020, NeurIPS.

[18]  Yee Whye Teh,et al.  Augmented Neural ODEs , 2019, NeurIPS.

[19]  Maoan Han,et al.  Dynamics of an SIS reaction-diffusion epidemic model for disease transmission. , 2010, Mathematical biosciences and engineering : MBE.

[20]  Valentin Flunkert,et al.  DeepAR: Probabilistic Forecasting with Autoregressive Recurrent Networks , 2017, International Journal of Forecasting.

[21]  Ingmar Schuster,et al.  Multi-variate Probabilistic Time Series Forecasting via Conditioned Normalizing Flows , 2020, ICLR.

[22]  Michael M. Bronstein,et al.  GRAND: Graph Neural Diffusion , 2021, ICML.

[23]  Tim Januschowski,et al.  Deep Factors for Forecasting , 2019, ICML.

[24]  Philip S. Yu,et al.  A Comprehensive Survey on Graph Neural Networks , 2019, IEEE Transactions on Neural Networks and Learning Systems.