Spatio-Temporal Neural Networks for Space-Time Series Forecasting and Relations Discovery

We introduce a dynamical spatio-temporal model formalized as a recurrent neural network for forecasting time series of spatial processes, i.e. series of observations sharing temporal and spatial dependencies. The model learns these dependencies through a structured latent dynamical component, while a decoder predicts the observations from the latent representations. We consider several variants of this model, corresponding to different prior hypothesis about the spatial relations between the series. The model is evaluated and compared to state-of-the-art baselines, on a variety of forecasting problems representative of different application areas: epidemiology, geo-spatial statistics and car-traffic prediction. Besides these evaluations, we also describe experiments showing the ability of this approach to extract relevant spatial relations.

[1]  Yoshua Bengio,et al.  A Recurrent Latent Variable Model for Sequential Data , 2015, NIPS.

[2]  Geoffrey E. Hinton,et al.  Generating Text with Recurrent Neural Networks , 2011, ICML.

[3]  Yann LeCun,et al.  Dynamic Factor Graphs for Time Series Modeling , 2009, ECML/PKDD.

[4]  Max Welling,et al.  Auto-Encoding Variational Bayes , 2013, ICLR.

[5]  Nitish Srivastava,et al.  Unsupervised Learning of Video Representations using LSTMs , 2015, ICML.

[6]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[7]  Roger Woodard,et al.  Interpolation of Spatial Data: Some Theory for Kriging , 1999, Technometrics.

[8]  Richard S. Zemel,et al.  Gated Graph Sequence Neural Networks , 2015, ICLR.

[9]  Christian Osendorfer,et al.  Learning Stochastic Recurrent Networks , 2014, NIPS 2014.

[10]  Uri Shalit,et al.  Deep Kalman Filters , 2015, ArXiv.

[11]  Les E. Atlas,et al.  Recurrent neural networks and robust time series prediction , 1994, IEEE Trans. Neural Networks.

[12]  Qi Yu,et al.  Fast Multivariate Spatio-temporal Analysis via Low Rank Tensor Learning , 2014, NIPS.

[13]  Geoffrey E. Hinton,et al.  On the importance of initialization and momentum in deep learning , 2013, ICML.

[14]  Rob J Hyndman,et al.  25 years of time series forecasting , 2006 .

[15]  Xing Xie,et al.  T-drive: driving directions based on taxi trajectories , 2010, GIS '10.

[16]  Christopher K. Wikle,et al.  Modern perspectives on statistics for spatio‐temporal data , 2015 .

[17]  John V. Guttag,et al.  Learning Connections in Financial Time Series , 2013, ICML.

[18]  Guangzhong Sun,et al.  Driving with knowledge from the physical world , 2011, KDD.

[19]  Yoshua Bengio,et al.  Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation , 2014, EMNLP.

[20]  Hema Swetha Koppula,et al.  Learning Spatio-Temporal Structure from RGB-D Videos for Human Activity Detection and Anticipation , 2013, ICML.

[21]  Klaus-Robert Müller,et al.  Optimizing spatio-temporal filters for improving Brain-Computer Interfacing , 2005, NIPS.

[22]  Gunnar Rätsch,et al.  Using support vector machines for time series prediction , 1999 .

[23]  M. Hooten,et al.  A general science-based framework for dynamical spatio-temporal models , 2010 .

[24]  Yoshua Bengio,et al.  Neural net language models , 2008, Scholarpedia.

[25]  Rob J. Hyndman,et al.  Boosting multi-step autoregressive forecasts , 2014, ICML.

[26]  Yan Wu,et al.  Convolutional deep belief networks for feature extraction of EEG signal , 2014, 2014 International Joint Conference on Neural Networks (IJCNN).

[27]  Dit-Yan Yeung,et al.  Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting , 2015, NIPS.

[28]  Leonhard Held,et al.  Gaussian Markov Random Fields: Theory and Applications , 2005 .

[29]  Michelangelo Ceci,et al.  Predictive Modeling of PV Energy Production: How to Set Up the Learning Task for a Better Prediction? , 2017, IEEE Transactions on Industrial Informatics.

[30]  Geoffrey E. Hinton,et al.  Speech recognition with deep recurrent neural networks , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.