Not Only Pairwise Relationships: Fine-Grained Relational Modeling for Multivariate Time Series Forecasting

Recent graph-based methods achieve significant success in multivariate time series modeling and forecasting due to their ability to handle relationships among time series variables. However, only pairwise relationships are considered in most existing works. They ignore beyond-pairwise relationships and their potential categories in practical scenarios, which leads to incomprehensive relationship learning for multivariate time series forecasting. In this paper, we present ReMo, a Relational Modeling-based method, to promote fine-grained relational learning among multivariate time series data. Firstly, by treating time series variables and complex relationships as nodes and hyperedges, we extract multi-view hypergraphs from data to capture beyond-pairwise relationships. Secondly, a novel hypergraph message passing strategy is designed to characterize both nodes and hyperedges by inferring the potential categories of relationships and further distinguishing their impacts on time series variables. By integrating these two modules into the time series forecasting framework, ReMo effectively improves the performance of multivariate time series forecasting. The experimental results on seven commonly used datasets from different domains demonstrate the superiority of our model.

[1]  Mykel J. Kochenderfer,et al.  Interaction Modeling with Multiplex Attention , 2022, NeurIPS.

[2]  Yan Huang,et al.  Regularized Graph Structure Learning with Semantic Knowledge for Multi-variates Time-Series Forecasting , 2022, IJCAI.

[3]  Yanjie Fu,et al.  Learning the Evolutionary and Multi-scale Graph Structure for Multivariate Time Series Forecasting , 2022, KDD.

[4]  Siheng Chen,et al.  GroupNet: Multiscale Hypergraph Neural Networks for Trajectory Prediction with Relational Reasoning , 2022, 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[5]  Yulia R. Gel,et al.  Z-GCNETs: Time Zigzags at Graph Convolutional Networks for Time Series Forecasting , 2021, ICML.

[6]  Qi Zhang,et al.  Spectral Temporal Graph Neural Network for Multivariate Time-series Forecasting , 2020, NeurIPS.

[7]  Lina Yao,et al.  Adaptive Graph Convolutional Recurrent Network for Traffic Forecasting , 2020, NeurIPS.

[8]  Xiaojun Chang,et al.  Connecting the Dots: Multivariate Time Series Forecasting with Graph Neural Networks , 2020, KDD.

[9]  Bryan Lim,et al.  Time-series forecasting with deep learning: a survey , 2020, Philosophical Transactions of the Royal Society A.

[10]  Thomas G. Dietterich What is machine learning? , 2015, Archives of Disease in Childhood.

[11]  Cheng Wang,et al.  GMAN: A Graph Multi-Attention Network for Traffic Prediction , 2019, AAAI.

[12]  Jing Jiang,et al.  Graph WaveNet for Deep Spatial-Temporal Graph Modeling , 2019, IJCAI.

[13]  Manfred Mudelsee,et al.  Trend analysis of climate time series: A review of methods , 2019, Earth-Science Reviews.

[14]  Song Bai,et al.  Hypergraph Convolution and Hypergraph Attention , 2019, Pattern Recognit..

[15]  Yue Gao,et al.  Hypergraph Neural Networks , 2018, AAAI.

[16]  Hung-yi Lee,et al.  Temporal pattern attention for multivariate time series forecasting , 2018, Machine Learning.

[17]  Partha Pratim Talukdar,et al.  HyperGCN: A New Method of Training Graph Convolutional Networks on Hypergraphs , 2018 .

[18]  Haoyu Yang,et al.  A Treatment Engine by Predicting Next-Period Prescriptions , 2018, KDD.

[19]  R. Zemel,et al.  Neural Relational Inference for Interacting Systems , 2018, ICML.

[20]  Zhanxing Zhu,et al.  Spatio-temporal Graph Convolutional Neural Network: A Deep Learning Framework for Traffic Forecasting , 2017, IJCAI.

[21]  Cyrus Shahabi,et al.  Diffusion Convolutional Recurrent Neural Network: Data-Driven Traffic Forecasting , 2017, ICLR.

[22]  Guokun Lai,et al.  Modeling Long- and Short-Term Temporal Patterns with Deep Neural Networks , 2017, SIGIR.

[23]  Gautier Marti,et al.  Autoregressive Convolutional Neural Networks for Asynchronous Time Series , 2017, ICML.

[24]  Ben Poole,et al.  Categorical Reparameterization with Gumbel-Softmax , 2016, ICLR.

[25]  Yee Whye Teh,et al.  The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables , 2016, ICLR.

[26]  S Roberts,et al.  Gaussian processes for time-series modelling , 2013, Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[27]  Bernhard Schölkopf,et al.  Learning with Hypergraphs: Clustering, Classification, and Embedding , 2006, NIPS.

[28]  B. Ripley,et al.  Pattern Recognition , 1968, Nature.