Learning Decomposed Spatial Relations for Multi-Variate Time-Series Modeling

Modeling multi-variate time-series (MVTS) data is a long-standing research subject and has found wide applications. Recently, there is a surge of interest in modeling spatial relations between variables as graphs, i.e., first learning one static graph for each dataset and then exploiting the graph structure via graph neural networks. However, as spatial relations may differ substantially across samples, building one static graph for all the samples inherently limits flexibility and severely degrades the performance in practice. To address this issue, we propose a framework for fine-grained modeling and utilization of spatial correlation between variables. By analyzing the statistical properties of real-world datasets, a universal decomposition of spatial correlation graphs is first identified. Specifically, the hidden spatial relations can be decomposed into a prior part, which applies across all the samples, and a dynamic part, which varies between samples, and building different graphs is necessary to model these relations. To better coordinate the learning of the two relational graphs, we propose a min-max learning paradigm that not only regulates the common part of different dynamic graphs but also guarantees spatial distinguishability among samples. The experimental results show that our proposed model outperforms the state-of-the-art baseline methods on both time-series forecasting and time-series point prediction tasks.

[1]  Philip S. Yu,et al.  Graph Structure Learning with Variational Information Bottleneck , 2021, AAAI.

[2]  Yanjie Fu,et al.  Dynamic and Multi-faceted Spatio-temporal Deep Learning for Traffic Speed Forecasting , 2021, KDD.

[3]  Jianmin Wang,et al.  Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecasting , 2021, NeurIPS.

[4]  Jiang Bian,et al.  Learning Multiple Stock Trading Patterns with Temporal Routing Adaptor and Optimal Transport , 2021, KDD.

[5]  Dominique Beaini,et al.  Rethinking Graph Transformers with Spectral Attention , 2021, NeurIPS.

[6]  Yanfang Ye,et al.  Heterogeneous Graph Structure Learning for Graph Neural Networks , 2021, AAAI.

[7]  Jared A. Dunnmon,et al.  Self-Supervised Graph Neural Networks for Improved Electroencephalographic Seizure Analysis , 2021, ICLR.

[8]  Carl Yang,et al.  A Survey on Graph Structure Learning: Progress and Opportunities , 2021, 2103.03036.

[9]  Qi Zhang,et al.  Spectral Temporal Graph Neural Network for Multivariate Time-series Forecasting , 2020, NeurIPS.

[10]  Xiaojun Chang,et al.  Connecting the Dots: Multivariate Time Series Forecasting with Graph Neural Networks , 2020, KDD.

[11]  Suhang Wang,et al.  Graph Structure Learning for Robust Graph Neural Networks , 2020, KDD.

[12]  Chang-Tien Lu,et al.  TapNet: Multivariate Time Series Classification with Attentional Prototypical Network , 2020, AAAI.

[13]  Gerhard Nahler,et al.  Pearson Correlation Coefficient , 2020, Definitions.

[14]  Ruohong Zhang,et al.  Graph-Revised Convolutional Network , 2019, ECML/PKDD.

[15]  Geoffrey I. Webb,et al.  ROCKET: exceptionally fast and accurate time series classification using random convolutional kernels , 2019, Data Mining and Knowledge Discovery.

[16]  Geoffrey I. Webb,et al.  InceptionTime: Finding AlexNet for time series classification , 2019, Data Mining and Knowledge Discovery.

[17]  Jing Jiang,et al.  Graph WaveNet for Deep Spatial-Temporal Graph Modeling , 2019, IJCAI.

[18]  A. Galstyan,et al.  MixHop: Higher-Order Graph Convolutional Architectures via Sparsified Neighborhood Mixing , 2019, ICML.

[19]  N. J. Stevenson,et al.  Descriptor : A dataset of neonatal EEG recordings with seizure annotations , 2019 .

[20]  Hung-yi Lee,et al.  Temporal pattern attention for multivariate time series forecasting , 2018, Machine Learning.

[21]  Vladlen Koltun,et al.  An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling , 2018, ArXiv.

[22]  Houshang Darabi,et al.  Multivariate LSTM-FCNs for Time Series Classification , 2018, Neural Networks.

[23]  Ruoyu Li,et al.  Adaptive Graph Convolutional Neural Networks , 2018, AAAI.

[24]  Zhanxing Zhu,et al.  Spatio-temporal Graph Convolutional Neural Network: A Deep Learning Framework for Traffic Forecasting , 2017, IJCAI.

[25]  Cyrus Shahabi,et al.  Diffusion Convolutional Recurrent Neural Network: Data-Driven Traffic Forecasting , 2017, ICLR.

[26]  Guokun Lai,et al.  Modeling Long- and Short-Term Temporal Patterns with Deep Neural Networks , 2017, SIGIR.

[27]  Geoffrey E. Hinton,et al.  Layer Normalization , 2016, ArXiv.

[28]  Yoshua Bengio,et al.  Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation , 2014, EMNLP.

[29]  Jason Lines,et al.  Classification of time series by shapelet transformation , 2013, Data Mining and Knowledge Discovery.

[30]  S Roberts,et al.  Gaussian processes for time-series modelling , 2013, Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[31]  P. Young,et al.  Time series analysis, forecasting and control , 1972, IEEE Transactions on Automatic Control.