SSDNet: State Space Decomposition Neural Network for Time Series Forecasting

In this paper, we present SSDNet, a novel deep learning approach for time series forecasting. SSDNet combines the Transformer architecture with state space models to provide probabilistic and interpretable forecasts, including trend and seasonality components and previous time steps important for the prediction. The Transformer architecture is used to learn the temporal patterns and estimate the parameters of the state space model directly and efficiently, without the need for Kalman filters. We comprehensively evaluate the performance of SSDNet on five data sets, showing that SSDNet is an effective method in terms of accuracy and speed, outperforming state-of-the-art deep learning and statistical methods, and able to provide meaningful trend and seasonality components.

[1]  James Hensman,et al.  Identification of Gaussian Process State Space Models , 2017, NIPS.

[2]  Yoshua Bengio,et al.  BinaryConnect: Training Deep Neural Networks with binary weights during propagations , 2015, NIPS.

[3]  Tim Januschowski,et al.  Deep Factors for Forecasting , 2019, ICML.

[4]  Nicolas Chapados,et al.  N-BEATS: Neural basis expansion analysis for interpretable time series forecasting , 2019, ICLR.

[5]  Irena Koprinska,et al.  Temporal Convolutional Attention Neural Networks for Time Series Forecasting , 2021, 2021 International Joint Conference on Neural Networks (IJCNN).

[6]  George Athanasopoulos,et al.  Forecasting: principles and practice , 2013 .

[7]  Harit Pandya,et al.  Recurrent Kalman Networks: Factorized Inference in High-Dimensional Deep Feature Spaces , 2019, ICML.

[8]  Nino Antulov-Fantulin,et al.  Exploring Interpretable LSTM Neural Networks over Multi-Variable Data , 2019, ICML.

[9]  Matthias W. Seeger,et al.  Deep State Space Models for Time Series Forecasting , 2018, NeurIPS.

[10]  Wenhu Chen,et al.  Enhancing the Locality and Breaking the Memory Bottleneck of Transformer on Time Series Forecasting , 2019, NeurIPS.

[11]  Yang Lin,et al.  SpringNet: Transformer and Spring DTW for Time Series Forecasting , 2020, ICONIP.

[12]  Guokun Lai,et al.  Modeling Long- and Short-Term Temporal Patterns with Deep Neural Networks , 2017, SIGIR.

[13]  Valentin Flunkert,et al.  DeepAR: Probabilistic Forecasting with Autoregressive Recurrent Networks , 2017, International Journal of Forecasting.

[14]  Benjamin Letham,et al.  Forecasting at Scale , 2018, PeerJ Prepr..

[15]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[16]  Hui Xiong,et al.  Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting , 2020, AAAI.

[17]  Yaohui Jin,et al.  Learning Interpretable Deep State Space Model for Probabilistic Time Series Forecasting , 2019, IJCAI.

[18]  Melvin J. Hinich,et al.  Time Series Analysis by State Space Methods , 2001 .

[19]  Inderjit S. Dhillon,et al.  Temporal Regularized Matrix Factorization for High-dimensional Time Series Prediction , 2016, NIPS.

[20]  Uri Shalit,et al.  Deep Kalman Filters , 2015, ArXiv.