A deep learning model to effectively capture mutation information in multivariate time series prediction

Abstract In real-world complex multivariate time series data, mutation phenomena can significantly affect variation rules of target series. Meanwhile, there is no specific learning mechanism for the current deep learning model to capture mutation information in time series prediction. To this end, we propose a new deep learning model to capture mutation information between data. To capture the impact of mutation information on target series, a new function mapping is designed in the attention mechanism of the encoder to process the fusion of historical hidden state and cell state information; and an LSTM with transformation mechanism is proposed in the encoder to process the input information flow and learn the mutation information. In addition, an adaptive self-paced curriculum learning mechanism is designed to obtain mutation information that may be ignored among mini-batch samples. Finally, we define an objective function for multivariate time series prediction, which can extract the influence of temporal correlation information and mutation information within the data on target series. Our model can achieve superior performance than all baseline methods on five real-world datasets in different fields.

[1]  Jun Hu,et al.  Transformation-gated LSTM: efficient capture of short-term mutation dependencies for multivariate time series prediction tasks , 2019, 2019 International Joint Conference on Neural Networks (IJCNN).

[2]  Xuan Liang,et al.  Assessing Beijing's PM2.5 pollution: severity, weather impact, APEC and winter heating , 2015, Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[3]  Zhiguo Gong,et al.  A nonparametric model for online topic discovery with word embeddings , 2019, Inf. Sci..

[4]  Mehdi Khashei,et al.  A novel hybridization of artificial neural networks and ARIMA models for time series forecasting , 2011, Appl. Soft Comput..

[5]  Mustafa Gokce Baydogan,et al.  Autoregressive forests for multivariate time series modeling , 2018, Pattern Recognit..

[6]  Carola-Bibiane Schönlieb,et al.  A Combined First and Second Order Variational Approach for Image Reconstruction , 2012, Journal of Mathematical Imaging and Vision.

[7]  Quoc V. Le,et al.  Sequence to Sequence Learning with Neural Networks , 2014, NIPS.

[8]  Yanru Zhang,et al.  Using an ARIMA-GARCH Modeling Approach to Improve Subway Short-Term Ridership Forecasting Accounting for Dynamic Volatility , 2018, IEEE Transactions on Intelligent Transportation Systems.

[9]  Jonathan M. Garibaldi,et al.  Neural networks and AdaBoost algorithm based ensemble models for enhanced forecasting of nonlinear time series , 2014, 2014 International Joint Conference on Neural Networks (IJCNN).

[10]  Peter Tiño,et al.  Learning long-term dependencies in NARX recurrent neural networks , 1996, IEEE Trans. Neural Networks.

[11]  Nassir Navab,et al.  Revisiting NARX Recurrent Neural Networks for Long-Term Dependencies , 2017, ArXiv.

[12]  Shie-Jue Lee,et al.  A weighted LS-SVM based learning system for time series forecasting , 2015, Inf. Sci..

[13]  Hamido Fujita,et al.  A novel forecasting method based on multi-order fuzzy time series and technical analysis , 2016, Inf. Sci..

[14]  Bo Zong,et al.  Tensorized LSTM with Adaptive Shared Memory for Learning Trends in Multivariate Time Series , 2020, AAAI.

[15]  Kihwan Nam,et al.  Financial news-based stock movement prediction using causality analysis of influence in the Korean stock market , 2019, Decis. Support Syst..

[16]  Xavier Bresson,et al.  Structured Sequence Modeling with Graph Convolutional Recurrent Networks , 2016, ICONIP.

[17]  J. Friedman.  Greedy function approximation: A gradient boosting machine. , 2001 .

[18]  Kenli Li,et al.  Multiple convolutional neural networks for multivariate time series prediction , 2019, Neurocomputing.

[19]  Yoshua Bengio,et al.  Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling , 2014, ArXiv.

[20]  Robert Jenssen,et al.  Time Series Cluster Kernel for Learning Similarities between Multivariate Time Series with Missing Data , 2017, Pattern Recognit..

[21]  Qin Zhang,et al.  Time series feature learning with labeled and unlabeled data , 2019, Pattern Recognit..

[22]  Nino Antulov-Fantulin,et al.  Exploring Interpretable LSTM Neural Networks over Multi-Variable Data , 2019, ICML.