A novel time series forecasting model with deep learning

Abstract Time series forecasting is emerging as one of the most important branches of big data analysis. However, traditional time series forecasting models can not effectively extract good enough sequence data features and often result in poor forecasting accuracy. In this paper, a novel time series forecasting model, named SeriesNet, which can fully learn features of time series data in different interval lengths. The SeriesNet consists of two networks. The LSTM network aims to learn holistic features and to reduce dimensionality of multi-conditional data, and the dilated causal convolution network aims to learn different time interval. This model can learn multi-range and multi-level features from time series data, and has higher predictive accuracy compared those models using fixed time intervals. Moreover, this model adopts residual learning and batch normalization to improve generalization. Experimental results show our model has higher forecasting accuracy and has greater stableness on several typical time series datasets.

[1]  Geoffrey E. Hinton,et al.  Acoustic Modeling Using Deep Belief Networks , 2012, IEEE Transactions on Audio, Speech, and Language Processing.

[2]  T. Chow,et al.  Nonlinear autoregressive integrated neural network model for short-term load forecasting , 1996 .

[3]  Yoshua Bengio,et al.  A Neural Probabilistic Language Model , 2003, J. Mach. Learn. Res..

[4]  Haifeng Wang,et al.  Comparison of SVM and LS-SVM for Regression , 2005, 2005 International Conference on Neural Networks and Brain.

[5]  G. Box,et al.  Distribution of Residual Autocorrelations in Autoregressive-Integrated Moving Average Time Series Models , 1970 .

[6]  Yu-Bin Yang,et al.  Image Restoration Using Very Deep Convolutional Encoder-Decoder Networks with Symmetric Skip Connections , 2016, NIPS.

[7]  T. Martin McGinnity,et al.  Predicting a Chaotic Time Series using Fuzzy Neural network , 1998, Inf. Sci..

[8]  I. Osorio,et al.  Intrinsic time-scale decomposition: time–frequency–energy analysis and real-time filtering of non-stationary signals , 2007, Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[9]  Guoqiang Peter Zhang,et al.  Time series forecasting using a hybrid ARIMA and neural network model , 2003, Neurocomputing.

[10]  Nor Zila Abdul Hamid,et al.  Modeling of prediction system : an application of the nearest neighbor approach to chaotic data , 2013 .

[11]  Alex Graves,et al.  Conditional Image Generation with PixelCNN Decoders , 2016, NIPS.

[12]  Les E. Atlas,et al.  Recurrent neural networks and robust time series prediction , 1994, IEEE Trans. Neural Networks.

[13]  Jiwen Dong,et al.  Time-series prediction using a local linear wavelet neural network , 2006, Neurocomputing.

[14]  Kurt Hornik,et al.  Multilayer feedforward networks are universal approximators , 1989, Neural Networks.

[15]  Satish T. S. Bukkapatnam,et al.  Time series forecasting for nonlinear and non-stationary processes: a review and comparative study , 2015 .

[16]  Victor L. Berardi,et al.  Time series forecasting with neural network ensembles: an application for exchange rate prediction , 2001, J. Oper. Res. Soc..

[17]  Lutgarde M. C. Buydens,et al.  Using support vector machines for time series prediction , 2003 .

[18]  Çagdas Hakan Aladag,et al.  Forecasting nonlinear time series with a hybrid methodology , 2009, Appl. Math. Lett..

[19]  Mathew M. Noel,et al.  Control of a nonlinear liquid level system using a new artificial neural network based reinforcement learning approach , 2014, Appl. Soft Comput..

[20]  Jian Sun,et al.  Identity Mappings in Deep Residual Networks , 2016, ECCV.

[21]  Qiong Shen,et al.  Financial Time Series Forecasting Using Support Vector Machine , 2014, 2014 Tenth International Conference on Computational Intelligence and Security.

[22]  Jürgen Schmidhuber,et al.  Deep learning in neural networks: An overview , 2014, Neural Networks.

[23]  Rafal A. Angryk,et al.  Evaluating Preprocessing Strategies for Time Series Prediction using Deep Learning Architectures , 2017, FLAIRS Conference.

[24]  Edwin L. Bradley,et al.  The Equivalence of Maximum Likelihood and Weighted Least Squares Estimates in the Exponential Family , 1973 .

[25]  Xiangtao Zheng,et al.  Exploring Models and Data for Remote Sensing Image Caption Generation , 2017, IEEE Transactions on Geoscience and Remote Sensing.

[26]  Carl E. Rasmussen,et al.  Gaussian processes for machine learning , 2005, Adaptive computation and machine learning.

[27]  Francis Eng Hock Tay,et al.  Support vector machine with adaptive parameters in financial time series forecasting , 2003, IEEE Trans. Neural Networks.

[28]  Billy M. Williams,et al.  Urban Freeway Traffic Flow Prediction: Application of Seasonal Autoregressive Integrated Moving Average and Exponential Smoothing Models , 1998 .

[29]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[30]  Takayuki Osogami,et al.  Nonlinear Dynamic Boltzmann Machines for Time-Series Prediction , 2017, AAAI.

[31]  Gunnar Rätsch,et al.  Predicting Time Series with Support Vector Machines , 1997, ICANN.

[32]  Ashu Jain,et al.  Hybrid neural network models for hydrologic time series forecasting , 2007, Appl. Soft Comput..

[33]  Quoc V. Le,et al.  Sequence to Sequence Learning with Neural Networks , 2014, NIPS.

[34]  Clifford M. Hurvich,et al.  Regression and time series model selection in small samples , 1989 .

[35]  Trevor Darrell,et al.  Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation , 2013, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[36]  Xiangtao Zheng,et al.  Remote Sensing Scene Classification by Unsupervised Representation Learning , 2017, IEEE Transactions on Geoscience and Remote Sensing.

[37]  Yoshua Bengio,et al.  Learning long-term dependencies with gradient descent is difficult , 1994, IEEE Trans. Neural Networks.

[38]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[39]  Dong Yu,et al.  Context-Dependent Pre-Trained Deep Neural Networks for Large-Vocabulary Speech Recognition , 2012, IEEE Transactions on Audio, Speech, and Language Processing.

[40]  H. Akaike.  Fitting autoregressive models for prediction , 1969 .

[41]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[42]  Yoshua Bengio,et al.  Gated Feedback Recurrent Neural Networks , 2015, ICML.

[43]  Ying Zhang,et al.  Batch normalized recurrent neural networks , 2015, 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[44]  Gwilym M. Jenkins,et al.  Time series analysis, forecasting and control , 1972 .

[45]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[46]  Jürgen Schmidhuber,et al.  Applying LSTM to Time Series Predictable through Time-Window Approaches , 2000, ICANN.

[47]  Jeffrey Dean,et al.  Distributed Representations of Words and Phrases and their Compositionality , 2013, NIPS.

[48]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.