Time Series Forecasting Using LSTM Networks: A Symbolic Approach

Machine learning methods trained on raw numerical time series data exhibit fundamental limitations such as a high sensitivity to the hyper parameters and even to the initialization of random weights. A combination of a recurrent neural network with a dimension-reducing symbolic representation is proposed and applied for the purpose of time series forecasting. It is shown that the symbolic representation can help to alleviate some of the aforementioned problems and, in addition, might allow for faster training without sacrificing the forecast performance.

[1]  Houshang Darabi,et al.  LSTM Fully Convolutional Networks for Time Series Classification , 2017, IEEE Access.

[2]  Yulei Rao,et al.  A deep learning framework for financial time series using stacked autoencoders and long-short term memory , 2017, PloS one.

[3]  Navdeep Jaitly,et al.  Towards End-To-End Speech Recognition with Recurrent Neural Networks , 2014, ICML.

[4]  Rob J Hyndman,et al.  Another look at measures of forecast accuracy , 2006 .

[5]  Ah Chung Tsoi,et al.  Noisy Time Series Prediction using Recurrent Neural Networks and Grammatical Inference , 2001, Machine Learning.

[6]  Quoc V. Le,et al.  Sequence to Sequence Learning with Neural Networks , 2014, NIPS.

[7]  S. Chiba,et al.  Dynamic programming algorithm optimization for spoken word recognition , 1978 .

[8]  Min Han,et al.  Prediction of chaotic time series based on the recurrent predictor neural network , 2004, IEEE Transactions on Signal Processing.

[9]  Lovekesh Vig,et al.  Long Short Term Memory Networks for Anomaly Detection in Time Series , 2015, ESANN.

[10]  Amir F. Atiya,et al.  A comparison between neural-network forecasting techniques-case study: river flow forecasting , 1999, IEEE Trans. Neural Networks.

[11]  Jürgen Schmidhuber,et al.  Evolino: Hybrid Neuroevolution / Optimal Linear Search for Sequence Prediction , 2005, IJCAI 2005.

[12]  K. Nikolopoulos,et al.  The theta model: a decomposition approach to forecasting , 2000 .

[13]  George Bernard Shaw,et al.  LONG-RANGE FORECASTING From Crystal Ball to Computer , 2010 .

[14]  Nathalie Japkowicz,et al.  Anomaly Detection in Automobile Control Network Data with Long Short-Term Memory Networks , 2016, 2016 IEEE International Conference on Data Science and Advanced Analytics (DSAA).

[15]  Kai Chen,et al.  A LSTM-based method for stock returns prediction: A case study of China stock market , 2015, 2015 IEEE International Conference on Big Data (Big Data).

[16]  Li Li,et al.  Using LSTM and GRU neural network methods for traffic flow prediction , 2016, 2016 31st Youth Academic Annual Conference of Chinese Association of Automation (YAC).

[17]  Yoshua Bengio,et al.  On the Properties of Neural Machine Translation: Encoder–Decoder Approaches , 2014, SSST@EMNLP.

[18]  Yoshua Bengio,et al.  Understanding the difficulty of training deep feedforward neural networks , 2010, AISTATS.

[19]  Stefan Güttel,et al.  ABBA: adaptive Brownian bridge-based symbolic aggregation of time series , 2020, Data Mining and Knowledge Discovery.

[20]  Yong Qi,et al.  Multi-step Ahead Time Series Forecasting for Different Data Patterns Based on LSTM Recurrent Neural Network , 2017, WISA.

[21]  Louis B. Rall,et al.  Automatic differentiation , 1981 .

[22]  Jürgen Schmidhuber,et al.  LSTM: A Search Space Odyssey , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[23]  Ilya Sutskever,et al.  Language Models are Unsupervised Multitask Learners , 2019 .

[24]  Yoshua Bengio,et al.  Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation , 2014, EMNLP.

[25]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[26]  Rob J Hyndman,et al.  Unmasking the Theta Method , 2003 .

[27]  Geoffrey E. Hinton,et al.  Speech recognition with deep recurrent neural networks , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[28]  Desmond J. Higham,et al.  Edinburgh Research Explorer Deep learning: an introduction for applied mathematicians , 2022 .

[29]  Peter Stagge,et al.  Recurrent neural networks for time series classification , 2003, Neurocomputing.

[30]  Sepp Hochreiter,et al.  The Vanishing Gradient Problem During Learning Recurrent Neural Nets and Problem Solutions , 1998, Int. J. Uncertain. Fuzziness Knowl. Based Syst..

[31]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[32]  Luca Antiga,et al.  Automatic differentiation in PyTorch , 2017 .

[33]  Yan Liu,et al.  Recurrent Neural Networks for Multivariate Time Series with Missing Values , 2016, Scientific Reports.

[34]  T. Munich,et al.  Offline Handwriting Recognition with Multidimensional Recurrent Neural Networks , 2008, NIPS.

[35]  Jürgen Schmidhuber,et al.  Unconstrained On-line Handwriting Recognition with Recurrent Neural Networks , 2007, NIPS.

[36]  Eugen Diaconescu,et al.  The use of NARX neural networks to predict chaotic time series , 2008 .

[37]  Evangelos Spiliotis,et al.  Statistical and Machine Learning forecasting methods: Concerns and ways forward , 2018, PloS one.

[38]  Evangelos Spiliotis,et al.  The M4 Competition: 100,000 time series and 61 forecasting methods , 2020 .

[39]  Robert J. Genetski,et al.  Long-Range Forecasting: From Crystal Ball to Computer , 1981 .