Multi-Step Time Series Forecasting with an Ensemble of Varied Length Mixture Models

Many real-world problems require modeling and forecasting of time series, such as weather temperature, electricity demand, stock prices and foreign exchange (FX) rates. Often, the tasks involve predicting over a long-term period, e.g. several weeks or months. Most existing time series models are inheritably for one-step prediction, that is, predicting one time point ahead. Multi-step or long-term prediction is difficult and challenging due to the lack of information and uncertainty or error accumulation. The main existing approaches, iterative and independent, either use one-step model recursively or treat the multi-step task as an independent model. They generally perform poorly in practical applications. In this paper, as an extension of the self-organizing mixture autoregressive (AR) model, the varied length mixture (VLM) models are proposed to model and forecast time series over multi-steps. The key idea is to preserve the dependencies between the time points within the prediction horizon. Training data are segmented to various lengths corresponding to various forecasting horizons, and the VLM models are trained in a self-organizing fashion on these segments to capture these dependencies in its component AR models of various predicting horizons. The VLM models form a probabilistic mixture of these varied length models. A combination of short and long VLM models and an ensemble of them are proposed to further enhance the prediction performance. The effectiveness of the proposed methods and their marked improvements over the existing methods are demonstrated through a number of experiments on synthetic data, real-world FX rates and weather temperatures.

[1]  Jouko Lampinen,et al.  Self-Organizing Maps for Spatial and Temporal AR Models , 1989 .

[2]  Josep L. Rosselló,et al.  High-Density Liquid-State Machine Circuitry for Time-Series Forecasting , 2016, Int. J. Neural Syst..

[3]  Sven F. Crone,et al.  Advances in forecasting with neural networks? Empirical evidence from the NN3 competition on time series prediction , 2011 .

[4]  Herbert Jaeger,et al.  A tutorial on training recurrent neural networks , covering BPPT , RTRL , EKF and the " echo state network " approach - Semantic Scholar , 2005 .

[5]  Amir F. Atiya,et al.  Multi-step-ahead prediction using dynamic recurrent neural networks , 2000, Neural Networks.

[6]  Thomas Martinetz,et al.  'Neural-gas' network for vector quantization and its application to time-series prediction , 1993, IEEE Trans. Neural Networks.

[7]  Hujun Yin,et al.  The Self-Organizing Maps: Background, Theories, Extensions and Applications , 2008, Computational Intelligence: A Compendium.

[8]  Hujun Yin,et al.  Self-Organising Mixture autoregressive Model for Non-Stationary Time Series Modelling , 2008, Int. J. Neural Syst..

[9]  Hugo Valadares Siqueira.  Unorganized machines to seasonal streamflow series forecasting , 2013, Int. J. Neural Syst..

[10]  Hujun Yin,et al.  A neural gas mixture autoregressive network for modelling and forecasting FX time series , 2014, Neurocomputing.

[11]  Pei-Chann Chang,et al.  Iterated time series prediction with multiple support vector regression models , 2013, Neurocomputing.

[12]  Amaury Lendasse,et al.  Methodology for long-term prediction of time series , 2007, Neurocomputing.

[13]  Antti Sorjamaa,et al.  Multiple-output modeling for multi-step-ahead time series forecasting , 2010, Neurocomputing.

[14]  Xinjun Peng,et al.  TSVR: An efficient Twin Support Vector Machine for regression , 2010, Neural Networks.

[15]  Paulo J. L. Adeodato,et al.  Continuous Dynamical Combination of Short and Long-Term Forecasts for Nonstationary Time Series , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[16]  Andrew Skabar,et al.  Direction‐of‐Change Financial Time Series Forecasting using a Similarity‐Based Classification Model , 2013 .

[17]  Rob J Hyndman,et al.  25 years of time series forecasting , 2006 .

[18]  John G. Taylor,et al.  The temporal Kohönen map , 1993, Neural Networks.

[19]  W. Li,et al.  On a mixture autoregressive model , 2000 .

[20]  Hujun Yin,et al.  A self-organising mixture autoregressive network for FX time series modelling and prediction , 2009, Neurocomputing.

[21]  J. Muth.  Optimal Properties of Exponentially Weighted Forecasts , 1960 .

[22]  Thomas Voegtlin,et al.  Recursive self-organizing maps , 2002, Neural Networks.

[23]  Jun Wang,et al.  Chaotic Time Series Prediction Based on a Novel Robust Echo State Network , 2012, IEEE Transactions on Neural Networks and Learning Systems.

[24]  Rob J Hyndman,et al.  Another look at measures of forecast accuracy , 2006 .

[25]  Kurt Hornik,et al.  Multilayer feedforward networks are universal approximators , 1989, Neural Networks.

[26]  Paul Goodwin.  Using Naïve Forecasts to Assess Limits to Forecast Accuracy and the Quality of Fit of Forecasts to Time Series Data , 2014 .

[27]  Jukka Heikkonen,et al.  Time Series Predicition using Recurrent SOM with Local Linear Models , 1997 .

[28]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[29]  Afzel Noore,et al.  Software Reliability Prediction Using Recurrent Neural Network With Bayesian Regularization , 2004, Int. J. Neural Syst..

[30]  Johan A. K. Suykens,et al.  Financial time series prediction using least squares support vector machines within the evidence framework , 2001, IEEE Trans. Neural Networks.

[31]  Hujun Yin,et al.  Generalized Self-Organizing Mixture Autoregressive Model for Modeling Financial Time Series , 2009, ICANN.

[32]  Inés María Galván,et al.  A Selective Learning Method to Improve the Generalization of Multilayer Feedforward Neural Networks , 2001, Int. J. Neural Syst..

[33]  Hujun Yin,et al.  Self-organizing mixture networks for probability density estimation , 2001, IEEE Trans. Neural Networks.

[34]  Nick S. Jones,et al.  Highly Comparative Feature-Based Time-Series Classification , 2014, IEEE Transactions on Knowledge and Data Engineering.

[35]  M. V. Velzen,et al.  Self-organizing maps , 2007 .

[36]  Ah Chung Tsoi,et al.  Noisy Time Series Prediction using Recurrent Neural Networks and Grammatical Inference , 2001, Machine Learning.

[37]  Kazuyuki Murase,et al.  Faster Training Using Fusion of Activation Functions for Feed Forward Neural Networks , 2009, Int. J. Neural Syst..

[38]  J. Stock,et al.  A Comparison of Direct and Iterated Multistep Ar Methods for Forecasting Macroeconomic Time Series , 2005 .

[39]  Pablo M. Granitto,et al.  Artificial Neural Network Learning of Nonstationary Behavior in Time Series , 2003, Int. J. Neural Syst..

[40]  Michel Verleysen,et al.  Time series forecasting: Obtaining long term trends with self-organizing maps , 2005, Pattern Recognit. Lett..