High-Performance Time Series Prediction With Predictive Error Compensated Wavelet Neural Networks

Machine learning (ML) algorithms have gained prominence in time series prediction problems. Depending on the nature of the time series data, it can be difficult to build an accurate ML model with the proper structure and hyperparameters. In this study, we propose a predictive error compensation wavelet neural network model (PEC-WNN) for improving the prediction accuracy of chaotic and stochastic time series data. In the proposed model, an additional network is used for the prediction of the main network error to compensate the overall prediction error. The main network takes as inputs the time series data through moving frames in multiple-scales. The same structure and hyperparameter sets are applied for quite distinct four types of problems for verification of the robustness and accuracy of the proposed model. Specifically, the Mackey-Glass, Box-Jenkins, and Lorenz Attractor benchmark problems, as well as drought forecasting are used to characterize the performance of the model for chaotic and stochastic data cases. The results show that the PEC-WNN provides significantly more accurate predictions for all compared benchmark problems with respect to conventional machine learning and time series prediction methods without changing any hyperparameter or the structure. In addition, the time and space complexity of the PEC-WNN model is less than all other compared ML methods, including long short-term memory (LSTM) and convolutional neural networks (CNNs).

[1]  R. Scott Crowder,et al.  Predicting the Mackey-Glass Timeseries With Cascade-Correlation Learning , 1990 .

[2]  Waddah Waheeb,et al.  Forecasting the Behavior of Gas Furnace Multivariate Time Series Using Ridge Polynomial Based Neural Network Models , 2019, Int. J. Interact. Multim. Artif. Intell..

[3]  Heiga Zen,et al.  WaveNet: A Generative Model for Raw Audio , 2016, SSW.

[4]  C H López-Caraballo,et al.  Mackey-Glass noisy chaotic time series prediction by a swarm-optimized neural network , 2016 .

[5]  Meng Joo Er,et al.  NARMAX time series model prediction: feedforward and recurrent fuzzy neural network approaches , 2005, Fuzzy Sets Syst..

[6]  Amaury Lendasse,et al.  Methodology for long-term prediction of time series , 2007, Neurocomputing.

[7]  R. Tong The evaluation of fuzzy models derived from experimental data , 1980 .

[8]  H. Surmann,et al.  Self-Organizing and Genetic Algorithms for an Automatic Design of Fuzzy Control and Decision Systems , 1993 .

[9]  Samarjit Kar,et al.  A Hybridized Forecasting Method Based on Weight Adjustment of Neural Network Using Generalized Type-2 Fuzzy Set , 2018, Int. J. Fuzzy Syst..

[10]  Jinwu Gao,et al.  Using local learning with fuzzy transform: application to short term forecasting problems , 2019, Fuzzy Optimization and Decision Making.

[11]  Yulei Rao,et al.  A deep learning framework for financial time series using stacked autoencoders and long-short term memory , 2017, PloS one.

[12]  Guoqiang Peter Zhang,et al.  Time series forecasting using a hybrid ARIMA and neural network model , 2003, Neurocomputing.

[13]  Agnaldo J. R. Reis,et al.  Feature extraction via multiresolution analysis for short-term load forecasting , 2005, IEEE Transactions on Power Systems.

[14]  Sergio M. Vicente-Serrano,et al.  A Multiscalar Global Drought Dataset: The SPEIbase: A New Gridded Product for the Analysis of Drought Variability and Impacts , 2010 .

[15]  Jürgen Schmidhuber,et al.  Applying LSTM to Time Series Predictable through Time-Window Approaches , 2000, ICANN.

[16]  Qi-Lun Zheng,et al.  Chaotic Time Series Prediction Based on Evolving Recurrent Neural Networks , 2007, 2007 International Conference on Machine Learning and Cybernetics.

[17]  Marija Gorenc Novak,et al.  Prediction of stock price movement based on daily high prices , 2016 .

[18]  Waddah Waheeb,et al.  Ridge Polynomial Neural Network with Error Feedback for Time Series Forecasting , 2016, PloS one.

[19]  E. Lorenz Deterministic nonperiodic flow , 1963 .

[20]  Yoshua Bengio,et al.  Convolutional networks for images, speech, and time series , 1998 .

[21]  Michael R. Davenport,et al.  Continuous-time temporal back-propagation with adaptable time delays , 1993, IEEE Trans. Neural Networks.

[22]  Amir F. Atiya,et al.  An Empirical Comparison of Machine Learning Models for Time Series Forecasting , 2010 .

[23]  Ke Meng,et al.  Self-adaptive radial basis function neural network for short-term electricity price forecasting , 2009 .

[24]  Sergio M. Vicente-Serrano,et al.  A New Global 0.5° Gridded Dataset (1901–2006) of a Multiscalar Drought Index: Comparison with Current Drought Index Datasets Based on the Palmer Drought Severity Index , 2010 .

[25]  Fan Zhang,et al.  A review on time series forecasting techniques for building energy consumption , 2017 .

[26]  P. Young,et al.  Time series analysis, forecasting and control , 1972, IEEE Transactions on Automatic Control.

[27]  Yong-Zai Lu,et al.  Fuzzy Model Identification and Self-Learning for Dynamic Systems , 1987, IEEE Transactions on Systems, Man, and Cybernetics.

[28]  Jiwen Dong,et al.  Time-series prediction using a local linear wavelet neural network , 2006, Neurocomputing.

[29]  Chidchanok Lursinsap,et al.  Application of critical support vector machine to time series prediction , 2003, Proceedings of the 2003 International Symposium on Circuits and Systems, 2003. ISCAS '03..

[30]  Jiwen Dong,et al.  Time-series forecasting using flexible neural tree model , 2005, Inf. Sci..

[31]  Wei Zhang,et al.  Multivariate Chaotic Time Series Prediction Based on NARX Neural Networks , 2017 .

[32]  B. Berk Üstündag,et al.  An optimized two-level discrete wavelet implementation using residue number system , 2018, EURASIP J. Adv. Signal Process..

[33]  Gianluca Bontempi,et al.  Machine Learning Strategies for Time Series Forecasting , 2012, eBISS.

[34]  Nian Zhang,et al.  Time series prediction with recurrent neural networks trained by a hybrid PSO-EA algorithm , 2004, Neurocomputing.

[35]  Denisa Roberts,et al.  Neural Networks for Lorenz Map Prediction: A Trip Through Time. , 2020 .

[36]  Anastasios A. Tsonis,et al.  Nonlinear Prediction, Chaos, and Noise. , 1992 .