An improved Elman neural network with piecewise weighted gradient for time series prediction

Abstract Time series prediction is an important tool for system analysis. Traditional forecasting methods usually achieve prediction by establishing static model or only mining the information from the sequence itself, but without considering the dynamics of the system or its triggering factors. Elman neural network (ENN) is a dynamical model that can remember historical states. In order to better establish the real multivariate time series model and improve the single-step prediction accuracy, this study proposes an improved regularized ENN method based on piecewise time weighted gradient (PWRENN). Each time the parameters of PWRENN model are updated, the weighted gradient is calculated by weighing the current gradient and the previous gradients according to a monotonically increasing temporal function. By considering both the current gradient and previous gradients, PWRENN is able to simulate the time-lag effect in the time series prediction problem. Moreover, the system environment may change with time, and the historical data of the same slot at a different time should have different effects on the predicted data. This work uses a piecewise time function with different parameters at different time periods to more accurately model the real system. In PWRENN, L2 regularization is adopted to solve the overfitting problem and enhance the generalization performance. Furthermore, the effectiveness of the proposed method is verified in an artificial Mackey-Glass time series prediction and three landslide displacement predictions.

[1]  Suzanne Lacasse,et al.  Displacement prediction in colluvial landslides, Three Gorges Reservoir, China , 2013, Landslides.

[2]  Gwilym M. Jenkins,et al.  Time series analysis, forecasting and control , 1972 .

[3]  Li Bin.  Chaotic Time Series Prediction Based on ELM Learning Algorithm , 2011 .

[4]  Zhigang Zeng,et al.  A modified Elman neural network with a new learning rate scheme , 2018, Neurocomputing.

[5]  Ruiqing Niu,et al.  Displacement prediction of Baijiabao landslide based on empirical mode decomposition and long short-term memory neural network in Three Gorges area, China , 2018, Comput. Geosci..

[6]  Gwilym M. Jenkins,et al.  Time series analysis, forecasting and control , 1971 .

[7]  Bipan Tudu,et al.  A recurrent Elman network in conjunction with an electronic nose for fast prediction of optimum fermentation time of black tea , 2017, Neural Computing and Applications.

[8]  Reshma Khemchandani,et al.  Regularized least squares fuzzy support vector regression for financial time series forecasting , 2009, Expert Syst. Appl..

[9]  S. N. Deepa,et al.  ELMAN Neural Network with Modified Grey Wolf Optimizer for Enhanced Wind Speed Forecasting , 2016 .

[10]  Li Yun-meng.  A Method to Improve Economic Forecasting Precision Based on SVM Model , 2004 .

[11]  Sheng-De Wang,et al.  Fuzzy support vector machines , 2002, IEEE Trans. Neural Networks.

[12]  Lars Kai Hansen,et al.  Adaptive Regularization in Neural Network Modeling , 2012, Neural Networks: Tricks of the Trade.

[13]  Chen Zhou,et al.  Adapting extreme value statistics to financial time series: dealing with bias and serial dependence , 2016, Finance Stochastics.

[14]  Shan Liu,et al.  Optimal Forecast Combination Based on Neural Networks for Time Series Forecasting , 2018, Appl. Soft Comput..

[15]  Lijuan Cao,et al.  c-ascending support vector machines for financial time series forecasting , 2003, 2003 IEEE International Conference on Computational Intelligence for Financial Engineering, 2003. Proceedings..

[16]  Christopher M. Bishop,et al.  Neural networks for pattern recognition , 1995 .

[17]  Federico Divina,et al.  Stacking Ensemble Learning for Short-Term Electricity Consumption Forecasting , 2018 .

[18]  Masoud Monjezi,et al.  Prediction of seismic slope stability through combination of particle swarm optimization and neural network , 2015, Engineering with Computers.

[19]  Yasushi Minowa.  Verification for generalizability and accuracy of a thinning-trees selection model with the ensemble learning algorithm and the cross-validation method , 2008, Journal of Forest Research.

[20]  José David Martín-Guerrero,et al.  Regularized extreme learning machine for regression problems , 2011, Neurocomputing.

[21]  Ping Jiang,et al.  Displacement prediction of landslide based on generalized regression neural networks with K-fold cross-validation , 2016, Neurocomputing.

[22]  M. Xia,et al.  Deformation and mechanism of landslide influenced by the effects of reservoir water and rainfall, Three Gorges, China , 2013, Natural Hazards.

[23]  Alaa El. Sagheer,et al.  Time series forecasting of petroleum production using deep LSTM recurrent networks , 2019, Neurocomputing.

[24]  Saba Sehrish,et al.  BAT-ANN based earthquake prediction for Pakistan region , 2017, Soft Comput..

[26]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[27]  Jeffrey L. Elman,et al.  Finding Structure in Time , 1990, Cogn. Sci..

[28]  Junfei Qiao,et al.  A Self-Organizing Fuzzy Neural Network Based on a Growing-and-Pruning Algorithm , 2010, IEEE Transactions on Fuzzy Systems.

[29]  Zhigang Zeng,et al.  Ensemble of extreme learning machine for landslide displacement prediction based on time series analysis , 2013, Neural Computing and Applications.

[30]  E. Lorenz,et al.  The essence of chaos , 1993 .

[31]  J. Sheng,et al.  Seismotectonics Considered Artificial Neural Network Earthquake Prediction in Northeast Seismic Region of China , 2015 .

[32]  E. Spencer.  A Method of Analysis of the Stability of Embankments Assuming Parallel Inter-Slice Forces , 1967 .

[33]  Wu Yiping,et al.  Landslide stability analysis based on random-fuzzy reliability: taking Liangshuijing landslide as a case , 2014, Stochastic Environmental Research and Risk Assessment.

[34]  Amaury Lendasse,et al.  Autoregressive time series prediction by means of fuzzy inference systems using nonparametric residual variance estimation , 2010, Fuzzy Sets Syst..

[35]  Yue Lu,et al.  Hybrid ARIMA-BPNN model for time series prediction of the Chinese stock market , 2017, 2017 3rd International Conference on Information Management (ICIM).

[36]  Krzysztof Siwek,et al.  Regularization of neural networks for improved load forecasting in power system , 2001, ICECS 2001. 8th IEEE International Conference on Electronics, Circuits and Systems (Cat. No.01EX483).

[37]  Ergin Erdem,et al.  ARMA based approaches for forecasting the tuple of wind speed and direction , 2011 .

[38]  Xiao Zhi Gao,et al.  Genetic Algorithm Training of Elman Neural Network in Motor Fault Detection , 2002, Neural Computing & Applications.

[39]  Jun Wang,et al.  A One-Layer Recurrent Neural Network for Constrained Nonsmooth Optimization , 2011, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[40]  Peter Stagge,et al.  Recurrent neural networks for time series classification , 2003, Neurocomputing.

[41]  Jun Wang,et al.  Forecasting energy market indices with recurrent neural networks: Case study of crude oil price fluctuations , 2016 .

[42]  E Omid Mahdi Ebadati,et al.  An efficient hybrid machine learning method for time series Stock market forecasting , 2018 .

[43]  Meng Joo Er,et al.  Dynamic fuzzy neural networks-a novel approach to function approximation , 2000, IEEE Trans. Syst. Man Cybern. Part B.