Improved extreme learning machine for multivariate time series online sequential prediction

Abstract Multivariate time series has attracted increasing attention due to its rich dynamic information of the underlying systems. This paper presents an improved extreme learning machine for online sequential prediction of multivariate time series. The multivariate time series is first phase-space reconstructed to form the input and output samples. Extreme learning machine, which has simple structure and good performance, is used as prediction model. On the basis of the specific network function of extreme learning machine, an improved Levenberg–Marquardt algorithm, in which Hessian matrix and gradient vector are calculated iteratively, is developed to implement online sequential prediction. Finally, simulation results of artificial and real-world multivariate time series are provided to substantiate the effectiveness of the proposed method.

[1]  Amaury Lendasse,et al.  OP-ELM: Optimally Pruned Extreme Learning Machine , 2010, IEEE Transactions on Neural Networks.

[2]  Mikko Kolehmainen,et al.  Evolving the neural network model for forecasting air pollution time series , 2004, Eng. Appl. Artif. Intell..

[3]  Guang-Bin Huang,et al.  An Insight into Extreme Learning Machines: Random Neurons, Random Features and Kernels , 2014, Cognitive Computation.

[4]  Kishan G. Mehrotra,et al.  Forecasting the behavior of multivariate time series using neural networks , 1992, Neural Networks.

[5]  Hao Yu,et al.  Neural Network Learning Without Backpropagation , 2010, IEEE Transactions on Neural Networks.

[6]  S. Balasundaram,et al.  On extreme learning machine for ε-insensitive regression in the primal by Newton method , 2011, Neural Computing and Applications.

[7]  Francesco Piazza,et al.  Online sequential extreme learning machine in nonstationary environments , 2013, Neurocomputing.

[8]  Yuan Lan,et al.  Ensemble of online sequential extreme learning machine , 2009, Neurocomputing.

[9]  Ravi Sankar,et al.  Time Series Prediction Using Support Vector Machines: A Survey , 2009, IEEE Computational Intelligence Magazine.

[10]  Qinghua Zheng,et al.  Regularized Extreme Learning Machine , 2009, 2009 IEEE Symposium on Computational Intelligence and Data Mining.

[11]  Minxia Luo,et al.  A hybrid approach combining extreme learning machine and sparse representation for image classification , 2014, Eng. Appl. Artif. Intell..

[12]  Hongming Zhou,et al.  Extreme Learning Machine for Regression and Multiclass Classification , 2012, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[13]  Narasimhan Sundararajan,et al.  A Fast and Accurate Online Sequential Learning Algorithm for Feedforward Networks , 2006, IEEE Transactions on Neural Networks.

[14]  Danilo Comminiello,et al.  Online Sequential Extreme Learning Machine With Kernels , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[15]  Harald Haas,et al.  Harnessing Nonlinearity: Predicting Chaotic Systems and Saving Energy in Wireless Communication , 2004, Science.

[16]  Jun Wang,et al.  Chaotic Time Series Prediction Based on a Novel Robust Echo State Network , 2012, IEEE Transactions on Neural Networks and Learning Systems.

[17]  Min Han,et al.  Support Vector Echo-State Machine for Chaotic Time-Series Prediction , 2007, IEEE Transactions on Neural Networks.

[18]  Antonio J. Serrano,et al.  BELM: Bayesian Extreme Learning Machine , 2011, IEEE Transactions on Neural Networks.

[19]  A. Mees,et al.  Dynamics from multivariate time series , 1998 .

[20]  A.H. Nizar,et al.  Power Utility Nontechnical Loss Analysis With Extreme Learning Machine Method , 2008, IEEE Transactions on Power Systems.

[21]  Noureddine Zerhouni,et al.  Recurrent radial basis function network for time-series prediction , 2003 .

[22]  Chee Kheong Siew,et al.  Extreme learning machine: Theory and applications , 2006, Neurocomputing.

[23]  Li Ying Jiang,et al.  Multivariate Time Series Prediction Based on a Simple RBF Network , 2012 .

[24]  Qinyu. Zhu.  Extreme Learning Machine , 2013 .

[25]  Min Han,et al.  Online sequential extreme learning machine with kernels for nonstationary time series prediction , 2014, Neurocomputing.

[26]  A. N. Sharkovskiĭ.  Dynamic systems and turbulence , 1989 .

[27]  Yonggwan Won,et al.  Regularized online sequential learning algorithm for single-hidden layer feedforward neural networks , 2011, Pattern Recognit. Lett..

[28]  Min Han,et al.  Multivariate chaotic time series prediction based on improved extreme learning machine , 2017, 2017 36th Chinese Control Conference (CCC).

[29]  Junyi Yu,et al.  Chaotic time series prediction: From one to another , 2009 .

[30]  Dianhui Wang,et al.  Extreme learning machines: a survey , 2011, Int. J. Mach. Learn. Cybern..

[31]  Seokjin Lee,et al.  Low complexity adaptive forgetting factor for online sequential extreme learning machine (OS-ELM) for application to nonstationary system estimations , 2012, Neural Computing and Applications.

[32]  C. J. Franco,et al.  PREDICCIÓN DE SERIES TEMPORALES USANDO MÁQUINAS DE VECTORES DE SOPORTE TIME SERIES PREDICTION USING SUPPORT VECTOR MACHINES , 2010 .

[33]  Florin Popescu,et al.  Robust Statistics for Describing Causality in Multivariate Time Series , 2009, NIPS Mini-Symposium on Causality in Time Series.

[34]  Zhihong Man,et al.  Robust Single-Hidden Layer Feedforward Network-Based Pattern Classifier , 2012, IEEE Transactions on Neural Networks and Learning Systems.

[35]  Zhigang Zeng,et al.  Ensemble of extreme learning machine for landslide displacement prediction based on time series analysis , 2013, Neural Computing and Applications.

[36]  Dong Sun Park,et al.  Online sequential extreme learning machine with forgetting mechanism , 2012, Neurocomputing.

[37]  Chee Kheong Siew,et al.  Universal Approximation using Incremental Constructive Feedforward Networks with Random Hidden Nodes , 2006, IEEE Transactions on Neural Networks.

[38]  F. Takens.  Detecting strange attractors in turbulence , 1981 .

[39]  Meiling Xu,et al.  Multivariate chaotic time series prediction based on PLSR and MKELM , 2015, 2015 Sixth International Conference on Intelligent Control and Information Processing (ICICIP).

[40]  Tieshan Li,et al.  Robust fuzzy inference system for prediction of time series with outliers , 2012, 2012 International conference on Fuzzy Theory and Its Applications (iFUZZY2012).

[41]  Jiuyong Li,et al.  Using causal discovery for feature selection in multivariate numerical time series , 2015, Machine Learning.

[42]  Alberto Gómez,et al.  Forecasting next-day price of electricity in the Spanish energy market using artificial neural networks , 2008, Eng. Appl. Artif. Intell..

[43]  Narasimhan Sundararajan,et al.  Online Sequential Fuzzy Extreme Learning Machine for Function Approximation and Classification Problems , 2009, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[44]  Hao Yu,et al.  Improved Computation for Levenberg–Marquardt Training , 2010, IEEE Transactions on Neural Networks.

[45]  K. Müller,et al.  Finding stationary subspaces in multivariate time series. , 2009, Physical review letters.

[46]  Min Han,et al.  Multivariate time series prediction based on multiple kernel extreme learning machine , 2014, 2014 International Joint Conference on Neural Networks (IJCNN).

[47]  Rob J Hyndman,et al.  25 years of time series forecasting , 2006 .