A Dynamic ELM with Balanced Variance and Bias for Long-Term Online Prediction

For long-term online prediction of nonlinear time series, how to determine a feasible network architecture that conforms to the time-varying data stream is recognized to be a challenging problem. To deal with the issue, a dynamic ELM with balanced variance and bias has been proposed. A suitable fitting degree, which contains model applicability at sequential learning phase, is taken into consideration. Based on the shifting error of the sequence fragment, the automatic model update strategy is exploited. Transformable parameters help reduce the overfitting and underfitting at the same time, and avoid the trial and error caused by user intervention effectively, so as to guarantee the feasibility for long-term online prediction. Furthermore, hidden node number and the regularization parameter can be calculated according to the fast-changing test data, thus building an optimum network architecture quantificationally. Experimental results verify that the proposed algorithm has better generalization performance on various long-term regression problems.

[1]  Torsten Hothorn,et al.  Bagging Tree Classifiers for Laser Scanning Images: Data and Simulation Based Strategy , 2002, Artif. Intell. Medicine.

[2]  Robert K. L. Gay,et al.  Error Minimized Extreme Learning Machine With Growth of Hidden Nodes and Incremental Learning , 2009, IEEE Transactions on Neural Networks.

[3]  Naomi S. Altman,et al.  Points of Significance: Model selection and overfitting , 2016, Nature Methods.

[4]  Zhiping Lin,et al.  Self-Adaptive Evolutionary Extreme Learning Machine , 2012, Neural Processing Letters.

[5]  Danwei Wang,et al.  Sparse Extreme Learning Machine for Classification , 2014, IEEE Transactions on Cybernetics.

[6]  Min Han,et al.  Multivariate Chaotic Time Series Prediction Based on ELM–PLSR and Hybrid Variable Selection Algorithm , 2017, Neural Processing Letters.

[7]  He Jiang,et al.  FP-ELM: An online sequential learning algorithm for dealing with concept drift , 2016, Neurocomputing.

[8]  Yuan F. Zheng,et al.  Sequential Sample Consensus: A Robust Algorithm for Video-Based Face Recognition , 2015, IEEE Transactions on Circuits and Systems for Video Technology.

[9]  César Hervás-Martínez,et al.  PCA-ELM: A Robust and Pruned Extreme Learning Machine Approach Based on Principal Component Analysis , 2012, Neural Processing Letters.

[10]  Sundaram Suresh,et al.  A Meta-Cognitive Learning Algorithm for an Extreme Learning Machine Classifier , 2013, Cognitive Computation.

[11]  Riccardo Poli,et al.  Particle swarm optimization , 1995, Swarm Intelligence.

[12]  Prashant Kumar,et al.  Indoor air quality and energy management through real-time sensing in commercial buildings , 2016 .

[13]  Nicolas Garcia Trillos,et al.  A new analytical approach to consistency and overfitting in regularized empirical risk minimization , 2016, European Journal of Applied Mathematics.

[14]  Zongben Xu,et al.  Universal Approximation of Extreme Learning Machine With Adaptive Growth of Hidden Nodes , 2012, IEEE Transactions on Neural Networks and Learning Systems.

[15]  Fei Han,et al.  An improved incremental constructive single-hidden-layer feedforward networks for extreme learning machine based on particle swarm optimization , 2017, Neurocomputing.

[16]  Teresa Bernarda Ludermir,et al.  Investigating the use of alternative topologies on performance of the PSO-ELM , 2014, Neurocomputing.

[17]  Gavin C. Cawley,et al.  On Over-fitting in Model Selection and Subsequent Selection Bias in Performance Evaluation , 2010, J. Mach. Learn. Res..

[18]  Amaury Lendasse,et al.  Long-term time series prediction using OP-ELM , 2014, Neural Networks.

[19]  Zhiping Lin,et al.  Extreme Learning Machines on High Dimensional and Large Data Applications: A Survey , 2015 .

[20]  Guang-Bin Huang,et al.  Trends in extreme learning machines: A review , 2015, Neural Networks.

[21]  Zhi-Zhong Mao,et al.  An Ensemble ELM Based on Modified AdaBoost.RT Algorithm for Predicting the Temperature of Molten Steel in Ladle Furnace , 2010, IEEE Transactions on Automation Science and Engineering.

[22]  Chee Kheong Siew,et al.  Universal Approximation using Incremental Constructive Feedforward Networks with Random Hidden Nodes , 2006, IEEE Transactions on Neural Networks.

[23]  Amir F. Atiya,et al.  A Bias and Variance Analysis for Multistep-Ahead Time Series Forecasting , 2016, IEEE Transactions on Neural Networks and Learning Systems.

[24]  Philip A. Stephens,et al.  Model selection and model averaging in behavioural ecology: the utility of the IT-AIC framework , 2010, Behavioral Ecology and Sociobiology.

[25]  Xizhao Wang,et al.  Architecture Selection of ELM Networks Based on Sensitivity of Hidden Nodes , 2015, Neural Processing Letters.

[26]  Pierre Pinson,et al.  Very Short-Term Nonparametric Probabilistic Forecasting of Renewable Energy Generation— With Application to Solar Energy , 2016, IEEE Transactions on Power Systems.

[27]  Meng Joo Er,et al.  An Efficient Leave-One-Out Cross-Validation-Based Extreme Learning Machine (ELOO-ELM) With Minimal User Intervention , 2016, IEEE Transactions on Cybernetics.

[28]  Narasimhan Sundararajan,et al.  A Fast and Accurate Online Sequential Learning Algorithm for Feedforward Networks , 2006, IEEE Transactions on Neural Networks.

[29]  Huan Wang,et al.  A Short-term Traffic Flow Forecasting Method Based on the Hybrid PSO-SVR , 2015, Neural Processing Letters.

[30]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[31]  Zongben Xu,et al.  Dynamic Extreme Learning Machine and Its Approximation Capability , 2013, IEEE Transactions on Cybernetics.

[32]  Hsien-Chung Wu,et al.  The Karush-Kuhn-Tucker optimality conditions in an optimization problem with interval-valued objective function , 2007, Eur. J. Oper. Res..

[33]  Guang-Bin Huang,et al.  Convex incremental extreme learning machine , 2007, Neurocomputing.