Robust adaptive online sequential extreme learning machine for predicting nonstationary data streams with outliers

Data streams online modeling and prediction is an important research direction in the field of data mining. In practical applications, data streams are often of nonstationary nature and containing outliers, hence an online learning algorithm with dynamic tracking capability as well as anti-outlier capability is urgently needed. With this in mind, this paper proposes a novel robust adaptive online sequential extreme learning machine (RA-OSELM) algorithm for the online modeling and prediction of nonstationary data streams with outliers. The RA-OSELM is developed from the famous online sequential extreme learning machine algorithm, but it uses a more robust M-estimation loss function to replace the conventional least square loss function so as to suppress the incorrect online update of the learning algorithm with respect to outliers, and hence enhances its robustness in the presence of outliers. Moreover, the RA-OSELM adopts a variable forgetting factor method to automatically track the dynamic changes of the nonstationary data streams and timely eliminate the negative impacts of the outdated data, so it tends to produce satisfying tracking results in nonstationary environments. The performances of RA-OSELM are evaluated and compared with other representative algorithms with synthetic and real data sets, and the experimental results indicate that the proposed algorithm has better adaptive tracking capability with stronger robustness than its counterparts for predicting nonstationary data streams with outliers.

[1]  Badong Chen,et al.  Sequential extreme learning machine incorporating survival error potential , 2015, Neurocomputing.

[2]  He Jiang,et al.  FP-ELM: An online sequential learning algorithm for dealing with concept drift , 2016, Neurocomputing.

[3]  Yong Dou,et al.  Robust regularized extreme learning machine for regression using iteratively reweighted least squares , 2017, Neurocomputing.

[4]  Adriano Lorena Inácio de Oliveira,et al.  A sequential learning method with Kalman filter and extreme learning machine for regression and time series forecasting , 2019, Neurocomputing.

[5]  Plamen P. Angelov,et al.  Handling drifts and shifts in on-line data streams with evolving fuzzy systems , 2011, Appl. Soft Comput..

[6]  Xiangang Peng,et al.  A novel wind speed forecasting based on hybrid decomposition and online sequential outlier robust extreme learning machine , 2019, Energy Conversion and Management.

[7]  Gang Wang,et al.  AN ADAPTIVE ONLINE SEQUENTIAL EXTREME LEARNING MACHINE FOR SHORT-TERM WIND SPEED PREDICTION BASED ON IMPROVED ARTIFICIAL BEE COLONY ALGORITHM , 2018 .

[8]  Bing Li,et al.  Initial Alignment Error On-Line Identification Based on Adaptive Particle Swarm Optimization Algorithm , 2018, Mathematical Problems in Engineering.

[9]  Tao Xu,et al.  M-estimator-based online sequential extreme learning machine for predicting chaotic time series with outliers , 2017, Neural Computing and Applications.

[10]  Minxia Luo,et al.  Outlier-robust extreme learning machine for regression problems , 2015, Neurocomputing.

[11]  Dong Sun Park,et al.  Online sequential extreme learning machine with forgetting mechanism , 2012, Neurocomputing.

[12]  Marco Danelutto,et al.  Elastic-PPQ: A two-level autonomic system for spatial preference query processing over dynamic data streams , 2018, Future Gener. Comput. Syst..

[13]  Ying Tan,et al.  Recentness biased learning for time series forecasting , 2013, Inf. Sci..

[14]  Herna L. Viktor,et al.  Dynamic adaptation of online ensembles for drifting data streams , 2017, Journal of Intelligent Information Systems.

[15]  Peter J. Rousseeuw,et al.  Robust Regression and Outlier Detection , 2005, Wiley Series in Probability and Statistics.

[16]  Haigang Zhang,et al.  Online sequential ELM algorithm with forgetting factor for real applications , 2017, Neurocomputing.

[17]  Wei Guo,et al.  Online Sequential Extreme Learning Machine with Generalized Regularization and Adaptive Forgetting Factor for Time-Varying System Prediction , 2018 .

[18]  Qingshan Liu,et al.  Dynamic Structure Embedded Online Multiple-Output Regression for Streaming Data , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[19]  Seokjin Lee,et al.  Low complexity adaptive forgetting factor for online sequential extreme learning machine (OS-ELM) for application to nonstationary system estimations , 2012, Neural Computing and Applications.

[20]  Aranildo R. Lima,et al.  Variable complexity online sequential extreme learning machine, with applications to streamflow prediction , 2017 .

[21]  Yonggwan Won,et al.  Regularized online sequential learning algorithm for single-hidden layer feedforward neural networks , 2011, Pattern Recognit. Lett..

[22]  A. Bouchachia,et al.  A Bi-Criteria Active Learning Algorithm for Dynamic Data Streams , 2016 .

[23]  Ravinesh C. Deo,et al.  Multi-stage hybridized online sequential extreme learning machine integrated with Markov Chain Monte Carlo copula-Bat algorithm for rainfall forecasting , 2018, Atmospheric Research.

[24]  Feng Lu,et al.  Time Series Prediction Based on Adaptive Weight Online Sequential Extreme Learning Machine , 2017 .

[25]  Jian-Chuan Yin,et al.  A variable-structure online sequential extreme learning machine for time-varying system prediction , 2017, Neurocomputing.

[26]  Claus Weihs,et al.  A new dynamic weighted majority control chart for data streams , 2016, Soft Computing.

[27]  Narasimhan Sundararajan,et al.  A Fast and Accurate Online Sequential Learning Algorithm for Feedforward Networks , 2006, IEEE Transactions on Neural Networks.

[28]  Chee Kheong Siew,et al.  Extreme learning machine: Theory and applications , 2006, Neurocomputing.

[29]  Benoît Frénay,et al.  Reinforced Extreme Learning Machines for Fast Robust Regression in the Presence of Outliers , 2016, IEEE Transactions on Cybernetics.

[30]  Guilherme De A. Barreto,et al.  A Robust Extreme Learning Machine for pattern classification with outliers , 2016, Neurocomputing.

[31]  Oscar Fontenla-Romero,et al.  An online learning algorithm for adaptable topologies of neural networks , 2013, Expert Syst. Appl..

[32]  Jie Zhang,et al.  A Novel Online Sequential Extreme Learning Machine for Gas Utilization Ratio Prediction in Blast Furnaces , 2017, Sensors.

[33]  Feng Lu,et al.  Sensor Fault Diagnosis for Aero Engine Based on Online Sequential Extreme Learning Machine with Memory Principle , 2017 .

[34]  Wang Yongsheng,et al.  Prediction of the chaotic time series from parameter-varying systems using artificial neural networks , 2008 .

[35]  Rui Araújo,et al.  An adaptive ensemble of on-line Extreme Learning Machines with variable forgetting factor for dynamic system prediction , 2016, Neurocomputing.

[36]  S. Haykin,et al.  Adaptive Filter Theory , 1986 .