Robust extreme learning machine in the presence of outliers by iterative reweighted algorithm

Extreme learning machine (ELM) is widely used to derive the single-hidden layer feedforward neural networks. However, ELM faces a great challenge in the presence of outliers, which can result in the sensitivity and poor robustness. To overcome this dilemma, a non-convex 2-norm loss function is developed to reduce these negative influences by setting a fixed penalty on any potential outliers. A novel robust ELM is proposed in this paper, and the resultant optimization can be implemented by an iterative reweighted algorithm, called IRRELM. In each iteration, IRRELM solves a weighted ELM. Several artificial datasets, real-world datasets and financial time series datasets are employed in numerical experiments, which demonstrate that IRRELM has superior generalization performance and robustness for modeling datasets in the presence of outliers, especially at the higher outlier levels.

[1]  Ping Zhong,et al.  Training robust support vector regression with smooth non-convex loss function , 2012, Optim. Methods Softw..

[2]  Amaury Lendasse,et al.  Long-term prediction of time series using NNE-based projection and OP-ELM , 2008, 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence).

[3]  Min Han,et al.  Online sequential extreme learning machine with kernels for nonstationary time series prediction , 2014, Neurocomputing.

[4]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[5]  Qinghua Zheng,et al.  Regularized Extreme Learning Machine , 2009, 2009 IEEE Symposium on Computational Intelligence and Data Mining.

[6]  Q. M. Jonathan Wu,et al.  Human face recognition based on multidimensional PCA and extreme learning machine , 2011, Pattern Recognit..

[7]  M. Verleysen,et al.  Classification in the Presence of Label Noise: A Survey , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[8]  Werner A. Stahel,et al.  Robust Statistics: The Approach Based on Influence Functions , 1987 .

[9]  Ping Zhong,et al.  Robust non-convex least squares loss function for regression with outliers , 2014, Knowl. Based Syst..

[10]  Ping Zhong,et al.  Robust Support Vector Regression with Generalized Loss Function and Applications , 2015, Neural Processing Letters.

[11]  Yong Dou,et al.  Robust regularized extreme learning machine for regression using iteratively reweighted least squares , 2017, Neurocomputing.

[12]  Zhuo Ren,et al.  Robust Extreme Learning Machines with Different Loss Functions , 2018, Neural Processing Letters.

[13]  Minxia Luo,et al.  Outlier-robust extreme learning machine for regression problems , 2015, Neurocomputing.

[14]  R. Venkatesh Babu,et al.  No-reference image quality assessment using modified extreme learning machine classifier , 2009, Appl. Soft Comput..

[15]  Johan A. K. Suykens,et al.  Least Squares Support Vector Machine Classifiers , 1999, Neural Processing Letters.

[16]  Janez Demsar,et al.  Statistical Comparisons of Classifiers over Multiple Data Sets , 2006, J. Mach. Learn. Res..

[17]  S. García,et al.  An Extension on "Statistical Comparisons of Classifiers over Multiple Data Sets" for all Pairwise Comparisons , 2008 .

[18]  Chee Kheong Siew,et al.  Universal Approximation using Incremental Constructive Feedforward Networks with Random Hidden Nodes , 2006, IEEE Transactions on Neural Networks.

[19]  Hong-Jie Xing,et al.  Training extreme learning machine via regularized correntropy criterion , 2012, Neural Computing and Applications.

[20]  Xia Liu,et al.  The universal consistency of extreme learning machine , 2018, Neurocomputing.

[21]  Punyaphol Horata,et al.  Robust extreme learning machine , 2013, Neurocomputing.

[22]  Lai-Wan Chan,et al.  Support Vector Machine Regression for Volatile Stock Market Prediction , 2002, IDEAL.

[23]  Chee Kheong Siew,et al.  Extreme learning machine: Theory and applications , 2006, Neurocomputing.

[24]  Victoria J. Hodge,et al.  A Survey of Outlier Detection Methodologies , 2004, Artificial Intelligence Review.

[25]  Zongben Xu,et al.  Universal Approximation of Extreme Learning Machine With Adaptive Growth of Hidden Nodes , 2012, IEEE Transactions on Neural Networks and Learning Systems.

[26]  Hongming Zhou,et al.  Extreme Learning Machine for Regression and Multiclass Classification , 2012, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[27]  Johan A. K. Suykens,et al.  Robustness of Kernel Based Regression: A Comparison of Iterative Weighting Schemes , 2009, ICANN.

[28]  Glenn Fung,et al.  Proximal support vector machine classifiers , 2001, KDD '01.