Building a Robust Extreme Learning Machine for Classification in the Presence of Outliers

The Extreme Learning Machine (ELM), recently proposed by Huang et al. [6], is a single-hidden-layered neural network architecture which has been successfully applied to nonlinear regression and classification tasks [5]. A crucial step in the design of the ELM is the computation of the output weight matrix, a step usually performed by means of the ordinary least-squares (OLS) method - a.k.a. Moore-Penrose generalized inverse technique. However, it is well-known that the OLS method produces predictive models which are highly sensitive to outliers in the data. In this paper, we develop an extension of ELM which is robust to outliers caused by labelling errors. To deal with this problem, we suggest the use of M-estimators, a parameter estimation framework widely used in robust regression, to compute the output weight matrix, instead of using the standard OLS solution. The proposed model is robust to label noise not only near the class boundaries, but also far from the class boundaries which can result from mistakes in labelling or gross errors in measuring the input features. We show the usefulness of the proposed classification approach through simulation results using synthetic and real-world data.