Class-specific cost regulation extreme learning machine for imbalanced classification

Abstract Due to its much faster speed and better generalization performance, extreme learning machine (ELM) has attracted much attention as an effective learning approach. However, ELM rarely involves strategies for imbalanced data distributions which may exist in many fields. Existing approaches for imbalance learning only consider the effect of the number of the class samples ignoring the dispersion degree of the data, and may lead to the suboptimal learning results. In this paper, we will propose a novel ELM, class-specific cost regulation extreme learning machine (CCR-ELM), together with its kernel based extension, for binary and multiclass classification problems with imbalanced data distributions. CCR-ELM introduces class-specific regulation cost for misclassification of each class in the performance index as the tradeoff of structural risk and empirical risk. The performance of CCR-ELM is verified using a number of benchmark datasets and the real blast furnace status diagnosis problem. Experimental results show that CCR-ELM can achieve better performance for classification problems with imbalanced data distributions than the original ELM and existing ELM imbalance learning approach, and the kernel based CCR-ELM can improve the performance further.

[1]  Chee Kheong Siew,et al.  Extreme learning machine: Theory and applications , 2006, Neurocomputing.

[2]  Cheng Wu,et al.  Semi-Supervised and Unsupervised Extreme Learning Machines , 2014, IEEE Transactions on Cybernetics.

[3]  Lei Chen,et al.  Enhanced random search based incremental extreme learning machine , 2008, Neurocomputing.

[4]  K. S. Banerjee Generalized Inverse of Matrices and Its Applications , 1973 .

[5]  Hongming Zhou,et al.  Extreme Learning Machine for Regression and Multiclass Classification , 2012, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[6]  Chee Kheong Siew,et al.  Incremental extreme learning machine with fully complex hidden nodes , 2008, Neurocomputing.

[7]  Kurt Hornik,et al.  Approximation capabilities of multilayer feedforward networks , 1991, Neural Networks.

[8]  Q. M. Jonathan Wu,et al.  Human action recognition using extreme learning machine based on visual vocabularies , 2010, Neurocomputing.

[9]  Guang-Bin Huang,et al.  Convex incremental extreme learning machine , 2007, Neurocomputing.

[10]  Chee Kheong Siew,et al.  Universal Approximation using Incremental Constructive Feedforward Networks with Random Hidden Nodes , 2006, IEEE Transactions on Neural Networks.

[11]  Xin Yao,et al.  Multiclass Imbalance Problems: Analysis and Potential Solutions , 2012, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[12]  Guang-Bin Huang,et al.  An Insight into Extreme Learning Machines: Random Neurons, Random Features and Kernels , 2014, Cognitive Computation.

[13]  Amaury Lendasse,et al.  OP-ELM: Optimally Pruned Extreme Learning Machine , 2010, IEEE Transactions on Neural Networks.

[14]  María José del Jesús,et al.  Hierarchical fuzzy rule based classification systems with genetic rule selection for imbalanced data-sets , 2009, Int. J. Approx. Reason..

[15]  Jiu-sun Zeng,et al.  Data-driven predictive control for blast furnace ironmaking process , 2010, Comput. Chem. Eng..

[16]  Wendong Xiao,et al.  Daily Human Physical Activity Recognition Based on Kernel Discriminant Analysis and Extreme Learning Machine , 2015 .

[17]  Xi Chen,et al.  An ELM Based Online Soft Sensing Approach for Alumina Concentration Detection , 2015 .

[18]  Qunxiong Zhu,et al.  A hierarchical structure of extreme learning machine (HELM) for high-dimensional datasets with noise , 2014, Neurocomputing.

[19]  Yuan Lan,et al.  Constructive hidden nodes selection of extreme learning machine for regression , 2010, Neurocomputing.

[20]  Nan Liu,et al.  Landmark recognition with sparse representation classification and extreme learning machine , 2015, J. Frankl. Inst..

[21]  Tom Fawcett,et al.  An introduction to ROC analysis , 2006, Pattern Recognit. Lett..

[22]  Kar-Ann Toh,et al.  Deterministic Neural Classification , 2008, Neural Computation.

[23]  Zexuan Zhu,et al.  A fast pruned-extreme learning machine for classification problem , 2008, Neurocomputing.

[24]  Min Han,et al.  Partial Lanczos extreme learning machine for single-output regression problems , 2009, Neurocomputing.

[25]  Divya Tomar,et al.  An effective Weighted Multi-class Least Squares Twin Support Vector Machine for Imbalanced data classification , 2015, Int. J. Comput. Intell. Syst..

[26]  Haibo He,et al.  Learning from Imbalanced Data , 2009, IEEE Transactions on Knowledge and Data Engineering.

[27]  Rajesh Jha,et al.  Multi‐Objective Genetic Algorithms and Genetic Programming Models for Minimizing Input Carbon Rates in a Blast Furnace Compared with a Conventional Analytic Approach , 2014 .

[28]  Sangmin Choi,et al.  Numerical Modeling of Reaction and Flow Characteristics in a Blast Furnace with Consideration of Layered Burden , 2010 .

[29]  Narasimhan Sundararajan,et al.  A Fast and Accurate Online Sequential Learning Algorithm for Feedforward Networks , 2006, IEEE Transactions on Neural Networks.

[30]  Q. M. Jonathan Wu,et al.  Human face recognition based on multidimensional PCA and extreme learning machine , 2011, Pattern Recognit..

[31]  Antonio J. Serrano,et al.  BELM: Bayesian Extreme Learning Machine , 2011, IEEE Transactions on Neural Networks.

[32]  Zhiping Lin,et al.  Weighted Online Sequential Extreme Learning Machine for Class Imbalance Learning , 2013, Neural Processing Letters.

[33]  Yanqing Zhang,et al.  SVMs Modeling for Highly Imbalanced Classification , 2009, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[34]  V. Lakshmikantham,et al.  Stability of conditionally invariant sets and controlleduncertain dynamic systems on time scales , 1995 .

[35]  Tao Chen,et al.  Landmark recognition with compact BoW histogram and ensemble ELM , 2015, Multimedia Tools and Applications.

[36]  Lars Schmidt-Thieme,et al.  A new evaluation measure for learning from imbalanced data , 2011, The 2011 International Joint Conference on Neural Networks.

[37]  Yong Yang,et al.  Leukocyte image segmentation by visual attention and extreme learning machine , 2011, Neural Computing and Applications.

[38]  Yiqiang Chen,et al.  Weighted extreme learning machine for imbalance learning , 2013, Neurocomputing.

[39]  Qing He,et al.  Extreme Support Vector Machine Classifier , 2008, PAKDD.

[40]  D. Serre Matrices: Theory and Applications , 2002 .