Class-specific kernelized extreme learning machine for binary class imbalance learning

Abstract Class imbalance problem occurs when the training dataset contains significantly fewer samples of one class (minority-class) compared to another class (majority-class). Conventional extreme learning machine (ELM) gives equal importance to all the samples leading to the results which favor the majority-class. Numerous variants of ELM-like weighted ELM (WELM), class-specific cost regulation ELM (CCR-ELM), class-specific ELM (CS-ELM) etc. have been proposed in order to diminish the performance degradation which happens due to the class imbalance problem. ELM with Gaussian kernel outperforms the ELM with Sigmoid node. This work proposed a novel class-specific kernelized ELM (CSKELM) which is a variant of kernelized ELM to address the class imbalance problem more effectively. CSKELM with Gaussian kernel function avoids the non-optimal hidden node problem associated with CS-ELM and the other existing variants of ELM. This work is distinct from WELM because it does not require the assignment of weights to the training samples. In addition, the proposed work also has considerably lower computational cost in contrast with kernelized WELM. This work employs class-specific regularization in the same way as CS-ELM. This work differs from CS-ELM as the proposed CSKELM uses the Gaussian kernel function to map the input data to the feature space. The proposed work also has lower computational overhead in contrast with kernelized CCR-ELM. The proposed work is assessed by employing benchmark real-world imbalanced datasets downloaded from the KEEL dataset repository. The experimental results indicate the superiority of the proposed work in contrast with the rest of classifiers for the imbalanced classification problems.

[1]  Yiqiang Chen,et al.  Weighted extreme learning machine for imbalance learning , 2013, Neurocomputing.

[2]  Stephen Kwek,et al.  Applying Support Vector Machines to Imbalanced Datasets , 2004, ECML.

[3]  Andrew P. Bradley,et al.  The use of the area under the ROC curve in the evaluation of machine learning algorithms , 1997, Pattern Recognit..

[4]  Jesús Alcalá-Fdez,et al.  KEEL Data-Mining Software Tool: Data Set Repository, Integration of Algorithms and Experimental Analysis Framework , 2011, J. Multiple Valued Log. Soft Comput..

[5]  Dayong Shen,et al.  Traffic Sign Recognition Using Kernel Extreme Learning Machines With Deep Perceptual Features , 2017, IEEE Transactions on Intelligent Transportation Systems.

[6]  Haibo He,et al.  Learning from Imbalanced Data , 2009, IEEE Transactions on Knowledge and Data Engineering.

[7]  Songul Albayrak,et al.  Alleviating class imbalance problem in data mining , 2013, 2013 21st Signal Processing and Communications Applications Conference (SIU).

[8]  Taghi M. Khoshgoftaar,et al.  RUSBoost: A Hybrid Approach to Alleviating Class Imbalance , 2010, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[9]  Weidong Yang,et al.  Class-specific cost regulation extreme learning machine for imbalanced classification , 2017, Neurocomputing.

[10]  Chee Kheong Siew,et al.  Extreme learning machine: Theory and applications , 2006, Neurocomputing.

[11]  Sanyam Shukla,et al.  Class-specific extreme learning machine for handling binary class imbalance problem , 2018, Neural Networks.

[12]  Xiaoyi Jiang,et al.  Dynamic classifier ensemble model for customer classification with imbalanced class distribution , 2012, Expert Syst. Appl..

[13]  Mikel Galar,et al.  Evolutionary undersampling boosting for imbalanced classification of breast cancer malignancy , 2016, Appl. Soft Comput..

[14]  Yijing Li,et al.  Learning from class-imbalanced data: Review of methods and applications , 2017, Expert Syst. Appl..

[15]  Yong Zhang,et al.  Ensemble weighted extreme learning machine for imbalanced data classification based on differential evolution , 2017, Neural Computing and Applications.

[16]  Charles X. Ling,et al.  Using AUC and accuracy in evaluating learning algorithms , 2005, IEEE Transactions on Knowledge and Data Engineering.

[17]  Nello Cristianini,et al.  Controlling the Sensitivity of Support Vector Machines , 1999 .

[18]  Christophe Mues,et al.  An experimental comparison of classification algorithms for imbalanced credit scoring data sets , 2012, Expert Syst. Appl..

[19]  Alexandros Iosifidis,et al.  On the kernel Extreme Learning Machine speedup , 2015, Pattern Recognit. Lett..

[20]  Sanyam Shukla,et al.  Class-specific cost-sensitive boosting weighted ELM for class imbalance learning , 2018, Memetic Computing.

[21]  Qinghua Zheng,et al.  Regularized Extreme Learning Machine , 2009, 2009 IEEE Symposium on Computational Intelligence and Data Mining.

[22]  Kar-Ann Toh,et al.  Deterministic Neural Classification , 2008, Neural Computation.

[23]  Haydemar Núñez,et al.  Handling binary classification problems with a priority class by using Support Vector Machines , 2017, Appl. Soft Comput..

[24]  Joarder Kamruzzaman,et al.  z-SVM: An SVM for Improved Classification of Imbalanced Data , 2006, Australian Conference on Artificial Intelligence.

[25]  Sanyam Shukla,et al.  UnderBagging based reduced Kernelized weighted extreme learning machine for class imbalance learning , 2018, Eng. Appl. Artif. Intell..

[26]  Chee Khiang Pang,et al.  Classification of Imbalanced Data by Oversampling in Kernel Space of Support Vector Machines , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[27]  XuanLong Nguyen,et al.  Identification of the Dynamic Operating Envelope of HCCI Engines Using Class Imbalance Learning , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[28]  Hong-Liang Dai,et al.  Class imbalance learning via a fuzzy total margin based support vector machine , 2015, Appl. Soft Comput..

[29]  Jianping Yin,et al.  Boosting weighted ELM for imbalanced learning , 2014, Neurocomputing.

[30]  Marek Lubicz,et al.  Boosted SVM for extracting rules from imbalanced data in application to prediction of the post-operative life expectancy in the lung cancer patients , 2014, Appl. Soft Comput..

[31]  Janez Demsar,et al.  Statistical Comparisons of Classifiers over Multiple Data Sets , 2006, J. Mach. Learn. Res..

[32]  Alexandros Iosifidis,et al.  On the kernel Extreme Learning Machine classifier , 2015, Pattern Recognit. Lett..

[33]  Ram Narayan Yadav,et al.  Regularized Weighted Circular Complex-Valued Extreme Learning Machine for Imbalanced Learning , 2015, IEEE Access.

[34]  XuanLong Nguyen,et al.  Stochastic gradient based extreme learning machines for stable online learning of advanced combustion engines , 2016, Neurocomputing.

[35]  Hongming Zhou,et al.  Extreme Learning Machine for Regression and Multiclass Classification , 2012, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[36]  Yanqing Zhang,et al.  SVMs Modeling for Highly Imbalanced Classification , 2009, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).