Parallel one-class extreme learning machine for imbalance learning based on Bayesian approach

Extreme learning machine (ELM) is a competitive machine learning approach for training single hidden layer feedforward neural networks with fast learning speed and good generalization performance. However, ELM also suffers from the imbalanced classification problem, which exists in many fields and degrades the performance of classifier significantly. In order to address this issue, in this paper, we propose a novel parallel one-class ELM based on Bayesian approach for imbalanced classification problem, named P-ELM. In P-ELM, the training dataset is firstly divided into k (k is the number of classes) components according to the class attribution of the samples, and then the divided training datasets will be fed into the corresponding k Kernel-based one-class ELM classifiers. Estimation of the probability density based on the output function of Kernel-based one-class ELM classifier is constructed. Thus, the class attribution of the samples can be judged directly by comparing the values of the output function of each Kernel-based one-class ELM classifier of P-ELM based on the Bayesian approach. The detailed performance comparison and discussions of P-ELM are performed with several related class imbalance learning approaches on some selected benchmark datasets including both the binary classification and the multiclass classification problems, as well as a real world application, i.e., the blast furnace status diagnosis problem. The experimental results show that the proposed P-ELM can achieve better performance.

[1]  Jun Miao,et al.  One-Class Classification with Extreme Learning Machine , 2015 .

[2]  Klaus-Robert Müller,et al.  A consistency-based model selection for one-class classification , 2004, ICPR 2004.

[3]  Adam Kowalczyk,et al.  Extreme re-balancing for SVMs: a case study , 2004, SKDD.

[4]  Chee Kheong Siew,et al.  Extreme learning machine: Theory and applications , 2006, Neurocomputing.

[5]  Hongming Zhou,et al.  Representational Learning with ELMs for Big Data , 2013 .

[6]  Guang-Bin Huang,et al.  Trends in extreme learning machines: A review , 2015, Neural Networks.

[7]  Robert P. W. Duin,et al.  Combining One-Class Classifiers , 2001, Multiple Classifier Systems.

[8]  Francisco Herrera,et al.  Analysis of preprocessing vs. cost-sensitive learning for imbalanced classification. Open problems on intrinsic data characteristics , 2012, Expert Syst. Appl..

[9]  Chi-Man Vong,et al.  Sparse Bayesian extreme learning committee machine for engine simultaneous fault diagnosis , 2016, Neurocomputing.

[10]  Bernhard Schölkopf,et al.  Estimating the Support of a High-Dimensional Distribution , 2001, Neural Computation.

[11]  Pedro M. Domingos,et al.  On the Optimality of the Simple Bayesian Classifier under Zero-One Loss , 1997, Machine Learning.

[12]  Jianping Yin,et al.  Boosting weighted ELM for imbalanced learning , 2014, Neurocomputing.

[13]  Changyin Sun,et al.  ODOC-ELM: Optimal decision outputs compensation-based extreme learning machine for classifying imbalanced data , 2016, Knowl. Based Syst..

[14]  Haibo He,et al.  Learning from Imbalanced Data , 2009, IEEE Transactions on Knowledge and Data Engineering.

[15]  Aruna Tiwari,et al.  Construction of multi-class classifiers by Extreme Learning Machine based one-class classifiers , 2016, 2016 International Joint Conference on Neural Networks (IJCNN).

[16]  Yixin Yin,et al.  Quantitative association rule mining for blast furnace production data , 2017, 2017 Chinese Automation Congress (CAC).

[17]  Youcef Chibani,et al.  Multiple One-Class Classifier Combination for Multi-class Classification , 2014, 2014 22nd International Conference on Pattern Recognition.

[18]  Narasimhan Sundararajan,et al.  A Fast and Accurate Online Sequential Learning Algorithm for Feedforward Networks , 2006, IEEE Transactions on Neural Networks.

[19]  Khan Muhammad,et al.  Ductal carcinoma in situ detection in breast thermography by extreme learning machine and combination of statistical measure and fractal dimension , 2017 .

[20]  Hongming Zhou,et al.  Extreme Learning Machine for Regression and Multiclass Classification , 2012, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[21]  Chee Kheong Siew,et al.  Universal Approximation using Incremental Constructive Feedforward Networks with Random Hidden Nodes , 2006, IEEE Transactions on Neural Networks.

[22]  Francisco Herrera,et al.  A Review on Ensembles for the Class Imbalance Problem: Bagging-, Boosting-, and Hybrid-Based Approaches , 2012, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[23]  Chee Kheong Siew,et al.  Incremental extreme learning machine with fully complex hidden nodes , 2008, Neurocomputing.

[24]  W. Zhang,et al.  Fuzzy extreme learning machine for classification , 2013 .

[25]  Nan Liu,et al.  Ensemble of subset online sequential extreme learning machine for class imbalance and concept drift , 2015, Neurocomputing.

[26]  María José del Jesús,et al.  Hierarchical fuzzy rule based classification systems with genetic rule selection for imbalanced data-sets , 2009, Int. J. Approx. Reason..

[27]  Nan Liu,et al.  Landmark recognition with sparse representation classification and extreme learning machine , 2015, J. Frankl. Inst..

[28]  Xin Yao,et al.  Binarization With Boosting and Oversampling for Multiclass Classification , 2016, IEEE Transactions on Cybernetics.

[29]  Dipankar Das,et al.  Enhanced SenticNet with Affective Labels for Concept-Based Opinion Mining , 2013, IEEE Intelligent Systems.

[30]  Jie Zhang,et al.  Device-Free Localization via an Extreme Learning Machine with Parameterized Geometrical Feature Extraction , 2017, Sensors.

[31]  Guang-Bin Huang,et al.  Convex Incremental Extreme Learning Machine , 2007 .

[32]  Guang-Bin Huang,et al.  An Insight into Extreme Learning Machines: Random Neurons, Random Features and Kernels , 2014, Cognitive Computation.

[33]  Zongben Xu,et al.  Dynamic Extreme Learning Machine and Its Approximation Capability , 2013, IEEE Transactions on Cybernetics.

[34]  Jun-Fei Qiao,et al.  A structure optimisation algorithm for feedforward neural network construction , 2013, Neurocomputing.

[35]  Jiuwen Cao,et al.  Kernel-Based Multilayer Extreme Learning Machines for Representation Learning , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[36]  Aruna Tiwari,et al.  On the construction of extreme learning machine for online and offline one-class classification - An expanded toolbox , 2017, Neurocomputing.

[37]  Guang-Bin Huang,et al.  Extreme learning machine: a new learning scheme of feedforward neural networks , 2004, 2004 IEEE International Joint Conference on Neural Networks (IEEE Cat. No.04CH37541).

[38]  Chi-Man Vong,et al.  Post-boosting of classification boundary for imbalanced data using geometric mean , 2017, Neural Networks.

[39]  Weidong Yang,et al.  Class-specific cost regulation extreme learning machine for imbalanced classification , 2017, Neurocomputing.

[40]  Chunru Dong,et al.  An experimental study on stability and generalization of extreme learning machines , 2015, Int. J. Mach. Learn. Cybern..

[41]  Zexuan Zhu,et al.  A fast pruned-extreme learning machine for classification problem , 2008, Neurocomputing.

[42]  Yimin Yang,et al.  Multilayer Extreme Learning Machine With Subnetwork Nodes for Representation Learning , 2016, IEEE Transactions on Cybernetics.

[43]  Yiqiang Chen,et al.  Weighted extreme learning machine for imbalance learning , 2013, Neurocomputing.

[44]  Xin Yao,et al.  Online Ensemble Learning of Data Streams with Gradually Evolved Classes , 2016, IEEE Transactions on Knowledge and Data Engineering.

[45]  Ling Zhuang,et al.  Parameter Optimization of Kernel-based One-class Classifier on Imbalance Learning , 2006, J. Comput..

[46]  Zhiping Lin,et al.  Kernel based online learning for imbalance multiclass classification , 2018, Neurocomputing.

[47]  Nathalie Japkowicz,et al.  The class imbalance problem: A systematic study , 2002, Intell. Data Anal..

[48]  Tao Chen,et al.  Landmark recognition with compact BoW histogram and ensemble ELM , 2014, Multimedia Tools and Applications.

[49]  Jie Zhang,et al.  Residual compensation extreme learning machine for regression , 2018, Neurocomputing.

[50]  Cheng Wu,et al.  Semi-Supervised and Unsupervised Extreme Learning Machines , 2014, IEEE Transactions on Cybernetics.

[51]  Lei Chen,et al.  Enhanced random search based incremental extreme learning machine , 2008, Neurocomputing.

[52]  Jesus A. Gonzalez,et al.  Symbolic One-Class Learning from Imbalanced Datasets: Application in Medical Diagnosis , 2009, Int. J. Artif. Intell. Tools.

[53]  Siti Mariyam Shamsuddin,et al.  Classification with class imbalance problem: A review , 2015, SOCO 2015.

[54]  Kar-Ann Toh,et al.  Deterministic Neural Classification , 2008, Neural Computation.

[55]  Jie Zhang,et al.  A Novel Online Sequential Extreme Learning Machine for Gas Utilization Ratio Prediction in Blast Furnaces , 2017, Sensors.

[56]  Chi-Man Vong,et al.  Efficient extreme learning machine via very sparse random projection , 2018, Soft Comput..

[57]  Amaury Lendasse,et al.  OP-ELM: Optimally Pruned Extreme Learning Machine , 2010, IEEE Transactions on Neural Networks.

[58]  Xin Yao,et al.  Online Class Imbalance Learning and its Applications in Fault Detection , 2013, Int. J. Comput. Intell. Appl..

[59]  Chi-Man Vong,et al.  Predicting minority class for suspended particulate matters level by extreme learning machine , 2014, Neurocomputing.

[60]  Yunlu Li,et al.  Optional SVM for Fault Diagnosis of Blast Furnace with Imbalanced Data , 2011 .

[61]  Francisco Herrera,et al.  An insight into classification with imbalanced data: Empirical results and current trends on using data intrinsic characteristics , 2013, Inf. Sci..