Extreme learning machine with hybrid cost function of G-mean and probability for imbalance learning

Extreme learning machine(ELM) is a simple and fast machine learning algorithm. However, similar to other conventional learning algorithms, the classical ELM can not well process the problem of imbalanced data distribution. In this paper, in order to improve the learning performance of classical ELM for imbalanced data learning, we present a novel variant of the ELM algorithm based on a hybrid cost function which employs the probability that given training sample belong in each class to calculate the G-mean. We perform comparable experiments for our approach and the state-of-the-arts methods on standard classification datasets which consist of 58 binary datasets and 9 multiclass datasets under different degrees of imbalance ratio. Experimental results show that our proposed algorithm can improve the classification performance significantly compared with other state-of-the-art methods.

[1]  Xizhao Wang,et al.  A review on neural networks with random weights , 2018, Neurocomputing.

[2]  Francisco Herrera,et al.  Cost-sensitive linguistic fuzzy rule based classification systems under the MapReduce framework for imbalanced big data , 2015, Fuzzy Sets Syst..

[3]  F. L. Chen,et al.  Sales forecasting system based on Gray extreme learning machine with Taguchi method in retail industry , 2011, Expert Syst. Appl..

[4]  Dianhui Wang,et al.  Stochastic Configuration Networks: Fundamentals and Algorithms , 2017, IEEE Transactions on Cybernetics.

[5]  Yong Liu,et al.  Optimal Weighted Extreme Learning Machine for Imbalanced Learning with Differential Evolution [Research Frontier] , 2018, IEEE Computational Intelligence Magazine.

[6]  Jianping Yin,et al.  Boosting weighted ELM for imbalanced learning , 2014, Neurocomputing.

[7]  Changyin Sun,et al.  ODOC-ELM: Optimal decision outputs compensation-based extreme learning machine for classifying imbalanced data , 2016, Knowl. Based Syst..

[8]  Jorge Nocedal,et al.  Algorithm 778: L-BFGS-B: Fortran subroutines for large-scale bound-constrained optimization , 1997, TOMS.

[9]  Hongming Zhou,et al.  Extreme Learning Machine for Regression and Multiclass Classification , 2012, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[10]  Jorge Nocedal,et al.  A Limited Memory Algorithm for Bound Constrained Optimization , 1995, SIAM J. Sci. Comput..

[11]  Nitesh V. Chawla,et al.  SMOTE: Synthetic Minority Over-sampling Technique , 2002, J. Artif. Intell. Res..

[12]  Qinghua Zheng,et al.  Regularized Extreme Learning Machine , 2009, 2009 IEEE Symposium on Computational Intelligence and Data Mining.

[13]  Kar-Ann Toh,et al.  Deterministic Neural Classification , 2008, Neural Computation.

[14]  Jun Li,et al.  ${{\rm E}^{2}}{\rm LMs}$ : Ensemble Extreme Learning Machines for Hyperspectral Image Classification , 2014, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[15]  Wentao Mao,et al.  An ELM-based model with sparse-weighting strategy for sequential data imbalance problem , 2016, International Journal of Machine Learning and Cybernetics.

[16]  Huaxiang Zhang,et al.  RWO-Sampling: A random walk over-sampling approach to imbalanced data classification , 2014, Inf. Fusion.

[17]  Guang-Bin Huang,et al.  Convex incremental extreme learning machine , 2007, Neurocomputing.

[18]  Francisco Charte,et al.  MLSMOTE: Approaching imbalanced multilabel learning through synthetic instance generation , 2015, Knowl. Based Syst..

[19]  Jesús Alcalá-Fdez,et al.  KEEL Data-Mining Software Tool: Data Set Repository, Integration of Algorithms and Experimental Analysis Framework , 2011, J. Multiple Valued Log. Soft Comput..

[20]  Simon Günter,et al.  A Stochastic Quasi-Newton Method for Online Convex Optimization , 2007, AISTATS.

[21]  Wentao Mao,et al.  Online sequential prediction of bearings imbalanced fault diagnosis by extreme learning machine , 2017 .

[22]  Yiqiang Chen,et al.  Weighted extreme learning machine for imbalance learning , 2013, Neurocomputing.

[23]  Dianhui Wang,et al.  Extreme learning machines: a survey , 2011, Int. J. Mach. Learn. Cybern..

[24]  Francisco Herrera,et al.  A study of statistical techniques and performance measures for genetics-based machine learning: accuracy and interpretability , 2009, Soft Comput..

[25]  Chee Kheong Siew,et al.  Extreme learning machine: Theory and applications , 2006, Neurocomputing.

[26]  Mohammad Bagher Menhaj,et al.  Training feedforward networks with the Marquardt algorithm , 1994, IEEE Trans. Neural Networks.

[27]  Xizhao Wang,et al.  A deep stochastic weight assignment network and its application to chess playing , 2018, J. Parallel Distributed Comput..

[28]  Jie Du,et al.  Postboosting Using Extended G-Mean for Online Sequential Multiclass Imbalance Learning , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[29]  Chi-Man Vong,et al.  Post-boosting of classification boundary for imbalanced data using geometric mean , 2017, Neural Networks.

[30]  Ran Wang,et al.  Noniterative Deep Learning: Incorporating Restricted Boltzmann Machine Into Multilayer Random Weight Neural Networks , 2019, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[31]  Jorge Nocedal,et al.  Remark on “algorithm 778: L-BFGS-B: Fortran subroutines for large-scale bound constrained optimization” , 2011, TOMS.

[32]  Xizhao Wang,et al.  Non-iterative approaches in training feed-forward neural networks and their applications , 2018, Soft Computing.

[33]  T. M. Williams,et al.  Practical Methods of Optimization. Vol. 1: Unconstrained Optimization , 1980 .

[34]  Nasser M. Nasrabadi,et al.  Pattern Recognition and Machine Learning , 2006, Technometrics.

[35]  Gerald Schaefer,et al.  Cost-sensitive decision tree ensembles for effective imbalanced classification , 2014, Appl. Soft Comput..

[36]  Min Xia,et al.  Fashion retailing forecasting based on extreme learning machine with adaptive metrics of inputs , 2012, Knowl. Based Syst..

[37]  Aryan Mokhtari,et al.  RES: Regularized Stochastic BFGS Algorithm , 2014, IEEE Transactions on Signal Processing.

[38]  Changyin Sun,et al.  Support vector machine-based optimized decision threshold adjustment strategy for classifying imbalanced data , 2015, Knowl. Based Syst..

[39]  Aryan Mokhtari,et al.  Global convergence of online limited memory BFGS , 2014, J. Mach. Learn. Res..