Hybrid Ensemble Pruning Using Coevolution Binary Glowworm Swarm Optimization and Reduce-Error

Ensemble pruning has been widely applied to improve the capacity of multiple learner system. Both diversity and classification accuracy of learners are considered as two key factors for achieving an ensemble with competitive classification ability. Considering that extreme learning machine (ELM) is characterized by excellent training rate and generalization capability, it is employed as the base classifier. For a multiple ELM system, when we increase its constituents’ diversity, the mean accuracy of the whole members must be decreased. Therefore, a compromise between them can ensure that the ELMs remain good diversity and high precision, but finding the compromise brings a heavy computational burden. It is hard to look for the exact result via the searching of intelligent algorithms or pruning of diversity measures. On the basis, we propose a hybrid ensemble pruning approach employing coevolution binary glowworm swarm optimization and reduce-error (HEPCBR). Considering the good performance of reduce-error (RE) in selecting ELMs with high diversity and precision, we try to employ RE to choose the satisfactory ELMs from the generated ELMs. In addition, the constituents are further selected via the proposed coevolution binary glowworm swarm optimization, which are utilized to construct the promising ensemble. Experimental results indicate that, compared to other frequently used methods, the proposed HEPCBR achieves significantly superior performance in classification.

[1]  Gang Song,et al.  A novel double deep ELMs ensemble system for time series forecasting , 2017, Knowl. Based Syst..

[2]  Leo Breiman,et al.  Bagging Predictors , 1996, Machine Learning.

[3]  Xue Yang,et al.  An Extreme Learning Machine based on Cellular Automata of edge detection for remote sensing images , 2016, Neurocomputing.

[4]  Yun Yang,et al.  Ensemble Learning-Based Person Re-identification with Multiple Feature Representations , 2018, Complex..

[5]  Jia Wu,et al.  Memetic Extreme Learning Machine , 2016, Pattern Recognit..

[6]  Kai Li,et al.  D2C: Deep cumulatively and comparatively learning for human age estimation , 2017, Pattern Recognit..

[7]  Huijuan Lu,et al.  A cost-sensitive rotation forest algorithm for gene expression data classification , 2017, Neurocomputing.

[8]  Xinli Yang,et al.  TLEL: A two-layer ensemble learning approach for just-in-time defect prediction , 2017, Inf. Softw. Technol..

[9]  Tingzhang Liu,et al.  A novel attribute reduction algorithm based on rough set and improved artificial fish swarm algorithm , 2016, Neurocomputing.

[10]  Djamel Bouchaffra,et al.  An efficient ensemble pruning approach based on simple coalitional games , 2017, Inf. Fusion.

[11]  Rui Ye,et al.  Considering diversity and accuracy simultaneously for ensemble pruning , 2017, Appl. Soft Comput..

[12]  Grigorios Tsoumakas,et al.  Pruning an ensemble of classifiers via reinforcement learning , 2009, Neurocomputing.

[13]  Xiaojun Liu,et al.  Interactive Algorithms in Complex Image Processing Systems Based on Big Data , 2020, Complex..

[14]  Wang Xiao,et al.  An effective ensemble pruning algorithm based on frequent patterns , 2014 .

[15]  Yu-Bin Yang,et al.  Lung cancer cell identification based on artificial neural network ensembles , 2002, Artif. Intell. Medicine.

[16]  Mamun Bin Ibne Reaz,et al.  A survey of intrusion detection systems based on ensemble and hybrid classifiers , 2017, Comput. Secur..

[17]  Zhi Chen,et al.  Maximum relevancy maximum complementary based ordered aggregation for ensemble pruning , 2017, Applied Intelligence.

[18]  Gian Luca Marcialis,et al.  Palmprint recognition with an efficient data driven ensemble classifier , 2019, Pattern Recognit. Lett..

[19]  Thiago J. M. Moura,et al.  Combining diversity measures for ensemble pruning , 2016, Pattern Recognit. Lett..

[20]  Xiaobo Liu,et al.  Hierarchical ensemble of Extreme Learning Machine , 2018, Pattern Recognit. Lett..

[21]  Yu Xue,et al.  Wavelet twin support vector machines based on glowworm swarm optimization , 2017, Neurocomputing.

[22]  R. Naresh,et al.  Binary fish swarm algorithm for profit-based unit commitment problem in competitive electricity market with ramp rate constraints , 2015 .

[23]  Pádraig Cunningham,et al.  Stability problems with artificial neural networks and the ensemble solution , 2000, Artif. Intell. Medicine.

[24]  Chee Kheong Siew,et al.  Extreme learning machine: Theory and applications , 2006, Neurocomputing.

[25]  Ana Maria A. C. Rocha,et al.  Improved binary artificial fish swarm algorithm for the 0-1 multidimensional knapsack problems , 2014, Swarm Evol. Comput..

[26]  Ting Zhang,et al.  A new reverse reduce-error ensemble pruning algorithm , 2015, Appl. Soft Comput..

[27]  Huijuan Lu,et al.  Kernel principal component analysis combining rotation forest method for linearly inseparable data , 2018, Cognitive Systems Research.

[28]  Daniel Hernández-Lobato,et al.  An Analysis of Ensemble Pruning Techniques Based on Ordered Aggregation , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[29]  Qun Dai,et al.  A Novel Greedy Randomized Dynamic Ensemble Selection Algorithm , 2017, Neural Processing Letters.

[30]  Xianfeng Li,et al.  An integrated K-means - Laplacian cluster ensemble approach for document datasets , 2016, Neurocomputing.

[31]  Jian Yu,et al.  Deterministic annealing Gustafson-Kessel fuzzy clustering algorithm , 2017, Inf. Sci..

[32]  Yang Yu,et al.  Diversity Regularized Ensemble Pruning , 2012, ECML/PKDD.

[33]  Debasish Ghose,et al.  Glowworm swarm optimization for simultaneous capture of multiple local optima of multimodal functions , 2009, Swarm Intelligence.

[34]  Chuanwen JIANG,et al.  Binary glowworm swarm optimization for unit commitment , 2014 .

[35]  Mingliang Xu,et al.  Margin & diversity based ordering ensemble pruning , 2018, Neurocomputing.

[36]  Jean-Luc Dugelay,et al.  Effective training of convolutional neural networks for face-based gender and age prediction , 2017, Pattern Recognit..

[37]  Zhe Zhang,et al.  Deep Ensemble Learning for Human Action Recognition in Still Images , 2020, Complex..

[38]  Huijuan Lu,et al.  Applying Cost-Sensitive Extreme Learning Machine and Dissimilarity Integration to Gene Expression Data Classification , 2016, Comput. Intell. Neurosci..

[39]  Aytug Onan,et al.  A hybrid ensemble pruning approach based on consensus clustering and multi-objective evolutionary algorithm for sentiment classification , 2017, Inf. Process. Manag..

[40]  Gary R. Weckman,et al.  Selective ensemble based on extreme learning machine and improved discrete artificial fish swarm algorithm for haze forecast , 2017, Applied Intelligence.

[41]  Debasish Ghose,et al.  Glowworm swarm optimisation: a new method for optimising multi-modal functions , 2009, Int. J. Comput. Intell. Stud..

[42]  Samia Boukir,et al.  Margin-based ordered aggregation for ensemble pruning , 2013, Pattern Recognit. Lett..

[43]  Yi Lu,et al.  Dissimilarity based ensemble of extreme learning machine for gene expression data classification , 2014, Neurocomputing.

[44]  Xuhui Zhu,et al.  A Novel Prediction Method Based on Improved Binary Glowworm Swarm Optimization and Multi-Fractal Dimension for P2P Lending Investment Risk , 2020, IEEE Access.

[45]  Wei Tang,et al.  Ensembling neural networks: Many could be better than all , 2002, Artif. Intell..

[46]  Zhi Chen,et al.  Pruning the Ensemble of ANN Based on Decision Tree Induction , 2017, Neural Processing Letters.

[47]  Qun Dai,et al.  A novel ensemble pruning algorithm based on randomized greedy selective strategy and ballot , 2013, Neurocomputing.