Ensemble pruning of ELM via migratory binary glowworm swarm optimization and margin distance minimization

Ensemble pruning aims at attaining an ensemble composed of less size of leaners for improving classification ability. Extreme Learning Machine (ELM) is employed as a base learner in this work, in light of its salient features, an initial pool is constructed using ELM. An ensemble composed of ELMs with better performance and diversity can make it perform the best, but the average accuracy of the whole ELMs must be decreased as the increase of diversity among them. Hence there exists a balance between the diversity and the precision of ELMs. Existing works find it via diversity measures or heuristic algorithms, which cannot find the exact tradeoff. To solve the issue, ensemble pruning of ELM via migratory binary glowworm swarm optimization and margin distance minimization (EPEMBM) is proposed utilizing the integration of the proposed migratory binary glowworm swarm optimization (MBGSO) and margin distance minimization (MDM). First, the created ELMs in a pool can be pre-pruned by MDM, and it can markedly downsize the ELMs in the pool, and significantly alleviates its computation overhead. Second, the retaining ELMs are further pruned utilizing MBGSO, and the final ensemble is attained with a high efficiency. Experimental results on 21 UCI classification tasks indicate that EPEMBM outperforms techniques, and that its effectiveness and efficiency. It is a very useful tool for solving the selection problem of ELMs.

[1]  Chuanwen JIANG,et al.  Binary glowworm swarm optimization for unit commitment , 2014 .

[2]  Bipin Kumar Tripathi,et al.  On the complex domain deep machine learning for face recognition , 2017, Applied Intelligence.

[3]  Hosein Alizadeh,et al.  Hierarchical cluster ensemble selection , 2015, Eng. Appl. Artif. Intell..

[4]  Xiaodong Li,et al.  A time-varying transfer function for balancing the exploration and exploitation ability of a binary PSO , 2017, Appl. Soft Comput..

[5]  Fabio Roli,et al.  Design of effective multiple classifier systems by clustering of classifiers , 2000, Proceedings 15th International Conference on Pattern Recognition. ICPR-2000.

[6]  Debasish Ghose,et al.  Glowworm swarm based optimization algorithm for multimodal functions with collective robotics applications , 2006, Multiagent Grid Syst..

[7]  Siddhartha Bhattacharyya,et al.  A group incremental feature selection for classification using rough set theory based genetic algorithm , 2018, Appl. Soft Comput..

[8]  Zhi Chen,et al.  Pruning the Ensemble of ANN Based on Decision Tree Induction , 2017, Neural Processing Letters.

[9]  Daniel Hernández-Lobato,et al.  An Analysis of Ensemble Pruning Techniques Based on Ordered Aggregation , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  Wang Xiao,et al.  An effective ensemble pruning algorithm based on frequent patterns , 2014 .

[11]  Jean-Luc Dugelay,et al.  Effective training of convolutional neural networks for face-based gender and age prediction , 2017, Pattern Recognit..

[12]  Liang Zhao,et al.  A Network of Neural Oscillators for Fractal Pattern Recognition , 2015, Neural Processing Letters.

[13]  Mamun Bin Ibne Reaz,et al.  A survey of intrusion detection systems based on ensemble and hybrid classifiers , 2017, Comput. Secur..

[14]  Leo Breiman,et al.  Bagging Predictors , 1996, Machine Learning.

[15]  Xin Yao,et al.  An analysis of diversity measures , 2006, Machine Learning.

[16]  Yang Chu.  Classifier Ensemble with Diversity: Effectiveness Analysis and Ensemble Optimization , 2014 .

[17]  Yang Yu,et al.  Diversity Regularized Ensemble Pruning , 2012, ECML/PKDD.

[18]  Liang Bai,et al.  A multiple k-means clustering ensemble algorithm to find nonlinearly separable clusters , 2020, Inf. Fusion.

[19]  Chee Kheong Siew,et al.  Extreme learning machine: Theory and applications , 2006, Neurocomputing.

[20]  Debasish Ghose,et al.  Glowworm swarm optimization for simultaneous capture of multiple local optima of multimodal functions , 2009, Swarm Intelligence.

[21]  Guoqiang Wang,et al.  Embedded Manifold-Based Kernel Fisher Discriminant Analysis for Face Recognition , 2016, Neural Processing Letters.

[22]  Bin Li,et al.  Link prediction based on sampling in complex networks , 2017, Applied Intelligence.

[23]  Jia Wu,et al.  Memetic Extreme Learning Machine , 2016, Pattern Recognit..

[24]  Yaxin Bi.  The impact of diversity on the accuracy of evidential classifier ensembles , 2012, Int. J. Approx. Reason..

[25]  Leandro Nunes de Castro,et al.  Particle Swarm Clustering in clustering ensembles: Exploiting pruning and alignment free consensus , 2017, Appl. Soft Comput..

[26]  Lior Rokach,et al.  Collective-agreement-based pruning of ensembles , 2009, Comput. Stat. Data Anal..

[27]  Ling Jing,et al.  Application of image recognition in civil aviation security based on tensor learning , 2017, J. Intell. Fuzzy Syst..

[28]  Sunil Kumar Kashyap,et al.  Matrix-Binary Codes based Genetic Algorithm for path planning of mobile robot , 2017, Comput. Electr. Eng..

[29]  Kai Li,et al.  D2C: Deep cumulatively and comparatively learning for human age estimation , 2017, Pattern Recognit..

[30]  Djamel Bouchaffra,et al.  An efficient ensemble pruning approach based on simple coalitional games , 2017, Inf. Fusion.

[31]  Rui Ye,et al.  Considering diversity and accuracy simultaneously for ensemble pruning , 2017, Appl. Soft Comput..

[32]  Yi Lu,et al.  Dissimilarity based ensemble of extreme learning machine for gene expression data classification , 2014, Neurocomputing.

[33]  Wei Tang,et al.  Ensembling neural networks: Many could be better than all , 2002, Artif. Intell..

[34]  Manuel Graña,et al.  A Two Stage Sequential Ensemble Applied to the Classification of Alzheimer’s Disease Based on MRI Features , 2011, Neural Processing Letters.

[35]  Debasish Ghose,et al.  Glowworm swarm optimisation: a new method for optimising multi-modal functions , 2009, Int. J. Comput. Intell. Stud..

[36]  Zhi Chen,et al.  Maximum relevancy maximum complementary based ordered aggregation for ensemble pruning , 2017, Applied Intelligence.

[37]  Samia Boukir,et al.  Margin-based ordered aggregation for ensemble pruning , 2013, Pattern Recognit. Lett..

[38]  Liu Hong-xia,et al.  Discrete Glowworm Swarm Optimization Algorithm for TSP Problem , 2012 .

[39]  Gonzalo Martínez-Muñoz,et al.  Pruning in ordered bagging ensembles , 2006, ICML.

[40]  R. Naresh,et al.  Binary fish swarm algorithm for profit-based unit commitment problem in competitive electricity market with ramp rate constraints , 2015 .

[41]  Francisco Herrera,et al.  Ordering-based pruning for improving the performance of ensembles of classifiers in the framework of imbalanced datasets , 2016, Inf. Sci..

[42]  Zhiping Lin,et al.  Self-Adaptive Evolutionary Extreme Learning Machine , 2012, Neural Processing Letters.

[43]  S. Karthikeyan,et al.  An ensemble design of intrusion detection system for handling uncertainty using Neutrosophic Logic Classifier , 2012, Knowl. Based Syst..

[44]  Xiaobo Liu,et al.  Hierarchical ensemble of Extreme Learning Machine , 2018, Pattern Recognit. Lett..

[45]  Yu Xue,et al.  Wavelet twin support vector machines based on glowworm swarm optimization , 2017, Neurocomputing.

[46]  Thiago J. M. Moura,et al.  Combining diversity measures for ensemble pruning , 2016, Pattern Recognit. Lett..

[47]  Thomas G. Dietterich,et al.  Pruning Adaptive Boosting , 1997, ICML.

[48]  Mingliang Xu,et al.  Margin & diversity based ordering ensemble pruning , 2018, Neurocomputing.

[49]  Shinichiro Mori,et al.  Deep architecture neural network-based real-time image processing for image-guided radiotherapy. , 2017, Physica medica : PM : an international journal devoted to the applications of physics to medicine and biology : official journal of the Italian Association of Biomedical Physics.

[50]  Xindong Wu,et al.  Ensemble pruning via individual contribution ordering , 2010, KDD.

[51]  Grigorios Tsoumakas,et al.  Pruning an ensemble of classifiers via reinforcement learning , 2009, Neurocomputing.

[52]  Daijin Kim,et al.  Human Face Detection in Digital Video Using SVMEnsemble , 2003, Neural Processing Letters.

[53]  Robert Sabourin,et al.  Dynamic ensembles of exemplar-SVMs for still-to-video face recognition , 2017, Pattern Recognit..

[54]  Ana Maria A. C. Rocha,et al.  Improved binary artificial fish swarm algorithm for the 0-1 multidimensional knapsack problems , 2014, Swarm Evol. Comput..

[55]  Ting Zhang,et al.  A new reverse reduce-error ensemble pruning algorithm , 2015, Appl. Soft Comput..

[56]  A. Rezaee Jordehi,et al.  Binary particle swarm optimisation with quadratic transfer function: A new binary optimisation algorithm for optimal scheduling of appliances in smart homes , 2019, Appl. Soft Comput..