ABC-based stacking method for multilabel classification

Multilabel classification is a supervised learning problem wherein each individual instance is associated with multiple labels. Ensemble methods are effective in managing multilabel classification problems by creating a set of accurate, diverse classifiers and then combining their outputs to produce classifications. This paper presents a novel stacking-based ensemble algorithm, ABC-based stacking, for multilabel classification. The artificial bee colony algorithm, along with a single-layer artificial neural network, is used to find suitable meta-level classifier configurations. The optimization goal of the meta-level classifier is to maximize the average accuracy of classification of all the instances involved. We run an experiment on 10 benchmark datasets of varying domains and compare the proposed approach to five other ensemble algorithms to demonstrate the feasibility and effectiveness of ABC-based stacking.

[1]  Yufei Xia,et al.  A novel heterogeneous ensemble credit scoring model based on bstacking approach , 2018, Expert Syst. Appl..

[2]  Grigorios Tsoumakas,et al.  Multi-Label Classification of Music into Emotions , 2008, ISMIR.

[3]  Mengjie Zhang,et al.  Investigation on particle swarm optimisation for feature selection on high-dimensional data: local search and selection bias , 2016, Connect. Sci..

[4]  Haibing Li,et al.  Applying Ant Colony Optimization to configuring stacking ensembles for data mining , 2014, Expert Syst. Appl..

[5]  Dejan Gjorgjevikj,et al.  Two Stage Classifier Chain Architecture for efficient pair-wise multi-label learning , 2011, 2011 IEEE International Workshop on Machine Learning for Signal Processing.

[6]  Jianhua Xu,et al.  Fast multi-label core vector machine , 2013, Pattern Recognit..

[7]  Dervis Karaboga,et al.  AN IDEA BASED ON HONEY BEE SWARM FOR NUMERICAL OPTIMIZATION , 2005 .

[8]  Araceli Sanchis,et al.  Generating ensembles of heterogeneous classifiers using Stacked Generalization , 2015, WIREs Data Mining Knowl. Discov..

[9]  Changsheng Li,et al.  Contour detection via stacking random forest learning , 2018, Neurocomputing.

[10]  Eyke Hüllermeier,et al.  Multilabel classification via calibrated label ranking , 2008, Machine Learning.

[11]  Eyke Hüllermeier,et al.  Combining instance-based learning and logistic regression for multilabel classification , 2009, Machine Learning.

[12]  SunChengyu,et al.  A multi-label feature extraction algorithm via maximizing feature variance and feature-label dependence simultaneously , 2016 .

[13]  Xiaolong Wang,et al.  Reranking for Stacking Ensemble Learning , 2010, ICONIP.

[14]  Geoff Holmes,et al.  Multi-label Classification Using Ensembles of Pruned Sets , 2008, 2008 Eighth IEEE International Conference on Data Mining.

[15]  Yongguang Yu,et al.  A Modified Artificial Bee Colony Algorithm for Parameter Estimation of Fractional-Order Nonlinear Systems , 2018, IEEE Access.

[16]  Mengjie Zhang,et al.  Particle swarm optimisation for feature selection in classification: Novel initialisation and updating mechanisms , 2014, Appl. Soft Comput..

[17]  Grigorios Tsoumakas,et al.  Multi-Label Classification: An Overview , 2007, Int. J. Data Warehous. Min..

[18]  K. Vimala Devi,et al.  Multi-label text classification using optimised feature sets , 2017, Int. J. Data Min. Model. Manag..

[19]  Philip S. Yu,et al.  Multi-label Ensemble Learning , 2011, ECML/PKDD.

[20]  Haojie Li,et al.  Multi-Label Image Categorization With Sparse Factor Representation , 2014, IEEE Transactions on Image Processing.

[21]  Michael K. Ng,et al.  ML-FOREST: A Multi-Label Tree Ensemble Method for Multi-Label Classification , 2016, IEEE Transactions on Knowledge and Data Engineering.

[22]  Araceli Sanchis,et al.  GA-stacking: Evolutionary stacked generalization , 2010, Intell. Data Anal..

[23]  Ron Kohavi,et al.  Wrappers for Feature Subset Selection , 1997, Artif. Intell..

[24]  Hélio Pedrini,et al.  Data feature selection based on Artificial Bee Colony algorithm , 2013, EURASIP J. Image Video Process..

[25]  Grigorios Tsoumakas,et al.  Random k -Labelsets: An Ensemble Method for Multilabel Classification , 2007, ECML.

[26]  Geoff Holmes,et al.  Classifier chains for multi-label classification , 2009, Machine Learning.

[27]  Grigorios Tsoumakas,et al.  An Empirical Study of Lazy Multilabel Classification Algorithms , 2008, SETN.

[28]  S. Kanmani,et al.  Optimization of stacking ensemble configurations through Artificial Bee Colony algorithm , 2013, Swarm Evol. Comput..

[29]  Saso Dzeroski,et al.  Two stage architecture for multi-label learning , 2012, Pattern Recognit..

[30]  Alan Wee-Chung Liew,et al.  Heterogeneous classifier ensemble with fuzzy rule-based meta learner , 2018, Inf. Sci..

[31]  Janez Demsar,et al.  Statistical Comparisons of Classifiers over Multiple Data Sets , 2006, J. Mach. Learn. Res..

[32]  Hamed R. Bonab,et al.  A Novel Online Stacked Ensemble for Multi-Label Stream Classification , 2018, CIKM.

[33]  Mingxia Gao,et al.  A novel artificial bee colony algorithm based on the cosine similarity , 2018, Comput. Ind. Eng..

[34]  Saso Dzeroski,et al.  An extensive experimental comparison of methods for multi-label learning , 2012, Pattern Recognit..

[35]  Saso Dzeroski,et al.  Tree ensembles for predicting structured outputs , 2013, Pattern Recognit..