Contribution of data complexity features on dynamic classifier selection

Different dynamic classifier selection techniques have been proposed in the literature to determine among diverse classifiers available in a pool which should be used to classify a test instance. The individual competence of each classifier in the pool is usually evaluated taking into account its accuracy on the neighborhood of the test instance in a validation dataset. In this work we investigate the possible contribution of considering during the classifier evaluation the use of features related to the problem complexity. Since usually the pool generation technique does not assure diversity, the idea is to consider diversity during the selection. Basically, we select a classifier trained in subset of data showing similar complexity than that observed in neighborhood of the test instance. We expect that this similarity in terms of complexity allow us to select a more competent classifier. Experiments on 30 classification problems representing different levels of difficulty have shown that the proposed selection method is comparable to well known dynamic selection strategies. When compared with other DS approaches it was able to win on 123 over 150 experiments. This promising results indicate that further investigation must be done to increase diversity in terms of data complexity during the process of pool generation.

[1]  Jesús Alcalá-Fdez,et al.  KEEL Data-Mining Software Tool: Data Set Repository, Integration of Algorithms and Experimental Analysis Framework , 2011, J. Multiple Valued Log. Soft Comput..

[2]  Chun Yang,et al.  Sorting-Based Dynamic Classifier Ensemble Selection , 2013, 2013 12th International Conference on Document Analysis and Recognition.

[3]  Tin Kam Ho,et al.  The Random Subspace Method for Constructing Decision Forests , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[4]  Luiz Eduardo Soares de Oliveira,et al.  Dynamic selection of classifiers - A comprehensive review , 2014, Pattern Recognit..

[5]  George D. C. Cavalcanti,et al.  META-DES: A dynamic ensemble selection framework using meta-learning , 2015, Pattern Recognit..

[6]  Robert Sabourin,et al.  From dynamic classifier selection to dynamic ensemble selection , 2008, Pattern Recognit..

[7]  Pierre Loonis,et al.  Combination, Cooperation And Selection Of Classifiers: A State Of The Art , 2003, Int. J. Pattern Recognit. Artif. Intell..

[8]  Amar Mitiche,et al.  Classifier combination for hand-printed digit recognition , 1993, Proceedings of 2nd International Conference on Document Analysis and Recognition (ICDAR '93).

[9]  Anne M. P. Canuto,et al.  A Dynamic Classifier Selection Method to Build Ensembles using Accuracy and Diversity , 2006, 2006 Ninth Brazilian Symposium on Neural Networks (SBRN'06).

[10]  Gian Luca Marcialis,et al.  A study on the performances of dynamic classifier selection based on local accuracy estimation , 2005, Pattern Recognit..

[11]  Robert Sabourin,et al.  Ambiguity-guided dynamic selection of ensemble of classifiers , 2007, 2007 10th International Conference on Information Fusion.

[12]  Cao Feng,et al.  STATLOG: COMPARISON OF CLASSIFICATION ALGORITHMS ON LARGE REAL-WORLD PROBLEMS , 1995 .

[13]  Tin Kam Ho,et al.  Measures of Geometrical Complexity in Classification Problems , 2006 .

[14]  Ludmila I. Kuncheva,et al.  Measures of Diversity in Classifier Ensembles and Their Relationship with the Ensemble Accuracy , 2003, Machine Learning.

[15]  Kevin W. Bowyer,et al.  Combination of multiple classifiers using local accuracy estimates , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[16]  Fabio Roli,et al.  Selection of Classifiers Based on Multiple Classifier Behaviour , 2000, SSPR/SPR.

[17]  Leo Breiman,et al.  Bagging Predictors , 1996, Machine Learning.

[18]  A. Asuncion,et al.  UCI Machine Learning Repository, University of California, Irvine, School of Information and Computer Sciences , 2007 .

[19]  Tin Kam Ho,et al.  Complexity Measures of Supervised Classification Problems , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[20]  Jin Xiao,et al.  Dynamic Classifier Ensemble Selection Based on GMDH , 2009, 2009 International Joint Conference on Computational Sciences and Optimization.

[21]  Fabio Roli,et al.  Methods for dynamic classifier selection , 1999, Proceedings 10th International Conference on Image Analysis and Processing.

[22]  Fabio Roli,et al.  Adaptive Selection of Image Classifiers , 1997, ICIAP.

[23]  Juan José Rodríguez Diez,et al.  Classifier Ensembles with a Random Linear Oracle , 2007, IEEE Transactions on Knowledge and Data Engineering.

[24]  Yoav Freund,et al.  Experiments with a New Boosting Algorithm , 1996, ICML.