An efficient ensemble classification method based on novel classifier selection technique

Individual classification models have recently been challenged by ensemble of classifiers, also known as multiple classifier system, which often shows better classification accuracy. In terms of merging the outputs of an ensemble of classifiers, classifier selection has not attracted as much attention as classifier fusion in the past, mainly because of its higher computational burden. In this paper, we propose a novel technique for improving classifier selection. In our method, the simple divide-and-conquer strategy is adapted in that a complex classification problem is divided into simpler binary sub-classification problems. The proposed ensemble classification technique has the following advantages: f) it requires much less computation than the existing ensemble classification methods. 2) It improves overall classification accuracy. 3) It is also suitable for tackling the classification problems which have a relatively large number of target classes. We conduct extensive experiments on a series of multi-class datasets from the UCI (University of California, Irvine) repository and compare several well-known classification approaches. The experimental results demonstrate the advanced performance of the proposed method.

[1]  Noel E. Sharkey,et al.  The "Test and Select" Approach to Ensemble Combination , 2000, Multiple Classifier Systems.

[2]  Padraig Cunningham,et al.  Overfitting in Wrapper-Based Feature Subset Selection: The Harder You Try the Worse it Gets , 2004, SGAI Conf..

[3]  Jiri Matas,et al.  On Combining Classifiers , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[4]  Fabio Roli,et al.  A theoretical and experimental analysis of linear combiners for multiple classifier systems , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[5]  David W. Opitz,et al.  Feature Selection for Ensembles , 1999, AAAI/IAAI.

[6]  Fabio Roli,et al.  Adaptive Selection of Image Classifiers , 1997, ICIAP.

[7]  B.V. Dasarathy,et al.  A composite classifier system design: Concepts and methodology , 1979, Proceedings of the IEEE.

[8]  Leo Breiman,et al.  Bagging Predictors , 1996, Machine Learning.

[9]  Yoav Freund,et al.  A decision-theoretic generalization of on-line learning and an application to boosting , 1995, EuroCOLT.

[10]  Lawrence O. Hall,et al.  A Comparison of Decision Tree Ensemble Creation Techniques , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  Anders Krogh,et al.  Neural Network Ensembles, Cross Validation, and Active Learning , 1994, NIPS.

[12]  José Pedro Santos,et al.  Analysis of neural networks and analysis of feature selection with genetic algorithm to discriminate among pollutant gas , 2004 .

[13]  Subhash C. Bagui,et al.  Combining Pattern Classifiers: Methods and Algorithms , 2005, Technometrics.

[14]  R. Polikar,et al.  Ensemble based systems in decision making , 2006, IEEE Circuits and Systems Magazine.

[15]  Tin Kam Ho,et al.  MULTIPLE CLASSIFIER COMBINATION: LESSONS AND NEXT STEPS , 2002 .

[16]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[17]  Kevin W. Bowyer,et al.  Combination of multiple classifiers using local accuracy estimates , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[18]  Ludmila I. Kuncheva,et al.  Combining Pattern Classifiers: Methods and Algorithms , 2004 .

[19]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[20]  Oleksandr Makeyev,et al.  Neural network with ensembles , 2010, The 2010 International Joint Conference on Neural Networks (IJCNN).

[21]  Eric Bauer,et al.  An Empirical Comparison of Voting Classification Algorithms: Bagging, Boosting, and Variants , 1999, Machine Learning.

[22]  D. J. Newman,et al.  UCI Repository of Machine Learning Database , 1998 .

[23]  Fabio Roli,et al.  An approach to the automatic design of multiple classifier systems , 2001, Pattern Recognit. Lett..

[24]  Robert Sabourin,et al.  A dynamic overproduce-and-choose strategy for the selection of classifier ensembles , 2008, Pattern Recognit..

[25]  Bernhard Schölkopf,et al.  Feature selection for support vector machines by means of genetic algorithm , 2003, Proceedings. 15th IEEE International Conference on Tools with Artificial Intelligence.

[26]  Tin Kam Ho,et al.  The Random Subspace Method for Constructing Decision Forests , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[27]  Lior Rokach,et al.  Genetic algorithm-based feature set partitioning for classification problems , 2008, Pattern Recognit..

[28]  Xiaoyi Jiang,et al.  A dynamic classifier ensemble selection approach for noise data , 2010, Inf. Sci..