A Reinforcement-based Mechanism to Select Features for Classifiers in Ensemble Systems

Classifier ensemble are systems composed of a set of individual classifiers structured in a parallel way and a combination module, which is responsible for providing the final output of the system. One way of increasing diversity in classifier ensembles is to use feature selection methods in order to select subsets of attributes for the individual classifiers. In this paper, it is investigated the use of a simple reinforcementbased method, called ReinSel, in ensemble systems. This method is inserted into the filter approach of feature selection methods and it chooses only the attributes that are important only for a specific class through the use of a reinforcement procedure.

[1]  Tong Zhang,et al.  On the Consistency of Feature Selection using Greedy Least Squares Regression , 2009, J. Mach. Learn. Res..

[2]  Ludmila I. Kuncheva,et al.  Measures of Diversity in Classifier Ensembles and Their Relationship with the Ensemble Accuracy , 2003, Machine Learning.

[3]  Juan José Rodríguez Diez,et al.  Rotation Forest: A New Classifier Ensemble Method , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[4]  Robert Sabourin,et al.  Combining Diversity and Classification Accuracy for Ensemble Selection in Random Subspaces , 2006, The 2006 IEEE International Joint Conference on Neural Network Proceedings.

[5]  M. Bacauskienea,et al.  A feature selection technique for generation of classification committees and its application to categorization of laryngeal images , 2013 .

[6]  L. Darrell Whitley,et al.  Genetic Approach to Feature Selection for Ensemble Creation , 1999, GECCO.

[7]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[8]  Kevin W. Bowyer,et al.  Combination of multiple classifiers using local accuracy estimates , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[9]  Tin Kam Ho,et al.  The Random Subspace Method for Constructing Decision Forests , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[10]  Byung Ro Moon,et al.  Hybrid Genetic Algorithms for Feature Selection , 2004, IEEE Trans. Pattern Anal. Mach. Intell..

[11]  Kagan Tumer,et al.  Input Decimation Ensembles: Decorrelation through Dimensionality Reduction , 2001, Multiple Classifier Systems.

[12]  Jill P. Mesirov,et al.  Consensus Clustering: A Resampling-Based Method for Class Discovery and Visualization of Gene Expression Microarray Data , 2003, Machine Learning.

[13]  C. Spearman The proof and measurement of association between two things. , 2015, International journal of epidemiology.

[14]  Lakhmi C. Jain,et al.  Designing classifier fusion systems by genetic algorithms , 2000, IEEE Trans. Evol. Comput..

[15]  Anne M. P. Canuto,et al.  Investigating the influence of the choice of the ensemble members in accuracy and diversity of selection-based and fusion-based methods for ensembles , 2007, Pattern Recognit. Lett..

[16]  Kagan Tumer,et al.  Input decimated ensembles , 2003, Pattern Analysis & Applications.

[17]  Francisco Herrera,et al.  A First Study on the Use of Coevolutionary Algorithms for Instance and Feature Selection , 2009, HAIS.

[18]  Anne M. P. Canuto,et al.  A Comparative Analysis of Feature Selection Methods for Ensembles with Different Combination Methods , 2007, 2007 International Joint Conference on Neural Networks.

[19]  Lipo Wang,et al.  Data dimensionality reduction with application to simplifying RBF network structure and improving classification performance , 2003, IEEE Trans. Syst. Man Cybern. Part B.

[20]  Mykola Pechenizkiy,et al.  Diversity in search strategies for ensemble feature selection , 2005, Inf. Fusion.

[21]  Alexey Tsymbal,et al.  Ensemble feature selection with the simple Bayesian classification , 2003, Inf. Fusion.

[22]  Chris H. Q. Ding,et al.  Evolving Feature Selection , 2005, IEEE Intell. Syst..

[23]  Qiang Shen,et al.  New Approaches to Fuzzy-Rough Feature Selection , 2009, IEEE Transactions on Fuzzy Systems.

[24]  Vasant Honavar,et al.  Decision Tree Induction from Distributed Heterogeneous Autonomous Data Sources , 2003 .

[25]  Luiz Eduardo Soares de Oliveira,et al.  Feature selection for ensembles applied to handwriting recognition , 2006, International Journal of Document Analysis and Recognition (IJDAR).

[26]  David W. Opitz,et al.  Feature Selection for Ensembles , 1999, AAAI/IAAI.

[27]  Bing Liu,et al.  An efficient semi-unsupervised gene selection method via spectral biclustering , 2006, IEEE Transactions on NanoBioscience.

[28]  J. Downing,et al.  Classification, subtype discovery, and prediction of outcome in pediatric acute lymphoblastic leukemia by gene expression profiling. , 2002, Cancer cell.

[29]  Luc Vandendorpe,et al.  Decision Fusion for Face Authentication , 2004, ICBA.

[30]  Pragnesh Jay Modi,et al.  Classification of examples by multiple agents with private features , 2005, IEEE/WIC/ACM International Conference on Intelligent Agent Technology.

[31]  Anne M. P. Canuto,et al.  A Class-Based Feature Selection Method for Ensemble Systems , 2008, 2008 Eighth International Conference on Hybrid Intelligent Systems.

[32]  Anne M. P. Canuto,et al.  Ensembles of ARTMAP-based neural networks: an experimental study , 2011, Applied Intelligence.