Decision tree ensemble construction incorporating feature values modification and random subspace method

The goal of ensemble construction with several classifiers is to achieve better generalization ability over individual classifiers. An ensemble method produces diverse classifiers and combines their decisions for ensemble's decision. A number of methods have been investigated for constructing ensemble in which some of them train classifiers with the generated patterns. This study investigates a decision tree ensemble method incorporating some generated patterns with random subspace method (RSM). The proposed hybrid ensemble method were evaluated on several benchmark classification problems, and was found to achieve performance better than or competitive with related conventional methods.

[1]  Xin Yao,et al.  Ensemble learning via negative correlation , 1999, Neural Networks.

[2]  Xin Yao,et al.  Diversity creation methods: a survey and categorisation , 2004, Inf. Fusion.

[3]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[4]  Lawrence O. Hall,et al.  A Comparison of Decision Tree Ensemble Creation Techniques , 2007 .

[5]  KohaviRon,et al.  An Empirical Comparison of Voting Classification Algorithms , 1999 .

[6]  Haytham Elghazel,et al.  A semi-supervised feature ranking method with ensemble learning , 2012, Pattern Recognit. Lett..

[7]  Sotiris B. Kotsiantis,et al.  Combining bagging, boosting, rotation forest and random subspace methods , 2011, Artificial Intelligence Review.

[8]  Gonzalo Martínez-Muñoz,et al.  Switching class labels to generate classification ensembles , 2005, Pattern Recognit..

[9]  Yoav Freund,et al.  Experiments with a New Boosting Algorithm , 1996, ICML.

[10]  Dan Zhu,et al.  A hybrid approach for efficient ensembles , 2010, Decis. Support Syst..

[11]  Laurent Heutte,et al.  Dynamic Random Forests , 2012, Pattern Recognit. Lett..

[12]  Leo Breiman,et al.  Bagging Predictors , 1996, Machine Learning.

[13]  Amanda J. C. Sharkey,et al.  On Combining Artificial Neural Nets , 1996, Connect. Sci..

[14]  Mykola Pechenizkiy,et al.  Diversity in search strategies for ensemble feature selection , 2005, Inf. Fusion.

[15]  Chun-Xia Zhang,et al.  RotBoost: A technique for combining Rotation Forest and AdaBoost , 2008, Pattern Recognit. Lett..

[16]  Kazuyuki Murase,et al.  Ensembles of Neural Networks Based on the Alteration of Input Feature Values , 2012, Int. J. Neural Syst..

[17]  R. Polikar,et al.  Bootstrap - Inspired Techniques in Computation Intelligence , 2007, IEEE Signal Processing Magazine.

[18]  Ian H. Witten,et al.  Weka: Practical machine learning tools and techniques with Java implementations , 1999 .

[19]  Eric Bauer,et al.  An Empirical Comparison of Voting Classification Algorithms: Bagging, Boosting, and Variants , 1999, Machine Learning.

[20]  D. J. Newman,et al.  UCI Repository of Machine Learning Database , 1998 .

[21]  Kazuyuki Murase,et al.  A Comparative Study of Data Sampling Techniques for Constructing Neural Network Ensembles , 2009, Int. J. Neural Syst..

[22]  Huanhuan Chen,et al.  Trade-Off Between Diversity and Accuracy in Ensemble Generation , 2006, Multi-Objective Machine Learning.

[23]  Tin Kam Ho,et al.  The Random Subspace Method for Constructing Decision Forests , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[24]  Raymond J. Mooney,et al.  Creating diversity in ensembles using artificial data , 2005, Inf. Fusion.

[25]  Nicolás García-Pedrajas,et al.  Random feature weights for decision tree ensemble construction , 2012, Inf. Fusion.

[26]  Aiko M. Hormann,et al.  Programs for Machine Learning. Part I , 1962, Inf. Control..

[27]  D. Opitz,et al.  Popular Ensemble Methods: An Empirical Study , 1999, J. Artif. Intell. Res..