Model level combination of tree ensemble hyperboxes via GFMM

An ensemble of decision trees defines an overlapping set of hyperboxes. These hyperboxes in turn define a disjoint set of hyperboxes each with an associated vector of individual decisions. These vectors can be used to robustly label the hyperboxes by class, or to define soft labels. We sample from these hyperboxes and use them to build a single classifier within the General Fuzzy Min-Max (GFMM) framework that gains information from many different resamplings of the data through the ensemble from which it is built. This method is found to build robust GFMM models, with improved performance on most datasets compared to the basic GFMM.

[1]  Jacek M. Zurada,et al.  Could Decision Trees Improve the Classification Accuracy and Interpretability of Loan Granting Decisions? , 2010, 2010 43rd Hawaii International Conference on System Sciences.

[2]  Vadlamani Ravi,et al.  Predicting credit card customer churn in banks using data mining , 2008, Int. J. Data Anal. Tech. Strateg..

[3]  Wan-Jui Lee,et al.  Kernel Combination Versus Classifier Combination , 2007, MCS.

[4]  Subhash C. Bagui,et al.  Combining Pattern Classifiers: Methods and Algorithms , 2005, Technometrics.

[5]  Padraig Cunningham,et al.  A Case-Based Explanation System for Black-Box Systems , 2005, Artificial Intelligence Review.

[6]  Bogdan Gabrys,et al.  Learning hybrid neuro-fuzzy classifier models from data: to combine or not to combine? , 2004, Fuzzy Sets Syst..

[7]  Ludmila I. Kuncheva,et al.  Combining Pattern Classifiers: Methods and Algorithms , 2004 .

[8]  Nello Cristianini,et al.  Learning the Kernel Matrix with Semidefinite Programming , 2002, J. Mach. Learn. Res..

[9]  Leo Breiman,et al.  Bagging Predictors , 1996, Machine Learning.

[10]  José Hernández-Orallo,et al.  From Ensemble Methods to Comprehensible Models , 2002, Discovery Science.

[11]  Bogdan Gabrys,et al.  Combining neuro-fuzzy classifiers for improved generalisation and reliability , 2002, Proceedings of the 2002 International Joint Conference on Neural Networks. IJCNN'02 (Cat. No.02CH37290).

[12]  Bogdan Gabrys,et al.  Agglomerative Learning Algorithms for General Fuzzy Min-Max Neural Network , 2002, J. VLSI Signal Process..

[13]  Robert P.W. Duin,et al.  PRTools3: A Matlab Toolbox for Pattern Recognition , 2000 .

[14]  Pedro M. Domingos.  Knowledge Discovery Via Multiple Models , 1998, Intell. Data Anal..

[15]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[16]  George J. Klir,et al.  Fuzzy sets and fuzzy logic - theory and applications , 1995 .

[17]  M. Hall.  Combining particles and waves for fluid animation , 1992 .