Fundamentals for Design and Construction of a Fuzzy Random Forest

Following Breiman’s methodology, we propose the fundamentals to design and construct a “forest” of randomly generated fuzzy decision trees, i.e., a Fuzzy Random Forest. This approach combines the robustness of multi-classifiers, the construction efficiency of decision trees, the power of the randomness to increase the diversity of the trees in the forest, and the flexibility of fuzzy logic and the fuzzy sets for data managing. A prototype for the method has been constructed and we have implemented some specific strategies for inference in the Fuzzy Random Forest. Some experimental results are given.

[1]  David H. Wolpert,et al.  Stacked generalization , 1992, Neural Networks.

[2]  Hyung Lee-Kwang,et al.  A fuzzy decision tree induction method for fuzzy data , 1999, FUZZ-IEEE'99. 1999 IEEE International Fuzzy Systems. Conference Proceedings (Cat. No.99CH36315).

[3]  R. Schapire.  The Strength of Weak Learnability , 1990, Machine Learning.

[4]  Leo Breiman,et al.  Bagging Predictors , 1996, Machine Learning.

[5]  Juan José Rodríguez Diez,et al.  Rotation Forest: A New Classifier Ensemble Method , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[6]  Tin Kam Ho,et al.  The Random Subspace Method for Constructing Decision Forests , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[7]  James J. Chen,et al.  Classification by ensembles from random partitions of high-dimensional data , 2007, Comput. Stat. Data Anal..

[8]  J.-S.R. Jang,et al.  Structure determination in fuzzy modeling: a fuzzy CART approach , 1994, Proceedings of 1994 IEEE 3rd International Fuzzy Systems Conference.

[9]  María N. Moreno,et al.  An experimental comparative study of web mining methods for recommender systems , 2006 .

[10]  Torsten Hothorn,et al.  Double-Bagging: Combining Classifiers by Bootstrap Aggregation , 2002, Pattern Recognit..

[11]  Ludmila I. Kuncheva,et al.  "Fuzzy" versus "nonfuzzy" in combining classifiers designed by Boosting , 2003, IEEE Trans. Fuzzy Syst..

[12]  Gonzalo Martínez-Muñoz,et al.  Switching class labels to generate classification ensembles , 2005, Pattern Recognit..

[13]  Cezary Z. Janikow,et al.  Fuzzy decision trees: issues and methods , 1998, IEEE Trans. Syst. Man Cybern. Part B.

[14]  Thomas G. Dietterich.  An Experimental Comparison of Three Methods for Constructing Ensembles of Decision Trees: Bagging, Boosting, and Randomization , 2000, Machine Learning.

[15]  Denis Larocque,et al.  An empirical comparison of ensemble methods based on classification trees , 2003 .

[16]  Ian H. Witten,et al.  Data mining: practical machine learning tools and techniques, 3rd Edition , 1999 .

[17]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[18]  Ron Kohavi,et al.  Supervised and Unsupervised Discretization of Continuous Features , 1995, ICML.

[19]  Subhash C. Bagui,et al.  Combining Pattern Classifiers: Methods and Algorithms , 2005, Technometrics.

[20]  Victor J. Rayward-Smith,et al.  Discretisation of Continuous Commercial Database Features for a Simulated Annealing Data Mining Algorithm , 1999, Applied Intelligence.

[21]  Ludmila I. Kuncheva,et al.  Combining Pattern Classifiers: Methods and Algorithms , 2004 .

[22]  Ludmila I. Kuncheva,et al.  A Theoretical Study on Six Classifier Fusion Strategies , 2002, IEEE Trans. Pattern Anal. Mach. Intell..