Evolutionary stratified training set selection for extracting classification rules with trade off precision-interpretability

The generation of predictive models is a frequent task in data mining with the objective of generating highly precise and interpretable models. The data reduction is an interesting preprocessing approach that can allow us to obtain predictive models with these characteristics in large size data sets. In this paper, we analyze the rule classification model based on decision trees using a training selected set via evolutionary stratified instance selection. This method faces the scaling problem that appears in the evaluation of large size data sets, and the trade off interpretability-precision of the generated models.

[1]  J. Ross Quinlan,et al.  C4.5: Programs for Machine Learning , 1992 .

[2]  Ludmila I. Kuncheva,et al.  Editing for the k-nearest neighbors rule by a genetic algorithm , 1995, Pattern Recognit. Lett..

[3]  Lawrence O. Hall,et al.  Error-Based Pruning of Decision Trees Grown on Very Large Data Sets Can Work! , 2002, ICTAI.

[4]  Tony R. Martinez,et al.  Reduction Techniques for Instance-Based Learning Algorithms , 2000, Machine Learning.

[5]  LastMark,et al.  A Compact and Accurate Model for Classification , 2004 .

[6]  G. Gates,et al.  The reduced nearest neighbor rule (Corresp.) , 1972, IEEE Trans. Inf. Theory.

[7]  Tim Oates,et al.  The Effects of Training Set Size on Decision Tree Complexity , 1997, ICML.

[8]  D. J. Newman,et al.  UCI Repository of Machine Learning Database , 1998 .

[9]  Christopher J. Merz,et al.  UCI Repository of Machine Learning Databases , 1996 .

[10]  Anja Vogler,et al.  An Introduction to Multivariate Statistical Analysis , 2004 .

[11]  Miguel Toro,et al.  Finding representative patterns with ordered projections , 2003, Pattern Recognit..

[12]  Peter E. Hart,et al.  The condensed nearest neighbor rule (Corresp.) , 1968, IEEE Trans. Inf. Theory.

[13]  Ian H. Witten,et al.  Data mining: practical machine learning tools and techniques with Java implementations , 2002, SGMD.

[14]  Francisco Herrera,et al.  Using evolutionary algorithms as instance selection for data reduction in KDD: an experimental study , 2003, IEEE Trans. Evol. Comput..

[15]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[16]  D. Fogel.  Evolutionary algorithms in theory and practice , 1997, Complex..

[17]  D. E. Goldberg,et al.  Genetic Algorithms in Search , 1989 .

[18]  George H. John.  Robust Decision Trees: Removing Outliers from Databases , 1995, KDD.

[19]  Huan Liu,et al.  On Issues of Instance Selection , 2002, Data Mining and Knowledge Discovery.

[20]  Ian Witten,et al.  Data Mining , 2000 .

[21]  Ivan Bratko,et al.  Trading Accuracy for Simplicity in Decision Trees , 1994, Machine Learning.

[22]  Carlos Eduardo Pedreira,et al.  Learning vector quantization with training data selection , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[23]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[24]  Aiko M. Hormann,et al.  Programs for Machine Learning. Part I , 1962, Inf. Control..

[25]  Tim Oates,et al.  Large Datasets Lead to Overly Complex Models: An Explanation and a Solution , 1998, KDD.

[26]  HerreraFrancisco,et al.  Stratification for scaling up evolutionary prototype selection , 2005 .

[27]  Ian H. Witten,et al.  Data mining: practical machine learning tools and techniques, 3rd Edition , 1999 .

[28]  Marek Grochowski,et al.  Comparison of Instances Seletion Algorithms I. Algorithms Survey , 2004, ICAISC.

[29]  Colin R. Reeves,et al.  Using Genetic Algorithms for Training Data Selection in RBF Networks , 2001 .

[30]  Larry J. Eshelman,et al.  The CHC Adaptive Search Algorithm: How to Have Safe Search When Engaging in Nontraditional Genetic Recombination , 1990, FOGA.

[31]  David W. Aha,et al.  Simplifying decision trees: A survey , 1997, The Knowledge Engineering Review.

[32]  Richard Nock,et al.  Instance Pruning as an Information Preserving Problem , 2000, ICML.

[33]  Inés María Galván,et al.  How the Selection of Training Patterns can Improve the Generalization Capability in Radial Basis Neural Networks , 2003, Applied Informatics.

[34]  Donato Malerba,et al.  A Comparative Analysis of Methods for Pruning Decision Trees , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[35]  Ian H. Witten,et al.  Comprar Data Mining . Practical Machine Learning Tools and Techniques | Ian H. Witten | 9780120884070 | Morgan Kaufmann , 2008 .

[36]  David W. Aha,et al.  Instance-Based Learning Algorithms , 1991, Machine Learning.

[37]  Miguel Toro,et al.  Data set Editing by Ordered Projection , 2000, Intell. Data Anal..

[38]  Marek Grochowski,et al.  Comparison of Instance Selection Algorithms II. Results and Comments , 2004, ICAISC.

[39]  Zhi-Hua Zhou,et al.  Medical diagnosis with C4.5 rule preceded by artificial neural network ensemble , 2003, IEEE Transactions on Information Technology in Biomedicine.

[40]  Shinn-Ying Ho,et al.  Design of an optimal nearest neighbor classifier using an intelligent genetic algorithm , 2002, Pattern Recognit. Lett..

[41]  Richard Nock,et al.  Impact of learning set quality and size on decision tree performances , 2000, Int. J. Comput. Syst. Signals.

[42]  Pedro Larrañaga,et al.  Prototype Selection and Feature Subset Selection by Estimation of Distribution Algorithms. A Case Study in the Survival of Cirrhotic Patients Treated with TIPS , 2001, AIME.

[43]  Kweku-Muata Osei-Bryson,et al.  Evaluation of decision trees: a multi-criteria approach , 2004, Comput. Oper. Res..

[44]  David W. Aha,et al.  Learning Representative Exemplars of Concepts: An Initial Case Study , 1987 .

[45]  Thomas Bäck,et al.  Evolutionary algorithms in theory and practice - evolution strategies, evolutionary programming, genetic algorithms , 1996 .

[46]  Cullen Schaffer.  When Does Overfitting Decrease Prediction Accuracy in Induced Decision Trees and Rule Sets? , 1991, EWSL.

[47]  Mark Last,et al.  A compact and accurate model for classification , 2004, IEEE Transactions on Knowledge and Data Engineering.

[48]  Francisco Herrera,et al.  Stratification for scaling up evolutionary prototype selection , 2005, Pattern Recognit. Lett..

[49]  Hugh B. Woodruff,et al.  An algorithm for a selective nearest neighbor decision rule (Corresp.) , 1975, IEEE Trans. Inf. Theory.