MERBIS - A Multi-Objective Evolutionary Rule Base Induction System

Classifier induction is a specific data-mining task that is pa rt of the knowledge discovery process. Its objectives are to extract ac urate, comprehensible and interesting knowledge. However, most classifier i nduction approaches focus only on one of these objectives. This paper introduces a Multi-Objective Evolutionary Rule Base Induction System (MERBIS) that is ca pable of optimising both, accuracy and comprehensibility objectives at the same time. We investigate different parameter sets for this approach and valid ate its performance on several benchmark data sets.

[1]  H. Schwefel,et al.  Applications of Evolutionary Algorithms , 1993 .

[2]  Dr. Alex A. Freitas.  Data Mining and Knowledge Discovery with Evolutionary Algorithms , 2002, Natural Computing Series.

[3]  Michael J. Pazzani,et al.  Comprehensible Knowledge-Discovery in Databases , 1997 .

[4]  John R. Koza,et al.  Genetic Programming II , 1992 .

[5]  Sholom M. Weiss,et al.  Computer Systems That Learn , 1990 .

[6]  B. Silverman.  Density estimation for statistics and data analysis , 1986 .

[7]  J. Ross Quinlan,et al.  C4.5: Programs for Machine Learning , 1992 .

[8]  Ludmila I. Kuncheva,et al.  Fuzzy Classifier Design , 2000, Studies in Fuzziness and Soft Computing.

[9]  Peter Norvig,et al.  Artificial Intelligence: A Modern Approach , 1995 .

[10]  John J. Grefenstette,et al.  Optimization of Control Parameters for Genetic Algorithms , 1986, IEEE Transactions on Systems, Man, and Cybernetics.

[11]  R. Stephenson.  A and V , 1962, The British journal of ophthalmology.

[12]  Marco Laumanns,et al.  Why Quality Assessment Of Multiobjective Optimizers Is Difficult , 2002, GECCO.

[13]  William M. Spears,et al.  Adapting Crossover in Evolutionary Algorithms , 1995, Evolutionary Programming.

[14]  Wlodzislaw Duch,et al.  Optimization and Interpretation of Rule-based Classifiers , 2000, Intelligent Information Systems.

[15]  Daryl Pregibon,et al.  A Statistical Perspective on Knowledge Discovery in Databases , 1996, Advances in Knowledge Discovery and Data Mining.

[16]  David G. Stork,et al.  Pattern classification, 2nd Edition , 2000 .

[17]  C. Apte,et al.  Data mining with decision trees and decision rules , 1997, Future Gener. Comput. Syst..

[18]  Vasant Dhar,et al.  Discovering Interesting Patterns for Investment Decision Making with GLOWER ☹—A Genetic Learner Overlaid with Entropy Reduction , 2000, Data Mining and Knowledge Discovery.

[19]  K. Dejong,et al.  An analysis of the behavior of a class of genetic adaptive systems , 1975 .

[20]  Hisao Ishibuchi,et al.  Three-objective genetics-based machine learning for linguistic rule extraction , 2001, Inf. Sci..

[21]  Joshua D. Knowles,et al.  On metrics for comparing nondominated sets , 2002, Proceedings of the 2002 Congress on Evolutionary Computation. CEC'02 (Cat. No.02TH8600).

[22]  Kalyanmoy Deb,et al.  Multi-objective optimization using evolutionary algorithms , 2001, Wiley-Interscience series in systems and optimization.

[23]  JOHANNES FÜRNKRANZ,et al.  Separate-and-Conquer Rule Learning , 1999, Artificial Intelligence Review.

[24]  Foster J. Provost,et al.  RL4: a tool for knowledge-based induction , 1990, [1990] Proceedings of the 2nd International IEEE Conference on Tools for Artificial Intelligence.

[25]  Rajarshi Das,et al.  A Study of Control Parameters Affecting Online Performance of Genetic Algorithms for Function Optimization , 1989, ICGA.

[26]  David J. Hand,et al.  A Simple Generalisation of the Area Under the ROC Curve for Multiple Class Classification Problems , 2001, Machine Learning.

[27]  Peter J. Fleming,et al.  An Adaptive Divide-and-ConquerMethodology forEvolutionary Multi-criterion Optimisation , 2003, EMO.

[28]  Ute St. Clair,et al.  Fuzzy Set Theory: Foundations and Applications , 1997 .

[29]  P J Lucas,et al.  Converting a rule-based expert system into a belief network. , 1993, Medical informatics = Medecine et informatique.

[30]  Sholom M. Weiss,et al.  Reduced Complexity Rule Induction , 1991, IJCAI.

[31]  A. Nurnberger,et al.  Analyzing borders between partially contradicting fuzzy classification rules , 2000, PeachFuzz 2000. 19th International Conference of the North American Fuzzy Information Processing Society - NAFIPS (Cat. No.00TH8500).

[32]  Marco Tomassini,et al.  Evolutionary Algorithms , 1995, Towards Evolvable Hardware.

[33]  Padhraic Smyth,et al.  From Data Mining to Knowledge Discovery in Databases , 1996, AI Mag..

[34]  Andrew P. Bradley,et al.  The use of the area under the ROC curve in the evaluation of machine learning algorithms , 1997, Pattern Recognit..

[35]  Lothar Thiele,et al.  Multiobjective evolutionary algorithms: a comparative case study and the strength Pareto approach , 1999, IEEE Trans. Evol. Comput..

[36]  Ian H. Witten,et al.  Knowledge Visualization Techniques for Machine Learning , 1998, Intell. Data Anal..

[37]  Tom Fawcett,et al.  Using rule sets to maximize ROC performance , 2001, Proceedings 2001 IEEE International Conference on Data Mining.

[38]  F. Herrera,et al.  A proposal on reasoning methods in fuzzy rule-based classification systems , 1999 .

[39]  増山 繁,et al.  グラフの構造的特徴と効率の良い並列アルゴリズムについて (Algorithm Engineering as a New Paradigm) , 2001 .

[40]  J. Hanley,et al.  The meaning and use of the area under a receiver operating characteristic (ROC) curve. , 1982, Radiology.

[41]  D. C. Howell.  Statistical Methods for Psychology , 1987 .

[42]  Marco Laumanns,et al.  SPEA2: Improving the strength pareto evolutionary algorithm , 2001 .

[43]  A. Gray,et al.  I. THE ORIGIN OF SPECIES BY MEANS OF NATURAL SELECTION , 1963 .

[44]  Kevin Barraclough,et al.  I and i , 2001, BMJ : British Medical Journal.

[45]  Marco Laumanns,et al.  On the Effects of Archiving, Elitism, and Density Based Selection in Evolutionary Multi-objective Optimization , 2001, EMO.

[46]  Lawrence. Davis,et al.  Handbook Of Genetic Algorithms , 1990 .

[47]  Friedrich Steimann,et al.  Fuzzy set theory in medicine , 1997, Artif. Intell. Medicine.

[48]  Wei-Yin Loh,et al.  A Comparison of Prediction Accuracy, Complexity, and Training Time of Thirty-Three Old and New Classification Algorithms , 2000, Machine Learning.

[49]  Marco Laumanns,et al.  Archiving With Guaranteed Convergence And Diversity In Multi-objective Optimization , 2002, GECCO.

[50]  Ivan Bratko,et al.  Machine Learning and Data Mining; Methods and Applications , 1998 .

[51]  Kenneth Alan De Jong,et al.  An analysis of the behavior of a class of genetic adaptive systems. , 1975 .

[52]  Peter J. Bentley,et al.  "Evolutionary, my dear Watson" - Investigating Committee-based Evolution of Fuzzy Rules for the Detection of Suspicious Insurance Claims , 2000, GECCO.

[53]  J. Jaccard.  Interaction Effects in Factorial Analysis of Variance , 1997 .

[54]  Peter Nordin,et al.  Genetic programming - An Introduction: On the Automatic Evolution of Computer Programs and Its Applications , 1998 .

[55]  Zbigniew Michalewicz,et al.  Evolutionary algorithms , 1997, Emerging Evolutionary Algorithms for Antennas and Wireless Communications.

[56]  David G. Stork,et al.  Pattern Classification , 1973 .

[57]  Nichael Lynn Cramer,et al.  A Representation for the Adaptive Generation of Simple Sequential Programs , 1985, ICGA.

[58]  I. Wegener.  UNIVERSITY OF DORTMUND REIHE COMPUTATIONAL INTELLIGENCE COLLABORATIVE RESEARCH CENTER 531 Design and Management of Complex Technical Processes and Systems by means of Computational Intelligence Methods On the Design and Analysis of Evolutionary Algorithms , 2004 .