Searching for the optimal ordering of classes in rule induction

Rule induction algorithms such as Ripper, solve a K > 2 class problem by converting it into a sequence of K - 1 two-class problems. As a usual heuristic, the classes are fed into the algorithm in the order of increasing prior probabilities. In this paper, we propose two algorithms to improve this heuristic. The first algorithm starts with the ordering the heuristic provides and searches for better orderings by swapping consecutive classes. The second algorithm transforms the ordering search problem into an optimization problem and uses the solution of the optimization problem to extract the optimal ordering. We compared our algorithms with the original Ripper on 8 datasets from UCI repository [2]. Simulation results show that our algorithms produce rulesets that are significantly better than those produced by Ripper proper.

[1]  K. Sumangala,et al.  Comparative Study on Bio-inspired Approach for Soil Classification , 2012 .

[2]  Johannes Fürnkranz,et al.  Incremental Reduced Error Pruning , 1994, ICML.

[3]  Alex Alves Freitas,et al.  Data mining with an ant colony optimization algorithm , 2002, IEEE Trans. Evol. Comput..

[4]  David Haussler,et al.  Quantifying Inductive Bias: AI Learning Algorithms and Valiant's Learning Framework , 1988, Artif. Intell..

[5]  Gilles Venturini,et al.  SIA: A Supervised Inductive Algorithm with Genetic Search for Learning Attributes based Concepts , 1993, ECML.

[6]  Ben Coppin,et al.  Artificial Intelligence Illuminated , 2004 .

[7]  J. Ross Quinlan,et al.  C4.5: Programs for Machine Learning , 1992 .

[8]  William W. Cohen.  Efficient Pruning Methods for Separate-and-Conquer Rule Learning Systems , 1993, IJCAI.

[9]  Michael J. Pazzani,et al.  An Investigation of Noise-Tolerant Relational Concept Learning Algorithms , 1991, ML.

[10]  J. Ross Quinlan,et al.  Learning logical definitions from relations , 1990, Machine Learning.

[11]  Nada Lavrac,et al.  The Multi-Purpose Incremental Learning System AQ15 and Its Testing Application to Three Medical Domains , 1986, AAAI.

[12]  H. Theron,et al.  BEXA: A Covering Algorithm for Learning Propositional Concept Descriptions , 1996, Machine Learning.

[13]  Yi-Chung Hu,et al.  Finding fuzzy classification rules using data mining techniques , 2003, Pattern Recognit. Lett..

[14]  Sholom M. Weiss,et al.  Reduced Complexity Rule Induction , 1991, IJCAI.

[15]  David J. Hand,et al.  Intelligent Data Analysis: An Introduction , 2005 .

[16]  Thomas G. Dietterich.  Approximate Statistical Tests for Comparing Supervised Classification Learning Algorithms , 1998, Neural Computation.

[17]  Peter Clark,et al.  Rule Induction with CN2: Some Recent Improvements , 1991, EWSL.

[18]  JOHANNES FÜRNKRANZ,et al.  Separate-and-Conquer Rule Learning , 1999, Artificial Intelligence Review.

[19]  Dieter Fensel,et al.  Refinement of Rule Sets with JoJo , 1993, ECML.

[20]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[21]  Rajen B. Bhatt,et al.  FRCT: fuzzy-rough classification trees , 2007, Pattern Analysis and Applications.

[22]  S. Matwin,et al.  Learning Two-Tiered Descriptions of Flexible Concepts: The POSEIDON System , 1992, Machine Learning.

[23]  Myeong-Kwan Kevin Cheon,et al.  Frank and I , 2012 .

[24]  J. Ross Quinlan,et al.  Induction of Decision Trees , 1986, Machine Learning.

[25]  Alex A. Freitas,et al.  An ant colony based system for data mining: applications to medical data , 2001 .

[26]  Alberto Maria Segre,et al.  Programs for Machine Learning , 1994 .

[27]  Monique Snoeck,et al.  Classification With Ant Colony Optimization , 2007, IEEE Transactions on Evolutionary Computation.

[28]  Jadzia Cendrowska,et al.  PRISM: An Algorithm for Inducing Modular Rules , 1987, Int. J. Man Mach. Stud..

[29]  Sebastián Ventura,et al.  Using Ant Programming Guided by Grammar for Building Rule-Based Classifiers , 2011, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[30]  Igor Kononenko,et al.  Learning as Optimization: Stochastic Generation of Multiple Knowledge , 1992, ML.

[31]  S. P. Yip.  Function finding in classification learning , 2006 .

[32]  William W. Cohen.  Fast Effective Rule Induction , 1995, ICML.

[33]  John B. Shoven,et al.  I , Edinburgh Medical and Surgical Journal.

[34]  Ian H. Witten,et al.  Generating Accurate Rule Sets Without Global Optimization , 1998, ICML.

[35]  Johannes Fürnkranz,et al.  Pruning Algorithms for Rule Learning , 1997, Machine Learning.

[36]  Ethem Alpaydin,et al.  Introduction to machine learning , 2004, Adaptive computation and machine learning.

[37]  Peter Clark,et al.  The CN2 Induction Algorithm , 1989, Machine Learning.

[38]  Geoffrey I. Webb.  Learning Decision Lists by Prepending Inferred Rules , 2005 .

[39]  Prasad Tadepalli,et al.  Learning Decision Rules by Randomized Iterative Local Search , 2002, ICML.