Subgroup Discovery with CN2-SD

This paper investigates how to adapt standard classification rule learning approaches to subgroup discovery. The goal of subgroup discovery is to find rules describing subsets of the population that are sufficiently large and statistically unusual. The paper presents a subgroup discovery algorithm, CN2-SD, developed by modifying parts of the CN2 classification rule learner: its covering algorithm, search heuristic, probabilistic classification of instances, and evaluation measures. Experimental evaluation of CN2-SD on 23 UCI data sets shows substantial reduction of the number of induced rules, increased rule coverage and rule significance, as well as slight improvements in terms of the area under ROC curve, when compared with the CN2 algorithm. Application of CN2-SD to a large traffic accident data set confirms these findings.

[1]  Yoram Singer,et al.  A simple, fast, and effective rule learner , 1999, AAAI 1999.

[2]  Nada Lavrac,et al.  Expert-Guided Subgroup Discovery: Methodology and Application , 2011, J. Artif. Intell. Res..

[3]  Luc De Raedt,et al.  Clausal Discovery , 1997, Machine Learning.

[4]  L. D. Raedt,et al.  Three companions for data mining in first order logic , 2001 .

[5]  Peter Ljubi,et al.  TIME-SERIES ANALYSIS OF UK TRAFFIC ACCIDENT DATA , 2002 .

[6]  Ian Witten,et al.  Data Mining , 2000 .

[7]  Sašo Džeroski,et al.  Using the m -estimate in rule induction , 1993 .

[8]  Ryszard S. Michalski,et al.  Pattern Recognition as Rule-Guided Inductive Inference , 1980, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[9]  Peter A. Flach,et al.  Rule Evaluation Measures: A Unifying View , 1999, ILP.

[10]  Nada Lavrac,et al.  Classification Rule Learning with APRIORI-C , 2001, EPIA.

[11]  Branko Kavsek,et al.  APRIORI-SD: ADAPTING ASSOCIATION RULE LEARNING TO SUBGROUP DISCOVERY , 2006, IDA.

[12]  Arno J. Knobbe,et al.  Propositionalisation and Aggregates , 2001, PKDD.

[13]  Abraham Silberschatz,et al.  On Subjective Measures of Interestingness in Knowledge Discovery , 1995, KDD.

[14]  Peter A. Flach.  The Geometry of ROC Space: Understanding Machine Learning Metrics through ROC Isometrics , 2003, ICML.

[15]  Willi Klösgen,et al.  Explora: A Multipattern and Multistrategy Discovery Assistant , 1996, Advances in Knowledge Discovery and Data Mining.

[16]  Peter A. Flach,et al.  Predictive Performance of Weghted Relative Accuracy , 2000, PKDD.

[17]  Nicholas I. Fisher,et al.  Bump hunting in high-dimensional data , 1999, Stat. Comput..

[18]  N. Lavra,et al.  Predictive Performance of Weighted Relative Accuracy , 2000 .

[19]  Bojan Cestnik,et al.  Estimating Probabilities: A Crucial Task in Machine Learning , 1990, ECAI.

[20]  Ronald L. Rivest,et al.  Learning decision lists , 2004, Machine Learning.

[21]  Johannes Fürnkranz,et al.  An Analysis of Rule Evaluation Metrics , 2003, ICML.

[22]  Stefan Wrobel,et al.  An Algorithm for Multi-relational Discovery of Subgroups , 1997, PKDD.

[23]  Yoram Singer,et al.  Improved Boosting Algorithms Using Confidence-rated Predictions , 1998, COLT' 98.

[24]  Peter A. Flach,et al.  RSD: Relational Subgroup Discovery through First-Order Feature Construction , 2002, ILP.

[25]  K. Chidananda Gowda,et al.  Symbolic clustering using a new similarity measure , 1992, IEEE Trans. Syst. Man Cybern..

[26]  Heikki Mannila,et al.  Fast Discovery of Association Rules , 1996, Advances in Knowledge Discovery and Data Mining.

[27]  Peter A. Flach,et al.  Adapting classification rule induction to subgroup discovery , 2002, 2002 IEEE International Conference on Data Mining, 2002. Proceedings..

[28]  Peter Clark,et al.  The CN2 Induction Algorithm , 1989, Machine Learning.

[29]  David J. Hand,et al.  A Simple Generalisation of the Area Under the ROC Curve for Multiple Class Classification Problems , 2001, Machine Learning.

[30]  Nada Lavrac,et al.  The Multi-Purpose Incremental Learning System AQ15 and Its Testing Application to Three Medical Domains , 1986, AAAI.

[31]  H. Ross.  Principles of Numerical Taxonomy , 1964 .

[32]  Oren Etzioni,et al.  A Redundant Covering Algorithm Applied to Text Classification , 1998 .

[33]  Bruce G. Buchanan,et al.  Knowledge-Based Learning in Exploratory Science: Learning Rules to Predict Rodent Carcinogenicity , 1998, Machine Learning.

[34]  Tom Fawcett,et al.  Robust Classification for Imprecise Environments , 2000, Machine Learning.

[35]  Arno J. Knobbe,et al.  Involving Aggregate Functions in Multi-relational Search , 2002, PKDD.

[36]  Yoram Singer,et al.  Improved Boosting Algorithms Using Confidence-rated Predictions , 1998, COLT' 98.

[37]  Ian H. Witten,et al.  Data mining: practical machine learning tools and techniques with Java implementations , 2002, SGMD.

[38]  William W. Cohen.  Fast Effective Rule Induction , 1995, ICML.

[39]  Peter A. Flach,et al.  Database Dependency Discovery: A Machine Learning Approach , 1999, AI Commun..

[40]  Edwin Diday,et al.  Symbolic clustering using a new dissimilarity measure , 1991, Pattern Recognit..

[41]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[42]  Peter A. Flach,et al.  Learning Decision Trees Using the Area Under the ROC Curve , 2002, ICML.

[43]  Peter Clark,et al.  Rule Induction with CN2: Some Recent Improvements , 1991, EWSL.

[44]  Yoav Freund,et al.  Experiments with a New Boosting Algorithm , 1996, ICML.

[45]  Luc De Raedt,et al.  Using Logical Decision Trees for Clustering , 1997, ILP.

[46]  Pat Langley,et al.  Elements of Machine Learning , 1995 .

[47]  Stefan Wrobel,et al.  Inductive Logic Programming for Knowledge Discovery in Databases , 2001 .