Extending Rule-Based Classifiers to Improve Recognition of Imbalanced Classes

Knowledge discovery in general, and data mining in particular, have received a growing interest both from research and industry in recent years. Its main aim is to look for previously unknown relationships or patterns representing knowledge hidden in real-life data sets [16]. The typical representations of knowledge discovered from data are: associations, trees or rules, relational logic clauses, functions, clusters or taxonomies, or characteristic descriptions of concepts [16, 29, 21]. In this paper we focus on the rule-based representation. More precisely, we are interested in decision or classification rules that are considered in classification problems. In data mining other types of rules are also considered, e.g., association rules or action rules [16, 29, 34], however, in the text hereafter we will use the general term “rules” to refer specifically to decision rules.

[1]  Gary M. Weiss.  Mining with rarity: a unifying framework , 2004, SKDD.

[2]  Szymon Wilk,et al.  Evaluating business credit risk by means of approach-integrating decision rules and case-based learning , 2001, Intell. Syst. Account. Finance Manag..

[3]  Ryszard S. Michalski,et al.  A theory and methodology of inductive learning , 1993 .

[4]  Daniel Vanderpooten,et al.  Induction of decision rules in classification and discovery-oriented perspectives , 2001, Int. J. Intell. Syst..

[5]  R. Suganya,et al.  Data Mining Concepts and Techniques , 2010 .

[6]  Nitesh V. Chawla,et al.  Data Mining for Imbalanced Datasets: An Overview , 2005, The Data Mining and Knowledge Discovery Handbook.

[7]  Shusaku Tsumoto,et al.  Foundations of Intelligent Systems, 15th International Symposium, ISMIS 2005, Saratoga Springs, NY, USA, May 25-28, 2005, Proceedings , 2005, ISMIS.

[8]  Jerzy W. Grzymala-Busse,et al.  An Approach to Imbalanced Data Sets Based on Changing Rule Strength , 2004, Rough-Neural Computing: Techniques for Computing with Words.

[9]  Andrzej Skowron,et al.  Boolean Reasoning for Decision Rules Generation , 1993, ISMIS.

[10]  William W. Cohen.  Fast Effective Rule Induction , 1995, ICML.

[11]  Wojtek Michalowski,et al.  Supporting triage of children with abdominal pain in the emergency room , 2005, Eur. J. Oper. Res..

[12]  Jerzy W. Grzymala-Busse,et al.  A Comparison of Two Approaches to Data Mining from Imbalanced Data , 2004, J. Intell. Manuf..

[13]  Marcel Holsheimer,et al.  Data Surveyor: Searching the Nuggets in Parallel , 1996, Advances in Knowledge Discovery and Data Mining.

[14]  Jiawei Han,et al.  Data Mining: Concepts and Techniques , 2000 .

[15]  C.J.H. Mann,et al.  Handbook of Data Mining and Knowledge Discovery , 2004 .

[16]  Jerzy Stefanowski,et al.  On Combined Classifiers, Rule Induction and Rough Sets , 2007, Trans. Rough Sets.

[17]  Nathalie Japkowicz,et al.  The class imbalance problem: A systematic study , 2002, Intell. Data Anal..

[18]  W. Michalowski,et al.  Development of a Decision Algorithm to Support Emergency Triage of Scrotal Pain and its Implementation in the met system , 2005 .

[19]  Jerzy W. Grzymala-Busse,et al.  Rough Sets , 1995, Commun. ACM.

[20]  Jerzy W. Grzymala-Busse,et al.  LERS-A System for Learning from Examples Based on Rough Sets , 1992, Intelligent Decision Support.

[21]  Janusz Zalewski,et al.  Rough sets: Theoretical aspects of reasoning about data , 1996 .

[22]  Jerzy Stefanowski,et al.  Handling Continuous Attributes in Discovery of Strong Decision Rules , 1998, Rough Sets and Current Trends in Computing.

[23]  Jerzy W. Grzymala-Busse,et al.  A Comparison of Two Approaches to Data Mining from Imbalanced Data , 2004, J. Intell. Manuf..

[24]  JapkowiczNathalie,et al.  The class imbalance problem: A systematic study , 2002 .

[25]  Jerzy W. Grzymala-Busse,et al.  Transactions on Rough Sets VI, Commemorating the Life and Work of Zdzislaw Pawlak, Part I , 2007, Trans. Rough Sets.

[26]  John R. Anderson,et al.  MACHINE LEARNING An Artificial Intelligence Approach , 2009 .

[27]  Thomas G. Dietterich.  What is machine learning? , 2020, Archives of Disease in Childhood.

[28]  J. Palous,et al.  Machine Learning and Data Mining , 2002 .

[29]  Sašo Džeroski,et al.  Using the m -estimate in rule induction , 1993 .

[30]  J. Stefanowski,et al.  Improving Rule-Based Classifiers Induced by MODLEM by Selective Pre-processing of Imbalanced Data , 2007 .

[31]  Peter Clark,et al.  The CN2 induction algorithm , 2004, Machine Learning.

[32]  Nathalie Japkowicz,et al.  Boosting support vector machines for imbalanced data sets , 2008, Knowledge and Information Systems.

[33]  Szymon Wilk,et al.  Rough Sets for Handling Imbalanced Data: Combining Filtering and Rule-based Classifiers , 2006, Fundam. Informaticae.

[34]  Thorsten Kuhlmann,et al.  Intelligent decision support , 1998 .

[35]  Jorma Laurikkala,et al.  Improving Identification of Difficult Small Classes by Balancing Class Distribution , 2001, AIME.

[36]  Wynne Hsu,et al.  Integrating Classification and Association Rule Mining , 1998, KDD.

[37]  JOHANNES FÜRNKRANZ,et al.  Separate-and-Conquer Rule Learning , 1999, Artificial Intelligence Review.

[38]  Howard J. Hamilton,et al.  Knowledge discovery and measures of interest , 2001 .

[39]  Jan Komorowski,et al.  Principles of Data Mining and Knowledge Discovery , 2001, Lecture Notes in Computer Science.

[40]  Stan Matwin,et al.  Addressing the Curse of Imbalanced Training Sets: One-Sided Selection , 1997, ICML.

[41]  Lior Rokach,et al.  Data Mining And Knowledge Discovery Handbook , 2005 .

[42]  Gustavo E. A. P. A. Batista,et al.  A study of the behavior of several methods for balancing machine learning training data , 2004, SKDD.

[43]  Zbigniew W. Ras,et al.  Methodologies for Intelligent Systems , 1991, Lecture Notes in Computer Science.

[44]  Lakhmi C. Jain,et al.  Knowledge-Based Intelligent Information and Engineering Systems , 2004, Lecture Notes in Computer Science.

[45]  Taghi M. Khoshgoftaar,et al.  Experimental perspectives on learning from imbalanced data , 2007, ICML '07.

[46]  Yoram Singer,et al.  A simple, fast, and effective rule learner , 1999, AAAI 1999.

[47]  Aiko M. Hormann,et al.  Programs for Machine Learning. Part I , 1962, Inf. Control..

[48]  David D. Lewis,et al.  Heterogeneous Uncertainty Sampling for Supervised Learning , 1994, ICML.

[49]  J. Ross Quinlan,et al.  C4.5: Programs for Machine Learning , 1992 .

[50]  Tom M. Mitchell,et al.  Machine Learning and Data Mining , 2012 .

[51]  Oren Etzioni,et al.  Representation design and brute-force induction in a Boeing manufacturing domain , 1994, Appl. Artif. Intell..

[52]  Szymon Wilk,et al.  Selective Pre-processing of Imbalanced Data for Improving Classification Performance , 2008, DaWaK.

[53]  Sadaaki Miyamoto,et al.  Rough Sets and Current Trends in Computing , 2012, Lecture Notes in Computer Science.

[54]  Zbigniew W. Ras,et al.  Action-Rules: How to Increase Profit of a Company , 2000, PKDD.

[55]  Johannes Fürnkranz,et al.  Pruning Algorithms for Rule Learning , 1997, Machine Learning.

[56]  Nitesh V. Chawla,et al.  SMOTE: Synthetic Minority Over-sampling Technique , 2002, J. Artif. Intell. Res..

[57]  N. Japkowicz.  Learning from Imbalanced Data Sets: A Comparison of Various Strategies * , 2000 .

[58]  Jerzy W. Grzymala-Busse,et al.  Global discretization of continuous attributes as preprocessing for machine learning , 1996, Int. J. Approx. Reason..