Micro and macro evaluation of classification rules

Rule evaluation plays an important role in the rule learning and classification process. Many existing rule inductive learning algorithms are based on single rule evaluation measures. However, the overall rule induction system performance and the classification process are involved with the evaluation of a set of rules. This brings the needs for studying the connections between single rule and rule set evaluation measures. The main objective of this paper is to introduce a general framework of classification rule evaluation which connects two types of evaluations, called micro and macro evaluation. Micro evaluation is based on single rules which can be measured by the common empirical measures. Macro evaluation is based on rule sets, depending on the relationships between rules in the set, different resolutions can be applied. By analyzing the relationships between these two types of evaluations, we suggest that under certain conditions, macro evaluation measures can be explicitly expressed by micro evaluation measures.

[1]  Vasant Dhar,et al.  Abstract-Driven Pattern Discovery in Databases , 1992, IEEE Trans. Knowl. Data Eng..

[2]  Nick Cercone,et al.  Rule Quality Measures for Rule Induction Systems: Description and Evaluation , 2001, Comput. Intell..

[3]  Hiroshi Tanaka,et al.  Automated Discovery of Functional Components of Proteins from Amino-Acid Sequences Based on Rough Sets and Change of Representation , 1995, KDD.

[4]  Gregory Piatetsky-Shapiro,et al.  Advances in Knowledge Discovery and Data Mining , 2004, Lecture Notes in Computer Science.

[5]  Craig A. Kaplan,et al.  Foundations of cognitive science , 1989 .

[6]  Jinyan Li,et al.  CAEP: Classification by Aggregating Emerging Patterns , 1999, Discovery Science.

[7]  Victor J. Rayward-Smith,et al.  Discovering Knowledge in Commercial Databases Using Modern Heuristic Techniques , 1996, KDD.

[8]  Yiyu Yao,et al.  An Analysis of Quantitative Measures Associated with Rules , 1999, PAKDD.

[9]  Carlos Bento,et al.  A Metric for Selection of the Most Promising Rules , 1998, PKDD.

[10]  Wynne Hsu,et al.  Using General Impressions to Analyze Discovered Classification Rules , 1997, KDD.

[11]  Howard J. Hamilton,et al.  Interestingness measures for data mining: A survey , 2006, CSUR.

[12]  Willi Klösgen,et al.  Explora: A Multipattern and Multistrategy Discovery Assistant , 1996, Advances in Knowledge Discovery and Data Mining.

[13]  Thomas G. Dietterich.  What is machine learning? , 2020, Archives of Disease in Childhood.

[14]  Ke Wang,et al.  Growing decision trees on support-less association rules , 2000, KDD '00.

[15]  Yingxu Wang.  On Cognitive Informatics , 2003 .

[16]  Yingxu Wang.  Cognitive Informatics: A New Transdisciplinary Research Field , 2003 .

[17]  Wei-Min Shen,et al.  Metapattern Generation for Integrated Data Mining , 1996, KDD.

[18]  Wynne Hsu,et al.  Integrating Classification and Association Rule Mining , 1998, KDD.

[19]  J. Ross Quinlan,et al.  Learning Efficient Classification Procedures and Their Application to Chess End Games , 1983 .

[20]  Gregory Piatetsky-Shapiro,et al.  Discovery, Analysis, and Presentation of Strong Rules , 1991, Knowledge Discovery in Databases.

[21]  Peter A. Flach,et al.  Rule Evaluation Measures: A Unifying View , 1999, ILP.

[22]  Yingxu Wang,et al.  On Cognitive Informatics , 2002, Proceedings First IEEE International Conference on Cognitive Informatics.

[23]  Alex Alves Freitas,et al.  On rule interestingness measures , 1999, Knowl. Based Syst..

[24]  Jian Pei,et al.  CMAR: accurate and efficient classification based on multiple class-association rules , 2001, Proceedings 2001 IEEE International Conference on Data Mining.