On Combined Classifiers, Rule Induction and Rough Sets

Problems of using elements of rough sets theory and rule induction to create efficient classifiers are discussed. In the last decade many researches attempted to increase a classification accuracy by combining several classifiers into integrated systems. The main aim of this paper is to summarize the author's own experience with applying one of his rule induction algorithm, called MODLEM, in the framework of different combined classifiers, namely, the bagging, n2-classifier and the combiner aggregation. We also discuss how rough approximations are applied in rule induction. The results of carried out experiments have shown that the MODLEM algorithm can be efficiently used within the framework of considered combined classifiers.

[1]  Daniel Vanderpooten,et al.  Induction of decision rules in classification and discovery-oriented perspectives , 2001, Int. J. Intell. Syst..

[2]  Jerzy Stefanowski,et al.  Experiments on Solving Multiclass Learning Problems by n2-classifier , 1998, ECML.

[3]  Z. Bo.  Variable precision rough sets model based on(α,τ) limited similarity relation , 2009 .

[4]  Salvatore J. Stolfo,et al.  On the Accuracy of Meta-learning for Scalable Data Mining , 2004, Journal of Intelligent Information Systems.

[5]  Robert Tibshirani,et al.  Classification by Pairwise Coupling , 1997, NIPS.

[6]  Sadaaki Miyamoto,et al.  Rough Sets and Current Trends in Computing , 2012, Lecture Notes in Computer Science.

[7]  Sinh Hoa Nguyen,et al.  Rough Set Approach to Sunspot Classification Problem , 2005, RSFDGrC.

[8]  Jerzy W. Grzymala-Busse,et al.  LERS-A System for Learning from Examples Based on Rough Sets , 1992, Intelligent Decision Support.

[9]  Céline Rouveirol,et al.  Machine Learning: ECML-98 , 1998, Lecture Notes in Computer Science.

[10]  Christopher J. Merz,et al.  Using Correspondence Analysis to Combine Classifiers , 1999, Machine Learning.

[11]  Roman Słowiński,et al.  Intelligent Decision Support , 1992, Theory and Decision Library.

[12]  Leo Breiman,et al.  Bagging Predictors , 1996, Machine Learning.

[13]  Janusz Zalewski,et al.  Rough sets: Theoretical aspects of reasoning about data , 1996 .

[14]  Jerzy Stefanowski,et al.  The Bagging and n2-Classifiers Based on Rules Induced by MODLEM , 2004, Rough Sets and Current Trends in Computing.

[15]  Jörg H. Siekmann,et al.  Artificial Intelligence and Soft Computing - ICAISC 2004 , 2004, Lecture Notes in Computer Science.

[16]  Arkadiusz Wojna,et al.  RIONA: A New Classification System Combining Rule Induction and Instance-Based Learning , 2002, Fundam. Informaticae.

[17]  Jerzy W. Grzymala-Busse,et al.  Classification Strategies Using Certain and Possible Rules , 1998, Rough Sets and Current Trends in Computing.

[18]  Jerzy Stefanowski,et al.  An Experimental Study of Methods Combining Multiple Classifiers-Diversified both by Feature Selection and Bootstrap Sampling , 2005 .

[19]  Andrzej Skowron,et al.  Rough Set Methods in Approximation of Hierarchical Concepts , 2004, Rough Sets and Current Trends in Computing.

[20]  Ludmila I. Kuncheva,et al.  Measures of Diversity in Classifier Ensembles and Their Relationship with the Ensemble Accuracy , 2003, Machine Learning.

[21]  Jerzy W. Grzymala-Busse,et al.  A Comparison of Two Approaches to Data Mining from Imbalanced Data , 2004, J. Intell. Manuf..

[22]  Jerzy Stefanowski,et al.  On rough set based approaches to induction of decision rules , 1998 .

[23]  J. Grzymala-Busse,et al.  Three discretization methods for rule induction , 2001 .

[24]  Jan M. Zytkow,et al.  Handbook of Data Mining and Knowledge Discovery , 2002 .

[25]  Marcin Szczuka.  Refining classifiers with neural networks , 2001 .

[26]  Jiawei Han,et al.  Data Mining: Concepts and Techniques , 2000 .

[27]  Jerzy Stefanowski.  Classification Support Based on the Rough Sets Theory , 1993 .

[28]  Jerzy W. Grzymala-Busse,et al.  Rough sets : New horizons in commercial and industrial AI , 1995 .

[29]  Dominik Slezak,et al.  Approximate Entropy Reducts , 2002, Fundam. Informaticae.

[30]  Jan G. Bazan.  Chapter 17 a Comparison of Dynamic and Non{dynamic Rough Set Methods for Extracting Laws from Decision Tables , 1998 .

[31]  Andrzej Skowron,et al.  The Discernibility Matrices and Functions in Information Systems , 1992, Intelligent Decision Support.

[32]  Zbigniew W. Ras,et al.  Methodologies for Intelligent Systems , 1991, Lecture Notes in Computer Science.

[33]  Lakhmi C. Jain,et al.  Knowledge-Based Intelligent Information and Engineering Systems , 2004, Lecture Notes in Computer Science.

[34]  Giorgio Valentini,et al.  Ensembles of Learning Machines , 2002, WIRN.

[35]  Ivo Düntsch,et al.  Classificatory filtering in decision systems , 2000, Int. J. Approx. Reason..

[36]  Armando Freitas da Rocha,et al.  Neural Nets , 1992, Lecture Notes in Computer Science.

[37]  Salvatore Greco,et al.  Variable Consistency Model of Dominance-Based Rough Sets Approach , 2000, Rough Sets and Current Trends in Computing.

[38]  Sinh Hoa Nguyen,et al.  Rough Set Approach to Sunspot Classification Problem and Granular Computing, 10th International Conference; RSFDGrC 2005, Regina, Canada, August 31-September 3, 2005, Proceedings, Part II , 2005 .

[39]  Wojciech Ziarko,et al.  Variable Precision Rough Set Model , 1993, J. Comput. Syst. Sci..

[40]  R. Słowiński.  Intelligent Decision Support: Handbook of Applications and Advances of the Rough Sets Theory , 1992 .

[41]  Thomas G. Dietterich.  Multiple Classifier Systems , 2000, Lecture Notes in Computer Science.

[42]  Salvatore Greco,et al.  Inducing Robust Decision Rules from Rough Approximations of a Preference Relation , 2004, ICAISC.

[43]  Szymon Wilk,et al.  Evaluating business credit risk by means of approach-integrating decision rules and case-based learning , 2001, Intell. Syst. Account. Finance Manag..

[44]  Andrzej Skowron,et al.  Rough Sets: A Tutorial , 1998 .

[45]  Andrzej Skowron,et al.  Hyperrelations in version space , 2002, SAC '02.

[46]  Roman Słowiński,et al.  The Use of Rough Sets and Fuzzy Sets in MCDM , 1999 .

[47]  Jerzy Stefanowski,et al.  On using rule induction in multiple classifiers with a combiner aggregation strategy , 2005, 5th International Conference on Intelligent Systems Design and Applications (ISDA'05).

[48]  Andrzej Skowron,et al.  Boolean Reasoning for Decision Rules Generation , 1993, ISMIS.