Optimization of Decision Rules Based on Dynamic Programming Approach

This chapter is devoted to the study of an extension of dynamic programming approach which allows optimization of approximate decision rules relative to the length and coverage. We introduce an uncertainty measure that is the difference between number of rows in a given decision table and the number of rows labeled with the most common decision for this table divided by the number of rows in the decision table. We fix a threshold γ, such that 0 ≤ γ < 1, and study so-called γ-decision rules (approximate decision rules) that localize rows in subtables which uncertainty is at most γ. Presented algorithm constructs a directed acyclic graph Δγ T which nodes are subtables of the decision table T given by pairs “attribute = value”. The algorithm finishes the partitioning of a subtable when its uncertainty is at most γ. The chapter contains also results of experiments with decision tables from UCI Machine Learning Repository.

[1]  Shahid Hussain,et al.  Dagger: A Tool for Analysis and Optimization of Decision Trees and Rules , 2011 .

[2]  Peter Clark,et al.  The CN2 Induction Algorithm , 1989, Machine Learning.

[3]  Igor Chikalov,et al.  On Algorithm for Building of Optimal alpha-Decision Trees , 2010, RSCTC.

[4]  Roman Słowiński,et al.  Sequential covering rule induction algorithm for variable consistency rough set approaches , 2011, Inf. Sci..

[5]  Jerzy W. Grzymala-Busse,et al.  LERS-A System for Learning from Examples Based on Rough Sets , 1992, Intelligent Decision Support.

[6]  Hung Son Nguyen,et al.  Approximate Boolean Reasoning: Foundations and Applications in Data Mining , 2006, Trans. Rough Sets.

[7]  J. Rissanen,et al.  Modeling By Shortest Data Description* , 1978, Autom..

[8]  Roman Słowiński,et al.  Intelligent Decision Support , 1992, Theory and Decision Library.

[9]  Abdullah Al Mamun,et al.  An evolutionary memetic algorithm for rule extraction , 2010, Expert Syst. Appl..

[10]  J. Ross Quinlan,et al.  C4.5: Programs for Machine Learning , 1992 .

[11]  Mikhail Ju. Moshkov,et al.  On Partial Covers, Reducts and Decision Rules , 2008, Trans. Rough Sets.

[12]  Andrzej Skowron,et al.  Rough sets and Boolean reasoning , 2007, Inf. Sci..

[13]  Hussein A. Abbass,et al.  Classification rule discovery with ant colony optimization , 2003, IEEE/WIC International Conference on Intelligent Agent Technology, 2003. IAT 2003..

[14]  Igor Chikalov,et al.  Dynamic programming approach to optimization of approximate decision rules , 2013, Inf. Sci..

[15]  Igor Chikalov,et al.  Dynamic Programming Approach for Exact Decision Rule Optimization , 2013, Rough Sets and Intelligent Systems.

[16]  Igor Chikalov On Algorithm for Constructing of Decision Trees with Minimal Number of Nodes , 2000, Rough Sets and Current Trends in Computing.

[17]  Ramakrishnan Srikant,et al.  Fast Algorithms for Mining Association Rules in Large Databases , 1994, VLDB.

[18]  JOHANNES FÜRNKRANZ,et al.  Separate-and-Conquer Rule Learning , 1999, Artificial Intelligence Review.

[19]  Beata Zielosko,et al.  Sequential optimization of γ-decision rules , 2012, 2012 Federated Conference on Computer Science and Information Systems (FedCSIS).

[20]  Wojciech Kotlowski,et al.  ENDER: a statistical framework for boosting decision rules , 2010, Data Mining and Knowledge Discovery.

[21]  Andrzej Skowron,et al.  The Discernibility Matrices and Functions in Information Systems , 1992, Intelligent Decision Support.

[22]  Igor Chikalov,et al.  Dynamic Programming Approach for Partial Decision Rule Optimization , 2012, Fundam. Informaticae.

[23]  Kim Veltman,et al.  Computational informatics, social factors and new information technologies: Hypermedia perspectives and avant-garde : experiences in the era of communicability expansion , 2011 .

[24]  Igor Chikalov,et al.  On Algorithm for Constructing of Decision Trees with Minimal Depth , 2000, Fundam. Informaticae.

[25]  Dominik Slezak,et al.  Order Based Genetic Algorithms for the Search of Approximate Entropy Reducts , 2003, RSFDGrC.

[26]  Mikhail Ju. Moshkov,et al.  Combinatorial Machine Learning - A Rough Set Approach , 2011, Studies in Computational Intelligence.

[27]  Shahid Hussain,et al.  Extensions of Dynamic Programming as a New Tool for Decision Tree Optimization , 2013 .