Decision Rules Derived from Optimal Decision Trees with Hypotheses

Conventional decision trees use queries each of which is based on one attribute. In this study, we also examine decision trees that handle additional queries based on hypotheses. This kind of query is similar to the equivalence queries considered in exact learning. Earlier, we designed dynamic programming algorithms for the computation of the minimum depth and the minimum number of internal nodes in decision trees that have hypotheses. Modification of these algorithms considered in the present paper permits us to build decision trees with hypotheses that are optimal relative to the depth or relative to the number of the internal nodes. We compare the length and coverage of decision rules extracted from optimal decision trees with hypotheses and decision rules extracted from optimal conventional decision trees to choose the ones that are preferable as a tool for the representation of information. To this end, we conduct computer experiments on various decision tables from the UCI Machine Learning Repository. In addition, we also consider decision tables for randomly generated Boolean functions. The collected results show that the decision rules derived from decision trees with hypotheses in many cases are better than the rules extracted from conventional decision trees.

[1]  Dana Angluin.  Queries revisited , 2004, Theor. Comput. Sci..

[2]  Mohammad Azad,et al.  Minimizing Depth of Decision Trees with Hypotheses , 2021, IJCRS.

[3]  A. Asuncion,et al.  UCI Machine Learning Repository, University of California, Irvine, School of Information and Computer Sciences , 2007 .

[4]  川野 秀一.  An Introduction to Statistical Learning (with Applications in R), Gareth James,Daniela Witten,Trevor Hastie and Robert Tibshirani著, Springer, 2013年8月, 430pp., 価格 59.99〓, ISBN 978-1-4614-7137-0 , 2014 .

[5]  Mohammad Azad,et al.  Entropy-Based Greedy Algorithm for Decision Trees Using Hypotheses , 2021, Entropy.

[6]  Janusz Zalewski,et al.  Rough sets: Theoretical aspects of reasoning about data , 1996 .

[7]  Mohammad Azad,et al.  Optimization of Decision Trees with Hypotheses for Knowledge Representation , 2021, Electronics.

[8]  Lakhmi C. Jain,et al.  Intelligent Systems Reference Library , 2015 .

[9]  Dana Angluin,et al.  Queries and concept learning , 1988, Machine Learning.

[10]  Andreas Holzinger,et al.  Data Mining with Decision Trees: Theory and Applications , 2015, Online Inf. Rev..

[11]  Mohammad Azad,et al.  Minimizing Number of Nodes in Decision Trees with Hypotheses , 2021, KES.

[12]  Dana Angluin,et al.  Learning Regular Sets from Queries and Counterexamples , 1987, Inf. Comput..

[13]  Andrzej Skowron,et al.  Rudiments of rough sets , 2007, Inf. Sci..

[14]  Lior Rokach,et al.  Data Mining with Decision Trees - Theory and Applications , 2007, Series in Machine Perception and Artificial Intelligence.

[15]  Theresa Beaubouef,et al.  Rough Sets , 2019, Lecture Notes in Computer Science.

[16]  Igor Chikalov,et al.  Extensions of Dynamic Programming for Combinatorial Optimization and Data Mining , 2018, Intelligent Systems Reference Library.

[17]  Wei-Yin Loh,et al.  Classification and regression trees , 2011, WIREs Data Mining Knowl. Discov..

[18]  Igor Chikalov,et al.  Dynamic Programming Approach for Exact Decision Rule Optimization , 2013, Rough Sets and Intelligent Systems.

[19]  Mikhail Moshkov,et al.  Time Complexity of Decision Trees , 2005, Trans. Rough Sets.