Entropy-Based Greedy Algorithm for Decision Trees Using Hypotheses

In this paper, we consider decision trees that use both conventional queries based on one attribute each and queries based on hypotheses of values of all attributes. Such decision trees are similar to those studied in exact learning, where membership and equivalence queries are allowed. We present greedy algorithm based on entropy for the construction of the above decision trees and discuss the results of computer experiments on various data sets and randomly generated Boolean functions.

[1]  Alberto Maria Segre,et al.  Programs for Machine Learning , 1994 .

[2]  Andrzej Skowron,et al.  Rudiments of rough sets , 2007, Inf. Sci..

[3]  Yingqian Zhang,et al.  Learning Optimal Classification Trees Using a Binary Linear Program Formulation , 2019, BNAIC/BENELEARN.

[4]  Lior Rokach,et al.  Top-down induction of decision trees classifiers - a survey , 2005, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[5]  Mikhail Moshkov,et al.  Time Complexity of Decision Trees , 2005, Trans. Rough Sets.

[6]  Mohammad Azad,et al.  Minimizing Depth of Decision Trees with Hypotheses , 2021, IJCRS.

[7]  A. Asuncion,et al.  UCI Machine Learning Repository, University of California, Irvine, School of Information and Computer Sciences , 2007 .

[8]  Fawaz Alsolami,et al.  Decision and Inhibitory Trees and Rules for Decision Tables with Many-valued Decisions , 2019, Intelligent Systems Reference Library.

[9]  Andreas Holzinger,et al.  Data Mining with Decision Trees: Theory and Applications , 2015, Online Inf. Rev..

[10]  Igor Chikalov,et al.  Extensions of Dynamic Programming for Combinatorial Optimization and Data Mining , 2018, Intelligent Systems Reference Library.

[11]  Wei-Yin Loh,et al.  Classification and regression trees , 2011, WIREs Data Mining Knowl. Discov..

[12]  Dana Angluin,et al.  Queries and concept learning , 1988, Machine Learning.

[13]  J. Ross Quinlan,et al.  Induction of Decision Trees , 1986, Machine Learning.

[14]  Dana Angluin.  Queries revisited , 2004, Theor. Comput. Sci..

[15]  Mohammad Azad,et al.  Minimizing Number of Nodes in Decision Trees with Hypotheses , 2021, KES.

[16]  Pierre Schaus,et al.  Learning Optimal Decision Trees Using Caching Branch-and-Bound Search , 2020, AAAI.

[17]  Joao Marques-Silva,et al.  Learning Optimal Decision Trees with SAT , 2018, IJCAI.

[18]  Janusz Zalewski,et al.  Rough sets: Theoretical aspects of reasoning about data , 1996 .

[19]  Leslie G. Valiant,et al.  A theory of the learnable , 1984, STOC '84.

[20]  Theresa Beaubouef,et al.  Rough Sets , 2019, Lecture Notes in Computer Science.