Time and space complexity of deterministic and nondeterministic decision trees

In this paper, we study arbitrary infinite binary information systems each of which consists of an infinite set called universe and an infinite set of two-valued functions (attributes) defined on the universe. We consider the notion of a problem over information system which is described by a finite number of attributes and a mapping corresponding a decision to each tuple of attribute values. As algorithms for problem solving, we use deterministic and nondeterministic decision trees. As time and space complexity, we study the depth and the number of nodes in the decision trees. In the worst case, with the growth of the number of attributes in the problem description, (i) the minimum depth of deterministic decision trees grows either almost as logarithm or linearly, (ii) the minimum depth of nondeterministic decision trees either is bounded from above by a constant or grows linearly, (iii) the minimum number of nodes in deterministic decision trees has either polynomial or exponential growth, and (iv) the minimum number of nodes in nondeterministic decision trees has either polynomial or exponential growth. Based on these results, we divide the set of all infinite binary information systems into five complexity classes, and study for each class issues related to time-space trade-off for decision trees.

[1]  H. Buhrman,et al.  Complexity measures and decision tree complexity: a survey , 2002, Theor. Comput. Sci..

[2]  Norbert Sauer,et al.  On the Density of Families of Sets , 1972, J. Comb. Theory A.

[3]  A. Yao,et al.  An exponential lower bound on the size of algebraic decision trees for Max , 1998, computational complexity.

[4]  Michael Ben-Or,et al.  Lower Bounds for Algebraic Computation Trees (Preliminary Report) , 1983, STOC 1983.

[5]  Mikhail Ju. Moshkov,et al.  Comparative Analysis of Deterministic and Nondeterministic Decision Tree Complexity Local Approach , 1996, Trans. Rough Sets.

[6]  Mikhail Ju. Moshkov.  Optimization Problems for Decision Trees , 1994, Fundam. Informaticae.

[7]  S. Shelah.  A combinatorial problem; stability and order for models and theories in infinitary languages. , 1972 .

[8]  Z..  INFORMATION SYSTEMS THEORETICAL FOUNDATIONS , 2022 .

[9]  Daniel Q. Naiman,et al.  Independence number and the complexity of families of sets , 1996, Discret. Math..

[10]  Fawaz Alsolami,et al.  Decision and Inhibitory Trees and Rules for Decision Tables with Many-valued Decisions , 2019, Intelligent Systems Reference Library.

[11]  Mikhail Ju. Moshkov.  Two Approaches to Investigation of Deterministic and Nondeterministic Decision Trees Complexity , 1995, WOCFAI.

[12]  Jaroslav Morávek.  A localization problem in geometry and complexity of discrete programming , 1972, Kybernetika.

[13]  Igor Chikalov,et al.  Extensions of Dynamic Programming for Combinatorial Optimization and Data Mining , 2018, Intelligent Systems Reference Library.

[14]  Mikhail Ju. Moshkov,et al.  About the Depth of Decision Trees Computing Boolean Functions , 1995, Fundam. Informaticae.

[15]  Wei-Yin Loh,et al.  Classification and regression trees , 2011, WIREs Data Mining Knowl. Discov..

[16]  F. Heide.  A polynomial linear search algorithm for the n-dimensional knapsack problem , 1983, STOC.

[17]  M. Yu. Moshkov,et al.  On conditional tests , 1982 .

[18]  Mikhail Moshkov,et al.  Time Complexity of Decision Trees , 2005, Trans. Rough Sets.

[19]  Andrew Chi-Chih Yao.  Decision tree complexity and Betti numbers , 1994, STOC '94.

[20]  Mikhail Ju. Moshkov,et al.  Comparative Analysis of Deterministic and Nondeterministic Decision Tree Complexity , 1996, Fundam. Informaticae.

[21]  Manuel Blum,et al.  Generic Oracles and Oracle Classes (Extended Abstract) , 1987, FOCS 1987.

[22]  Marek Karpinski,et al.  Improved lower bound on testing membership to a polyhedron by algebraic decision trees , 1995, Proceedings of IEEE 36th Annual Foundations of Computer Science.

[23]  Mikhail Ju. Moshkov,et al.  Classification of Infinite Information Systems Depending on Complexity of Decision Trees and Decision Rule Systems , 2002, Fundam. Informaticae.

[24]  Richard J. Lipton,et al.  On the Complexity of Computations under Varying Sets of Primitives , 1975, J. Comput. Syst. Sci..

[25]  Andreas Holzinger,et al.  Data Mining with Decision Trees: Theory and Applications , 2015, Online Inf. Rev..

[26]  Richard J. Lipton,et al.  Multidimensional Searching Problems , 1976, SIAM J. Comput..

[27]  Mikhail Moshkov,et al.  Comparative Analysis of Deterministic and Nondeterministic Decision Trees for Decision Tables from Closed Classes , 2020, Intelligent Systems Reference Library.

[28]  Andrew Chi-Chih Yao.  Algebraic Decision Trees and Euler Characteristics , 1992, FOCS.

[29]  J. Michael Steele,et al.  Lower Bounds for Algebraic Decision Trees , 1982, J. Algorithms.

[30]  Mikhail Ju. Moshkov,et al.  Combinatorial Machine Learning - A Rough Set Approach , 2011, Studies in Computational Intelligence.

[31]  DAVID DOBKIN,et al.  A Lower Bound of the ½n² on Linear Search Programs for the Knapsack Problem , 1978, J. Comput. Syst. Sci..