A new decision-tree classification algorithm for machine learning

Although decision-tree classification algorithms have been widely used for machine learning in artificial intelligence, there has been little research toward evaluating the performance or quality of the current classification algorithms and investigating the time and computational complexity of constructing the smallest size decision tree which best distinguishes characteristics of multiple distinct groups. A known NP-complete problem, 3-exact cover, is used to prove that this problem is NP-complete. One prevalent classification algorithm in machine learning, ID3, is evaluated. The greedy search procedure used by ID3 is found to create anomalous behavior with inferior decision trees on a lot of occasions. A decision-tree classification algorithm, the intelligent decision-tree algorithm (IDA), that overcomes these anomalies with better classification performance is presented. A time analysis shows that IDA is more computationally efficient than ID3, and a simulation study indicates that IDA outperforms ID3.<<ETX>>

[1]  Thomas Marill,et al.  On the effectiveness of receptors in recognition systems , 1963, IEEE Trans. Inf. Theory.

[2]  J. Elashoff,et al.  On the choice of variables in classification problems with dichotomous variables. , 1967, Biometrika.

[3]  C. K. Chow,et al.  A Recognition Method Using Neighbor Dependence , 1962, IRE Trans. Electron. Comput..

[4]  Philip M. Lewis A Note on Realization of Decision Networks Using Summation Elements , 1961, Inf. Control..

[5]  Ronald L. Rivest,et al.  Constructing Optimal Binary Decision Trees is NP-Complete , 1976, Inf. Process. Lett..

[6]  Thomas M. Cover,et al.  The Best Two Independent Measurements Are Not the Two Best , 1974, IEEE Trans. Syst. Man Cybern..

[7]  Brian W. Kernighan,et al.  An Effective Heuristic Algorithm for the Traveling-Salesman Problem , 1973, Oper. Res..

[8]  J. Ross Quinlan,et al.  Learning Efficient Classification Procedures and Their Application to Chess End Games , 1983 .

[9]  J. D. F. Habbema,et al.  Comparative performance of four discriminant analysis methods for mixtures of continuous and discrete variables , 1983 .

[10]  M. Gessaman,et al.  A Comparison of Some Multivariate Discrimination Procedures , 1972 .

[11]  Jan M. Van Campenhout,et al.  On the Possible Orderings in the Measurement Selection Problem , 1977, IEEE Transactions on Systems, Man, and Cybernetics.

[12]  Godfried T. Toussaint,et al.  Note on optimal selection of independent binary-valued features for pattern recognition (Corresp.) , 1971, IEEE Trans. Inf. Theory.

[13]  James V. Hansen,et al.  Inducing rules for expert system development: an example using default and bankruptcy data , 1988 .

[14]  S. Kullback,et al.  Information Theory and Statistics , 1959 .

[15]  Chris Carter,et al.  Assessing Credit Card Applications Using Machine Learning , 1987, IEEE Expert.

[16]  Xiaojun Li,et al.  A note on dichotomous feature extraction , 1985, IEEE Trans. Inf. Theory.