An Iterative Growing and Pruning Algorithm for Classification Tree Design

A critical issue in classification tree design-obtaining right-sized trees, i.e. trees which neither underfit nor overfit the data-is addressed. Instead of stopping rules to halt partitioning, the approach of growing a large tree with pure terminal nodes and selectively pruning it back is used. A new efficient iterative method is proposed to grow and prune classification trees. This method divides the data sample into two subsets and iteratively grows a tree with one subset and prunes it with the other subset, successively interchanging the roles of the two subsets. The convergence and other properties of the algorithm are established. Theoretical and practical considerations suggest that the iterative free growing and pruning algorithm should perform better and require less computation than other widely used tree growing and pruning algorithms. Numerical results on a waveform recognition problem are presented to support this view. >

[1]  Ching Y. Suen,et al.  Analysis and Design of a Decision Tree Based on Entropy Reduction and Its Application to Large Character Set Recognition , 1984, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[2]  Philip H. Swain,et al.  Selective Radiant Temperature Mapping Using a Layered Classifier , 1976 .

[3]  Padhraic Smyth,et al.  Decision tree design from a communication theory standpoint , 1988, IEEE Trans. Inf. Theory.

[4]  E. M. Rounds.  A combined nonparametric approach to feature selection and binary decision tree design , 1980, Pattern Recognit..

[5]  Roland T. Chin,et al.  An Automated Approach to the Design of Decision Tree Classifiers , 1982 .

[6]  William S. Meisel,et al.  An Algorithm for Constructing Optimal Binary Decision Trees , 1977, IEEE Transactions on Computers.

[7]  Ishwar K. Sethi,et al.  Efficient decision tree design for discrete variable pattern recognition problems , 1977, Pattern Recognition.

[8]  Robert J. Marks,et al.  A performance comparison of trained multilayer perceptrons and trained classification trees , 1989, Conference Proceedings., IEEE International Conference on Systems, Man and Cybernetics.

[9]  G. H. Landeweerd,et al.  Binary tree versus single level tree classification of white blood cells , 1983, Pattern Recognit..

[10]  K. S. Fu,et al.  An Approach to the Design of a Linear Binary Tree Classifier , 1976 .

[11]  M. Kurzynski.  The optimal strategy of a tree classifier , 1983 .

[12]  King-Sun Fu,et al.  A method for the design of binary tree classifiers , 1983, Pattern Recognit..

[13]  Philip H. Swain,et al.  Purdue e-Pubs , 2022 .

[14]  George Nagy,et al.  Decision tree design using a probabilistic model , 1984, IEEE Trans. Inf. Theory.

[15]  King-Sun Fu,et al.  Automated classification of nucleated blood cells using a binary tree classifier , 1980, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[16]  King-Sun Fu,et al.  Automatic classification of cervical cells using a binary tree classifier , 1983, Pattern Recognition.

[17]  Philip A. Chou,et al.  Optimal pruning with applications to tree-structured source coding and modeling , 1989, IEEE Trans. Inf. Theory.

[18]  Saul Brian Gelfand.  A nonparametric multiclass partitioning method for classification , 1982 .

[19]  William S. Meisel,et al.  A Partitioning Algorithm with Application in Pattern Classification and the Optimization of Decision Trees , 1973, IEEE Transactions on Computers.

[20]  Jack Sklansky,et al.  Locally Trained Piecewise Linear Classifiers , 1980, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[21]  J. Taylor,et al.  Automated hierarchic decision structures for multiple category cell classification by TICAS. , 1978, Acta cytologica.

[22]  Jerome H. Friedman,et al.  A Recursive Partitioning Decision Rule for Nonparametric Classification , 1977, IEEE Transactions on Computers.

[23]  I. K. Sethi,et al.  Hierarchical Classifier Design Using Mutual Information , 1982, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[24]  Ashok V. Kulkarni.  On the Mean Accuracy of Hierarchical Classifiers , 1978, IEEE Transactions on Computers.

[25]  Ching Y. Suen,et al.  Large Tree Classifier with Heuristic Search and Global Training , 1987, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[26]  King-Sun Fu,et al.  A Nonparametric Partitioning Procedure for Pattern Classification , 1969, IEEE Transactions on Computers.

[27]  Marek W. Kurzyski.  Decision rules for a hierarchical classifier , 1983 .

[28]  Ching Y. Suen,et al.  Application of a Multilayer Decision Tree in Computer Recognition of Chinese Characters , 1983, IEEE Transactions on Pattern Analysis and Machine Intelligence.