Analysis and Design of a Decision Tree Based on Entropy Reduction and Its Application to Large Character Set Recognition

Based on a recursive process of reducing the entropy, the general decision tree classifier with overlap has been analyzed. Several theorems have been proposed and proved. When the number of pattern classes is very large, the theorems can reveal both the advantages of a tree classifier and the main difficulties in its implementation. Suppose H is Shannon's entropy measure of the given problem. The theoretical results indicate that the tree searching time can be minimized to the order O(H), but the error rate is also in the same order O(H) due to error accumulation. However, the memory requirement is in the order 0(H exp(H)) which poses serious problems in the implementation of a tree classifier for a large number of classes. To solve these problems, several theorems related to the bounds on the search time, error rate, memory requirement and overlap factor in the design of a decision tree have been proposed and some principles have been established to analyze the behaviors of the decision tree. When applied to classify sets of 64, 450, and 3200 Chinese characters, respectively, the experimental results support the theoretical predictions. For 3200 classes, a very high recognition rate of 99.88 percent was achieved at a high speed of 873 samples/s when the experiment was conducted on a Cyber 172 computer using a high-level language.

[1]  Harry C. Andrews,et al.  Multidimensional Rotations in Feature Selection , 1971, IEEE Transactions on Computers.

[2]  Laveen N. Kanal,et al.  Patterns in pattern recognition: 1968-1974 , 1974, IEEE Trans. Inf. Theory.

[3]  Philip H. Swain,et al.  Purdue e-Pubs , 2022 .

[4]  Theodosios Pavlidis,et al.  Fuzzy Decision Tree Algorithms , 1977, IEEE Transactions on Systems, Man, and Cybernetics.

[5]  C. H. Chen,et al.  On the Use of Distance and Information Measures in Pattern Recognition and Applications , 1977 .

[6]  Satosi Watanabe,et al.  Pattern recognition as a quest for minimum entropy , 1981, Pattern Recognit..

[7]  I. K. Sethi,et al.  Hierarchical Classifier Design Using Mutual Information , 1982, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[8]  Roland T. Chin,et al.  An Automated Approach to the Design of Decision Tree Classifiers , 1982, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[9]  King-Sun Fu,et al.  A method for the design of binary tree classifiers , 1983, Pattern Recognit..

[10]  King-Sun Fu,et al.  Automatic classification of cervical cells using a binary tree classifier , 1983, Pattern Recognition.

[11]  Ching Y. Suen,et al.  Application of a Multilayer Decision Tree in Computer Recognition of Chinese Characters , 1983, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[12]  Ching Y. Suen,et al.  ISOETRP - an interactive clustering algorithm with new objectives , 1984, Pattern Recognit..