Entropy Nets: From Decision Trees to Neural Networks

.A multiple-layer artificial network (ANN) structure is capable of implementing arbitrary input-output mappings. Similarly, hierarchical classifiers, more commonly known as decision trees, possess the capabilites of generating arbitrarily complex decision boundaries in an n-dimensional space. Given a decision tree, it is possible to restructure it as a multilayered neural network. The objective of this paper is to show how this mapping of decision trees into a multilayer neural network structure can be exploited for the systematic design of a class of layered neural networks, called entropy nets, that have far fewer connections. Several important issues such as the automatic tree generation, incorporation of incremental learning, and the generalization of knowledge acquired during the treedesign phase are discussed. Finally, a two-step methodology for designing entropy networks is presented. The advantages of this methodology are that it specifies the number of neurons needed in each layer, alongwith thedesired output. This leads to a faster progressive training procedure that allows each layer to be trained separately. Two examples are presented to show the success of neural network design through decision tree mapping.