Improving learning rate of neural tree networks using thermal perceptrons

A new neural network called the neural tree network (NTN) is a combination of decision trees and multi-layer perceptrons (MLP). The NTN grows the network as opposed to MLPs. The learning algorithm for growing NTNs is more efficient that standard decision tree algorithms. Simulation results have shown that the NTN is superior in performance to both decision trees and MLPs. A new NTN learning algorithm is proposed based on the thermal perceptron algorithm. It is shown that the new algorithm greatly increases the speed of learning of the NTN and attains similar classification performance as the previously used algorithm.<<ETX>>

[1]  C. D. Gelatt,et al.  Optimization by Simulated Annealing , 1983, Science.

[2]  R. Mammone,et al.  Neural tree networks , 1992 .

[3]  Alexander H. Waibel,et al.  Modular Construction of Time-Delay Neural Networks for Speech Recognition , 1989, Neural Computation.

[4]  Marcus R. Frean,et al.  Small nets and short paths : optimising neural computation , 1990 .

[5]  Richard J. Mammone,et al.  Speaker independent vowel recognition using neural tree networks , 1991, IJCNN-91-Seattle International Joint Conference on Neural Networks.

[6]  I ScottKirkpatrick Optimization by Simulated Annealing: Quantitative Studies , 1984 .

[7]  Geoffrey E. Hinton,et al.  A time-delay neural network architecture for isolated word recognition , 1990, Neural Networks.

[8]  Richard J. Mammone,et al.  Optimal pruning of neural tree networks for improved generalization , 1991, IJCNN-91-Seattle International Joint Conference on Neural Networks.

[9]  Mohamad T. Musavi,et al.  A neural network approach to character recognition , 1989, Neural Networks.

[10]  Yann LeCun,et al.  Handwritten zip code recognition with multilayer networks , 1990, [1990] Proceedings. 10th International Conference on Pattern Recognition.

[11]  Scott Kirkpatrick,et al.  Optimization by simulated annealing: Quantitative studies , 1984 .

[12]  Ronald L. Rivest,et al.  Constructing Optimal Binary Decision Trees is NP-Complete , 1976, Inf. Process. Lett..

[13]  Shun-ichi Amari,et al.  A Theory of Adaptive Pattern Classifiers , 1967, IEEE Trans. Electron. Comput..