The task of determining an effective architecture for multi-layer feed forward backpropagation like neural networks can be a time consuming effort. Over the past couple years several algorithms were proposed for dynamically constructing network architectures. Some of these algorithms have been shown to converge for binary data. Unfortunately, the results do not carry over for the nonbinary input case. For classificatory problems no guarantees are provided, which would suggest otherwise. In this paper, we present an extension to the basic network growing algorithms, which allows constructing networks in bounded time. The algorithm guarantees convergence for any classificatory domain, as long as no contradictory training examples are present. The extension is described for the Divide & Conquer Networks (DCN) algorithm. The derived mathematical model can be readily incorporated into other network growing approaches to ensure convergence. The model is dependent on the usage of simple threshold cells, which can be applied to a variety of learning rules.
[1]
Lawrence O. Hall,et al.
Divide and Conquer Neural Networks
,
1993,
Neural Networks.
[2]
James J. Solberg,et al.
Operations Research: Principles and Practice.
,
1977
.
[3]
Marcus Frean,et al.
The Upstart Algorithm: A Method for Constructing and Training Feedforward Neural Networks
,
1990,
Neural Computation.
[4]
John M. Zelle,et al.
Growing layers of perceptrons: introducing the Extentron algorithm
,
1992,
[Proceedings 1992] IJCNN International Joint Conference on Neural Networks.
[5]
Timur Ash,et al.
Dynamic node creation in backpropagation networks
,
1989
.
[6]
Christian Lebiere,et al.
The Cascade-Correlation Learning Architecture
,
1989,
NIPS.