Fault tolerant constructive algorithm for feedforward neural networks

In this paper, a constructive algorithm for fault tolerant feedforward neural network, called FTCA, is proposed. The algorithm starts with a network with a single hidden neuron, and a new hidden unit is added to the network whenever it fails to converge. Before inserting the new hidden neuron into the network, only the weights connecting the new hidden neuron to the other neurons are trained (i.e. updated) until there is no significant reduction of the output error. To generate a fault tolerant network, the relevance of synaptic weights is estimated in each cycle. And only the weights which have a relevance less than a specified threshold are updated in that cycle. The loss of connections between neurons (which are equivalent to stuck-at-0 faults) are assumed. The simulation results indicate that the network constructed by FTCA has a significant fault tolerance ability.

[1]  Y. Tan,et al.  Fault-tolerant back-propagation model and its generalization ability , 1993, Proceedings of 1993 International Conference on Neural Networks (IJCNN-93-Nagoya, Japan).

[2]  Dhananjay S. Phatak,et al.  Complete and partial fault tolerance of feedforward neural nets , 1995, IEEE Trans. Neural Networks.

[3]  Etienne Barnard,et al.  Avoiding false local minima by proper initialization of connections , 1992, IEEE Trans. Neural Networks.

[4]  Robert I. Damper,et al.  Determining and improving the fault tolerance of multilayer perceptrons in a pattern-recognition application , 1993, IEEE Trans. Neural Networks.

[5]  G. Bolt,et al.  Fault models for artificial neural networks , 1991, [Proceedings] 1991 IEEE International Joint Conference on Neural Networks.

[6]  Chita R. Das,et al.  A probabilistic model for the fault tolerance of multilayer perceptrons , 1996, IEEE Trans. Neural Networks.

[7]  Christian Lebiere,et al.  The Cascade-Correlation Learning Architecture , 1989, NIPS.

[8]  Alan F. Murray,et al.  Enhanced MLP performance and fault tolerance resulting from synaptic weight noise during training , 1994, IEEE Trans. Neural Networks.

[9]  Helge Ritter,et al.  Cascade network architectures , 1992, [Proceedings 1992] IJCNN International Joint Conference on Neural Networks.

[10]  Helge Ritter,et al.  Cascade LLM Networks , 1992 .

[11]  A Learning Algorithm for Fault Tolerant Feedforward Neural Networks , 1996 .

[12]  Hideo Ito,et al.  On the Activation Function and Fault Tolerance in Feedforward Neural Networks , 1998 .

[13]  Hideo Ito,et al.  Fault tolerant design using error correcting code for multilayer neural networks , 1994, IEEE International Workshop on Defect and Fault Tolerance in VLSI Systems.

[14]  Eric B. Bartlett,et al.  Dynamic node architecture learning: An information theoretic approach , 1994, Neural Networks.