Improving tolerance of neural networks against multi-node open fault

Neural networks are not intrinsically fault tolerant and their fault tolerance has to be improved by employing extra mechanisms. During the last decades, some simple fault types of feedforward neural networks have been widely investigated. In this paper, a rather complicated fault type, i.e. a multi-node open fault where several hidden nodes are out of work at the same time, is formally analyzed, and an approach named T3 is proposed. The ground of T3 is the recognition that the performance of trained neural networks does not linearly decrease with the increasing of the severity of fault. T3 utilizes a validation set to build the fault curve of a trained network. It then locates the inflection point of the fault curve and repeatedly trains the network according to the corresponding fault rate so that the redundancy are appended to the network appropriately. Experiments show that T3 can improve the tolerance against multi-node open fault of some feedforward neural networks at the expense of relatively small redundancy.

[1]  Dhananjay S. Phatak,et al.  Complete and partial fault tolerance of feedforward neural nets , 1995, IEEE Trans. Neural Networks.

[2]  Kishan G. Mehrotra,et al.  Training techniques to obtain fault-tolerant neural networks , 1994, Proceedings of IEEE 24th International Symposium on Fault- Tolerant Computing.

[3]  Jos Nijhuis,et al.  Limits to the fault-tolerance of a feedforward neural network with learning , 1990, [1990] Digest of Papers. Fault-Tolerant Computing: 20th International Symposium.

[4]  Terrence J. Sejnowski,et al.  Analysis of hidden units in a layered network trained to classify sonar targets , 1988, Neural Networks.

[5]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[6]  Hideo Ito,et al.  On the Activation Function and Fault Tolerance in Feedforward Neural Networks , 1998 .

[7]  Ahmed El-Amawy,et al.  On Fault Tolerant Training of Feedforward Neural Networks , 1997, Neural Networks.

[8]  Y. Tan,et al.  Fault-tolerant back-propagation model and its generalization ability , 1993, Proceedings of 1993 International Conference on Neural Networks (IJCNN-93-Nagoya, Japan).

[9]  Jim Austin,et al.  Fault Tolerant Multi-Layer Perceptron Networks , 1992 .

[10]  B. E. Segee,et al.  Fault tolerance of pruned multilayer networks , 1991, IJCNN-91-Seattle International Joint Conference on Neural Networks.

[11]  Heekuck Oh,et al.  Neural Networks for Pattern Recognition , 1993, Adv. Comput..

[12]  Nait Charif Hammadi,et al.  Dynamic Constructive Fault Tolerant Algorithm for Feedforward Neural Networks , 1998 .

[13]  Thomas Jackson,et al.  Neural Computing - An Introduction , 1990 .

[14]  Robert I. Damper,et al.  Determining and improving the fault tolerance of multilayer perceptrons in a pattern-recognition application , 1993, IEEE Trans. Neural Networks.

[15]  Benjamin W. Wah,et al.  Fault tolerant neural networks with hybrid redundancy , 1990, 1990 IJCNN International Joint Conference on Neural Networks.

[16]  Chalapathy Neti,et al.  Maximally fault tolerant neural networks , 1992, IEEE Trans. Neural Networks.

[17]  Shiyuan Yang,et al.  A modified learning algorithm for improving the fault tolerance of BP networks , 1996, Proceedings of International Conference on Neural Networks (ICNN'96).

[18]  Yutaka Hata,et al.  Activation function manipulation for fault tolerant feedforward neural networks , 1999, Proceedings Eighth Asian Test Symposium (ATS'99).

[19]  Zhi-Hua Zhou,et al.  FANNC: A Fast Adaptive Neural Network Classifier , 2000, Knowledge and Information Systems.

[20]  A Learning Algorithm for Fault Tolerant Feedforward Neural Networks , 1996 .

[21]  Terumine Hayashi,et al.  Evaluation function for fault tolerant multi-layer neural networks , 2000, Proceedings of the IEEE-INNS-ENNS International Joint Conference on Neural Networks. IJCNN 2000. Neural Computing: New Challenges and Perspectives for the New Millennium.

[22]  G. Bolt,et al.  Fault models for artificial neural networks , 1991, [Proceedings] 1991 IEEE International Joint Conference on Neural Networks.