Enhance Fault Tolerant Ability of Feedforward Neural Networks Against Multi-Node Open Fault

Neural networks are not intrinsically fault tolerant and their fault tolerance should be enhanced by extra mechanisms. Since 1990s, many work has addressed the issue of improving the fault tolerant ability of feedforward neural networks against some kinds of faults such as the single-node open fault where a hidden node and its associated weights in a neural network are out of work at the same time. However, few work touches the multi-node open fault where several hidden nodes and their associated weights in a neural network are out of work at the same time. In this paper, based on the recognition that the performance of a trained neural network does not linearly decrease with the increasing of the severity of fault, a three-phase approach T3 is proposed to enhance the fault tolerant ability of neural networks against the multi-node open fault. T3 is applicable to a specific class of feedforward neural networks whose hidden nodes can be dynamically appended during the training process. It trains a neural network at first, and then employs a validation data set to test the trained neural network to identify the inflection point of the fault curve of the neural network. After that, the neural network is retrained according to the fault rate corresponding to the inflection point of the fault curve of the network. During the training process some redundant hidden nodes are adaptively appended to the neural network. Such kind of training is repeatedly executed until the topology of the neural network does not change in two consecutive epochs. Experimental results show that T3 trades off the fault tolerant ability and the architectural complexity of feedforward neural networks in that it can significantly enhance the fault tolerant ability of neural networks against the 1multi-node open fault 1 with the cost of adding small amount of redundance.