A measure of fault tolerance for functional networks

This paper presents a study of the influence of perturbations in the parameters of a functional network. A quantitative measure is introduced, related to the change in the mean squared error when noise is applied to the network parameters. This measure, based on statistical sensitivity, provides a fault tolerance estimate for a functional network and allows the performance degradation of this kind of system to be predicted. It can be used, therefore, to evaluate performance differences between the training process carried out on a computer and its hardware implementation. The experimental results obtained for different functional network architectures and a feedforward multilayer neural network confirm the validity of the proposed model.

[1]  Amparo Alonso-Betanzos,et al.  A Measure of Noise Immunity for Functional Networks , 2001, IWANN.

[2]  Y. Tan,et al.  Fault-tolerant back-propagation model and its generalization ability , 1993, Proceedings of 1993 International Conference on Neural Networks (IJCNN-93-Nagoya, Japan).

[3]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[4]  Chin Park,et al.  A radial basis function neural network with on-chip learning , 1993, Proceedings of 1993 International Conference on Neural Networks (IJCNN-93-Nagoya, Japan).

[5]  Enrique F. Castillo,et al.  Functional networks with applications: a neural-based paradigm [Book Review] , 1999, IEEE Transactions on Neural Networks.

[6]  Robert I. Damper,et al.  Determining and improving the fault tolerance of multilayer perceptrons in a pattern-recognition application , 1993, IEEE Trans. Neural Networks.

[7]  Ignacio Rojas,et al.  An Accurate Measure for Multilayer Perceptron Tolerance to Weight Deviations , 1999, Neural Processing Letters.

[8]  Hideo Ito,et al.  On the Activation Function and Fault Tolerance in Feedforward Neural Networks , 1998 .

[9]  J. Aczél,et al.  Lectures on Functional Equations and Their Applications , 1968 .

[10]  Yann LeCun,et al.  Optimal Brain Damage , 1989, NIPS.

[11]  Enrique F. Castillo,et al.  Some Applications of Functional Networks in Statistics and Engineering , 2001, Technometrics.

[12]  Vincenzo Piuri,et al.  Sensitivity to errors in artificial neural networks: a behavioral approach , 1995 .

[13]  Bernard Widrow,et al.  Sensitivity of feedforward neural networks to weight errors , 1990, IEEE Trans. Neural Networks.

[14]  Julio Ortega Lopera,et al.  Assessing the Noise Immunity of Radial Basis Function Neural Networks , 2001, IWANN.

[15]  B. E. Segee,et al.  Fault tolerance of pruned multilayer networks , 1991, IJCNN-91-Seattle International Joint Conference on Neural Networks.

[16]  Zhi-Hua Zhou,et al.  Improving tolerance of neural networks against multi-node open fault , 2001, IJCNN'01. International Joint Conference on Neural Networks. Proceedings (Cat. No.01CH37222).

[17]  Ignacio Rojas,et al.  An Accurate Measure for Multilayer Perception Tolerance to Additive Weight Deviations , 1999, IWANN.

[18]  Marwan A. Jabri,et al.  Weight perturbation: an optimal architecture and learning technique for analog VLSI feedforward and recurrent multilayer networks , 1992, IEEE Trans. Neural Networks.

[19]  Dan Simon,et al.  Distributed fault tolerance in optimal interpolative nets , 2001, IEEE Trans. Neural Networks.

[20]  Dhananjay S. Phatak,et al.  Complete and partial fault tolerance of feedforward neural nets , 1995, IEEE Trans. Neural Networks.

[21]  Sang-Hoon Oh,et al.  Sensitivity analysis of single hidden-layer neural networks with threshold functions , 1995, IEEE Trans. Neural Networks.

[22]  A Learning Algorithm for Fault Tolerant Feedforward Neural Networks , 1996 .

[23]  Babak Hassibi,et al.  Second Order Derivatives for Network Pruning: Optimal Brain Surgeon , 1992, NIPS.

[24]  Ignacio Rojas,et al.  A Quantitative Study of Fault Tolerance, Noise Immunity, and Generalization Ability of MLPs , 2000, Neural Computation.

[25]  Chita R. Das,et al.  A probabilistic model for the fault tolerance of multilayer perceptrons , 1996, IEEE Trans. Neural Networks.

[26]  Enrique Castillo,et al.  Functional Networks , 1998, Neural Processing Letters.

[27]  Jerzy B. Lont.  Analog CMOS implementation of a multi-layer perceptron with nonlinear synapses , 1992, IEEE Trans. Neural Networks.

[28]  Michael J. Carter,et al.  Comparative Fault Tolerance of Parallel Distributed Processing Networks , 1994, IEEE Trans. Computers.

[29]  BattitiRoberto.  First- and second-order methods for learning , 1992 .

[30]  L. BuntineW.,et al.  Computing second derivatives in feed-forward networks , 1994 .

[31]  Chong-Ho Choi,et al.  Sensitivity analysis of multilayer perceptron with differentiable activation functions , 1992, IEEE Trans. Neural Networks.

[32]  Barry W. Johnson,et al.  Modeling of fault tolerance in neural networks , 1989, 15th Annual Conference of IEEE Industrial Electronics Society.

[33]  Wray L. Buntine,et al.  Computing second derivatives in feed-forward networks: a review , 1994, IEEE Trans. Neural Networks.

[34]  Zhi-Hua Zhou,et al.  Evolving Fault-Tolerant Neural Networks , 2003, Neural Computing & Applications.

[35]  Andries Petrus Engelbrecht,et al.  Using the Taylor expansion of multilayer feedforward neural networks , 2000, South Afr. Comput. J..

[36]  Antonio F. Díaz,et al.  Evaluating the impact of multiplicative input perturbations on radial basis function networks , 2002, ESANN.

[37]  L. Glass,et al.  Oscillation and chaos in physiological control systems. , 1977, Science.

[38]  Harry Wechsler,et al.  Distributed and Fault-Tolerant Computation for Retrieval Tasks Using Distributed Associative Memories , 1988, IEEE Trans. Computers.

[39]  Carver A. Mead,et al.  Implementing neural architectures using analog VLSI circuits , 1989 .

[40]  Daniele D. Caviglia,et al.  An experimental analog VLSI neural network with on-chip back-propagation learning , 1992, ESSCIRC '92: Eighteenth European Solid-State Circuits conference.

[41]  Ulrich Rückert,et al.  Implementation of artificial neural networks on a reconfigurable hardware accelerator , 2002, Proceedings 10th Euromicro Workshop on Parallel, Distributed and Network-based Processing.

[42]  Julio Ortega Lopera,et al.  Assessing the Noise Immunity and Generalization of Radial Basis Function Networks , 2004, Neural Processing Letters.

[43]  Roberto Battiti,et al.  First- and Second-Order Methods for Learning: Between Steepest Descent and Newton's Method , 1992, Neural Computation.

[44]  Chalapathy Neti,et al.  Maximally fault tolerant neural networks , 1992, IEEE Trans. Neural Networks.

[45]  James H. Garrett,et al.  Vehicle Detection Using a Hardware-Implemented Neural Net , 1997, IEEE Expert.

[46]  C. H. Sequin,et al.  Fault tolerance in artificial neural networks , 1990, 1990 IJCNN International Joint Conference on Neural Networks.