Analysis of sensitivity behavior of Madalines

This paper aims at exploring the behavior of the sensitivity to weight perturbation for an ensemble of Madalines. An analytical formula is derived first for the calculation of the Adalines' sensitivity, and then based on it an algorithm is given for the computation of the Madalines' sensitivity. The efficiency of the computation is verified by computer simulations. By means of the formula and the algorithm, the sensitivity tendency is analyzed with regard to the variation of some parameters, such as the dimension of input, the number of Adalines in a layer and the number of layers. The analysis results show that the dimension of input has little effect on the sensitivity as long as the dimension is sufficient large, while the increases in the number of Adalines in a layer and the number of layers will lead the sensitivity to increase under an upper bound. These tendencies of the Madalines' sensitivity will be useful for designing robust Madalines with appropriate architecture.

[1]  Chong-Ho Choi,et al.  Sensitivity analysis of multilayer perceptron with differentiable activation functions , 1992, IEEE Trans. Neural Networks.

[2]  Daniel S. Yeung,et al.  Sensitivity analysis of multilayer perceptron to input and weight perturbations , 2001, IEEE Trans. Neural Networks.

[3]  Steve W. Piche,et al.  The selection of weight accuracies for Madalines , 1995, IEEE Trans. Neural Networks.

[4]  L. Fu,et al.  Sensitivity analysis for input vector in multilayer feedforward neural networks , 1993, IEEE International Conference on Neural Networks.

[5]  Sang-Hoon Oh,et al.  Sensitivity analysis of single hidden-layer neural networks with threshold functions , 1995, IEEE Trans. Neural Networks.

[6]  Daniel S. Yeung,et al.  A Quantified Sensitivity Measure for Multilayer Perceptron to Input Perturbation , 2003, Neural Computation.

[7]  Xiaoqin Zeng,et al.  Sensitivity of Madalines to input and weight perturbations , 2003, Proceedings of the 2003 International Conference on Machine Learning and Cybernetics (IEEE Cat. No.03EX693).

[8]  Bernard Widrow,et al.  Sensitivity of feedforward neural networks to weight errors , 1990, IEEE Trans. Neural Networks.