Partially connected feedforward neural networks structured by input types

This paper proposes a new method to model partially connected feedforward neural networks (PCFNNs) from the identified input type (IT) which refers to whether each input is coupled with or uncoupled from other inputs in generating output. The identification is done by analyzing input sensitivity changes as amplifying the magnitude of inputs. The sensitivity changes of the uncoupled inputs are not correlated with the variation on any other input, while those of the coupled inputs are correlated with the variation on any one of the coupled inputs. According to the identified ITs, a PCFNN can be structured. Each uncoupled input does not share the neurons in the hidden layer with other inputs in order to contribute to output in an independent manner, while the coupled inputs share the neurons with one another. After deriving the mathematical input sensitivity analysis for each IT, several experiments, as well as a real example (blood pressure (BP) estimation), are described to demonstrate how well our method works.

[1]  Lorenzo Mussone,et al.  A review of feedforward neural networks in transportation research , 1999 .

[2]  John Shawe-Taylor,et al.  Feedforward Neural Networks: a Tutorial , 1989 .

[3]  M. Kenward,et al.  An Introduction to the Bootstrap , 2007 .

[4]  Wayne Ieee,et al.  Entropy Nets: From Decision Trees to Neural Networks , 1990 .

[5]  Sung-Mo Kang,et al.  Model-order reduction of weakly nonlinear MEMS devices with Taylor series expansion and Arnoldi process , 2000, Proceedings of the 43rd IEEE Midwest Symposium on Circuits and Systems (Cat.No.CH37144).

[6]  Bart Baesens,et al.  Sensitivity based pruning of input variables by means of weight cascaded retraining , 2000 .

[7]  Jonathan M. Roberts,et al.  Flight control using an artificial neural network , 2000 .

[8]  Jenq-Neng Hwang,et al.  Critical input data channels selection for progressive work exercise test by neural network sensitivity analysis , 1999, 1999 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings. ICASSP99 (Cat. No.99CH36258).

[9]  Russell Beale,et al.  Handbook of Neural Computation , 1996 .

[10]  Gregory J. Wolff,et al.  Optimal Brain Surgeon: Extensions and performance comparisons , 1993, NIPS 1993.

[11]  Morgan Mangeas,et al.  Forecasting Electricity Demand Using Non-linear Mixture of Experts , 1995 .

[12]  Yann LeCun,et al.  Optimal Brain Damage , 1989, NIPS.

[13]  Donald E. Duckro,et al.  Neural Network Pruning with Tukey-Kramer Multiple Comparison Procedure , 2002, Neural Computation.

[14]  Kazuo Furuta,et al.  Identification of Nonlinear Dynamic Models with Partially Connected Neural Networks Trained Using Orthogonal Least Square Estimation , 1999 .

[15]  Jacek M. Zurada,et al.  Perturbation method for deleting redundant inputs of perceptron networks , 1997, Neurocomputing.

[16]  Neil W. Bergmann,et al.  High-speed neural network based classifier for real-time application , 1998, ICSP '98. 1998 Fourth International Conference on Signal Processing (Cat. No.98TH8344).

[17]  N Petkov.  Systolic simulation of multilayer, feedforward neural networks , 1990 .

[18]  E. Fiesler,et al.  Comparative Bibliography of Ontogenic Neural Networks , 1994 .

[19]  Andrew H. Sung,et al.  Ranking input importance in neural network modeling of engineering problems , 1998, 1998 IEEE International Joint Conference on Neural Networks Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98CH36227).

[20]  David A. Elizondo,et al.  A Survey of Partially Connected Neural Networks , 1997, Int. J. Neural Syst..

[21]  David A. Elizondo,et al.  Non-ontogenic sparse neural networks , 1995, Proceedings of ICNN'95 - International Conference on Neural Networks.

[22]  R.J.F. Dow,et al.  Neural net pruning-why and how , 1988, IEEE 1988 International Conference on Neural Networks.

[23]  Douglas M. Bates,et al.  Nonlinear Regression Analysis and Its Applications , 1988 .

[24]  Ehud D. Karnin,et al.  A simple procedure for pruning back-propagation trained neural networks , 1990, IEEE Trans. Neural Networks.

[25]  Hiroyuki Mori,et al.  An artificial neural-net based method for predicting power system voltage harmonics , 1992 .