Low sensitivity time delay neural networks with cascade form structure

In current practice, tapped delay line models such as the time delay neural network (TDNN) are commonly implemented using a direct form structure. In this paper, we show that the problem of high parameter sensitivity, well known in linear systems, also applies to nonlinear models such as the TDNN. To overcome the consequent numerical problems, we propose a cascade form TDNN (CTDNN) and show its advantages over the commonly used direct form TDNN.

[1]  Patrick E. Mantey.  Eigenvalue sensitivity and state-variable selection , 1968 .

[2]  A. W. M. van den Enden,et al.  Discrete Time Signal Processing , 1989 .

[3]  Bernard Widrow,et al.  Sensitivity of feedforward neural networks to weight errors , 1990, IEEE Trans. Neural Networks.

[4]  K. Åström,et al.  Zeros of sampled systems , 1980 .

[5]  Chong-Ho Choi,et al.  Sensitivity analysis of multilayer perceptron with differentiable activation functions , 1992, IEEE Trans. Neural Networks.

[6]  Ramesh C. Agarwal,et al.  New recursive digital filter structures having very low sensitivity and roundoff noise , 1975 .

[7]  C. K. Yuen,et al.  Theory and Application of Digital Signal Processing , 1978, IEEE Transactions on Systems, Man, and Cybernetics.

[8]  Ah Chung Tsoi,et al.  A Low-Sensitivity Recurrent Neural Network , 1998, Neural Computation.

[9]  Karl Johan Åström,et al.  Zeros of sampled systems , 1980, 1980 19th IEEE Conference on Decision and Control including the Symposium on Adaptive Processes.

[10]  Darrell Williamson,et al.  Delay replacement in direct form structures , 1988, IEEE Trans. Acoust. Speech Signal Process..

[11]  C. Lee Giles,et al.  Alternative discrete-time operators and their application to nonlinear models , 1997 .

[12]  Geoffrey E. Hinton,et al.  A time-delay neural network architecture for isolated word recognition , 1990, Neural Networks.

[13]  Hong Chen,et al.  Approximations of continuous functionals by neural networks with application to dynamic systems , 1993, IEEE Trans. Neural Networks.

[14]  Günhan Dündar,et al.  The effects of quantization on multilayer neural networks , 1995, IEEE Trans. Neural Networks.

[15]  I. W. Sandberg.  Approximation theorems for discrete-time systems , 1991, [1991] Proceedings of the 34th Midwest Symposium on Circuits and Systems.

[16]  W. Rugh.  Nonlinear System Theory: The Volterra / Wiener Approach , 1981 .

[17]  Vincenzo Piuri,et al.  Sensitivity to errors in artificial neural networks: a behavioral approach , 1995 .

[18]  Ah Chung Tsoi,et al.  FIR and IIR Synapses, a New Neural Network Architecture for Time Series Modeling , 1991, Neural Computation.