STATIC AND DYNAMIC PREPROCESSING METHODS IN NEURAL NETWORKS

Preprocessing is recognized as an important tool in modeling, particularly when the data or underlying physical process involves complex nonlinear dynamical interactions. This paper will give a review of preprocessing methods used in linear and nonlinear models. The problem of static preprocessing will be considered first, where no dependence on time between the input vectors is assumed. Then, dynamic preprocessing methods which involve the modification of time-dependent input values before they are used in the linear or nonlinear models will be considered. Furthermore, the problem of an insufficient number of input vectors is considered. It is shown that one way in which this problem can be overcome is by expanding the weight vector in terms of the available input vectors. Finally, a new problem which involves both cases of: (1) transformation of input vectors; and (2) insufficient number of input vectors is considered. It is shown how a combination of the techniques used to solve the individual problems can be combined to solve this composite problem. Some open issues in this type of preprocessing methods are discussed.

[1]  Geoffrey E. Hinton,et al.  Phoneme recognition using time-delay neural networks , 1989, IEEE Trans. Acoust. Speech Signal Process..

[2]  Nasir Ahmed,et al.  Optimum Laguerre networks for a class of discrete-time systems , 1991, IEEE Trans. Signal Process..

[3]  Gene H. Golub,et al.  Matrix computations , 1983 .

[4]  Václav Peterka,et al.  Control of uncertain processes: applied theory and algorithms , 1986, Kybernetika.

[5]  Darrell Williamson,et al.  Delay replacement in direct form structures , 1988, IEEE Trans. Acoust. Speech Signal Process..

[6]  Edward B. Saff,et al.  Fundamentals of complex analysis for mathematics, science, and engineering , 1976 .

[7]  George Cybenko,et al.  Approximation by superpositions of a sigmoidal function , 1992, Math. Control. Signals Syst..

[8]  Biing-Hwang Juang,et al.  Fundamentals of speech recognition , 1993, Prentice Hall signal processing series.

[9]  Lennart Ljung,et al.  Theory and Practice of Recursive Identification , 1983 .

[10]  G. Goodwin,et al.  High-speed digital signal processing and control , 1992, Proc. IEEE.

[11]  José Carlos Príncipe,et al.  The gamma model--A new neural model for temporal processing , 1992, Neural Networks.

[12]  Ah Chung Tsoi,et al.  FIR and IIR Synapses, a New Neural Network Architecture for Time Series Modeling , 1991, Neural Computation.

[13]  Ah Chung Tsoi,et al.  A unifying view of some training algorithms for multilayer perceptrons with FIR filter synapses , 1994, Proceedings of IEEE Workshop on Neural Networks for Signal Processing.

[14]  Karl Johan Åström,et al.  Zeros of sampled systems , 1980, 1980 19th IEEE Conference on Decision and Control including the Symposium on Adaptive Processes.

[15]  Guy A. Dumont,et al.  On Determination of Laguerre Filter Pole through Step or Impulse Response Data , 1993 .

[16]  Ah Chung Tsoi,et al.  A Comparison of Discrete-Time Operator Models and for Nonlinear System Identification , 1994, NIPS.

[17]  José Carlos Príncipe,et al.  A Theory for Neural Networks with Time Delays , 1990, NIPS.

[18]  Graham C. Goodwin,et al.  Adaptive filtering prediction and control , 1984 .

[19]  W. M. Carey,et al.  Digital spectral analysis: with applications , 1986 .

[20]  Garrison W. Cottrell,et al.  Learning Mackey-Glass from 25 Examples, Plus or Minus 2 , 1993, NIPS.

[21]  Bart Kosko,et al.  Neural networks for signal processing , 1992 .

[22]  B. Wahlberg System identification using Laguerre models , 1991 .

[23]  Qiang Li,et al.  A delta -operator recursive gradient algorithm for adaptive signal processing , 1993, 1993 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[24]  Graham C. Goodwin,et al.  A Levinson-type algorithm for modeling fast-sampled data , 1991 .

[25]  Kurt Hornik,et al.  Multilayer feedforward networks are universal approximators , 1989, Neural Networks.

[26]  T. Oliveira e Silva,et al.  On the equivalence between Gamma and Laguerre filters , 1994, Proceedings of ICASSP '94. IEEE International Conference on Acoustics, Speech and Signal Processing.

[27]  D. Friedman,et al.  On approximating an FIR filter using discrete orthonormal exponentials , 1981 .

[28]  Sophocles J. Orfanidis,et al.  GramSchmidt Neural Nets , 1990, Neural Computation.

[29]  Bo Wahlberg,et al.  Parametric Signal Modelling using Laguerre Filters , 1993 .

[30]  Milos Doroslovacki,et al.  Cascade lattice IIR adaptive filters , 1994, IEEE Trans. Signal Process..

[31]  R. Palmer,et al.  Introduction to the theory of neural computation , 1994, The advanced book program.