On-line training of a continually adapting adaline-like network

A rigorous mathematical analysis is presented of an adaline-like network operating in continuous time with spatially continuous inputs and outputs. Weights adapt continually, whether or not a training signal is present. It is shown that consistent input-output pairs can be learned perfectly provided every pattern is repeated at least once in every N successive inputs, and the input patterns are nearly orthogonal, depending on N.

[1]  Sophocles J. Orfanidis,et al.  GramSchmidt Neural Nets , 1990, Neural Computation.

[2]  N. G. Parke,et al.  Ordinary Differential Equations. , 1958 .

[3]  B. Baird,et al.  Relation of olfactory EEG to behavior: spatial analysis. , 1987, Behavioral neuroscience.

[4]  Pineda,et al.  Generalization of back-propagation to recurrent neural networks. , 1987, Physical review letters.

[5]  R. Cuninghame-Green,et al.  Applied Linear Algebra , 1979 .

[6]  Robert C. Williamson,et al.  Local minima and attractors at infinity for gradient descent learning algorithms , 1996 .

[7]  Robert C. Williamson,et al.  Learning Nonlinearly Parametrized Decision Regions , 1993 .

[8]  Robert R. Bitmead,et al.  Persistence of excitation conditions and the convergence of adaptive schemes , 1984, IEEE Trans. Inf. Theory.

[9]  Kurt Hornik,et al.  Learning in linear neural networks: a survey , 1995, IEEE Trans. Neural Networks.

[10]  Ulrich Ramacher.  Hamiltonian dynamics of neural networks , 1993, Neural Networks.

[11]  Morris W. Hirsch.  On-line training of memory-driven attractor networks , 1997 .

[12]  W. Freeman,et al.  EEG Spatial Pattern Differences with Discriminated Odors Manifest Chaotic and Limit Cycle Attractors in Olfactory Bulb of Rabbits , 1986 .

[13]  Bernard Widrow,et al.  30 years of adaptive neural networks: perceptron, Madaline, and backpropagation , 1990, Proc. IEEE.

[14]  C. R. Johnson,et al.  Lectures on adaptive parameter estimation , 1988 .

[15]  J J Hopfield,et al.  Neurons with graded response have collective computational properties like those of two-state neurons. , 1984, Proceedings of the National Academy of Sciences of the United States of America.

[16]  V. W. Noonburg.  A neural network modeled by an adaptive Lotka-Volterra system , 1989 .

[17]  A. Selverston,et al.  Learning algorithms for oscillatory networks with gap junctions and membrane currents , 1991 .

[18]  Eduardo D. Sontag,et al.  Backpropagation separates when perceptrons do , 1989, International 1989 Joint Conference on Neural Networks.

[19]  S Townsend.  Social marketing of condoms: selling protection and changing behavior. , 1991, Network.