Dynamical Systems that Learn Subspaces

In this paper we define and study a class of nonlinear filters capable of solving a range of problems which arise in the design of adaptive analog systems. Even though the definitions used are compelling in terms of the phenomena and natural in terms of mathematics, the filters require careful analysis because they can exhibit discontinuous behavior. Using some recent results on a type of gradient flow on the orghogonal group, we are able to construct a differential equation realization of a smooth approximation to these filters.

[1]  Claude E. Shannon,et al.  A symbolic analysis of relay and switching circuits , 1938, Transactions of the American Institute of Electrical Engineers.

[2]  Jack B. Dennis,et al.  Mathematical Programming and Electrical Networks , 1959, The Mathematical Gazette.

[3]  R E Kalman,et al.  CANONICAL STRUCTURE OF LINEAR DYNAMICAL SYSTEMS. , 1962, Proceedings of the National Academy of Sciences of the United States of America.

[4]  S. Grossberg.  Nonlinear difference-differential equations in prediction and learning theory. , 1967, Proceedings of the National Academy of Sciences of the United States of America.

[5]  E. Oja.  Simplified neuron model as a principal component analyzer , 1982, Journal of mathematical biology.

[6]  Narendra Karmarkar,et al.  A new polynomial-time algorithm for linear programming , 1984, Comb..

[7]  Bernard Widrow,et al.  Adaptive Signal Processing , 1985 .

[8]  James L. McClelland,et al.  Parallel distributed processing: explorations in the microstructure of cognition, vol. 1: foundations , 1986 .

[9]  S. Shinomoto.  Memory maintenance in neural networks , 1987 .

[10]  Teuvo Kohonen,et al.  Self-Organization and Associative Memory , 1988 .

[11]  Wing Shing Wong,et al.  A Short-Term Neural Network Memory , 1988, SIAM J. Comput..

[12]  Terence D. Sanger,et al.  Optimal unsupervised learning in a single-layer linear feedforward neural network , 1989, Neural Networks.

[13]  Kurt Hornik,et al.  Neural networks and principal component analysis: Learning from examples without local minima , 1989, Neural Networks.

[14]  R. Brockett.  Least squares matching problems , 1989 .

[15]  R. Brockett.  Dynamical systems that sort lists, diagonalize matrices, and solve linear programming problems , 1991 .