Competitive Anti-Hebbian Learning of Invariants

Although the detection of invariant structure in a given set of input patterns is vital to many recognition tasks, connectionist learning rules tend to focus on directions of high variance (principal components). The prediction paradigm is often used to reconcile this dichotomy; here we suggest a more direct approach to invariant learning based on an anti-Hebbian learning rule. An unsupervised two-layer network implementing this method in a competitive setting learns to extract coherent depth information from random-dot stereograms.

[1]  E. Oja.  Simplified neuron model as a principal component analyzer , 1982, Journal of mathematical biology.

[2]  E. Oja,et al.  On stochastic approximation of the eigenvectors and eigenvalues of the expectation of a random matrix , 1985 .

[3]  Teuvo Kohonen,et al.  Self-Organization and Associative Memory , 1988 .

[4]  Teuvo Kohonen,et al.  Self-organization and associative memory: 3rd edition , 1989 .

[5]  Steven J. Nowlan,et al.  Maximum Likelihood Competitive Learning , 1989, NIPS.

[6]  Terence D. Sanger,et al.  Optimal unsupervised learning in a single-layer linear feedforward neural network , 1989, Neural Networks.

[7]  Peter Földiák,et al.  Adaptation and decorrelation in the cortex , 1989 .

[8]  Kurt Hornik,et al.  Neural networks and principal component analysis: Learning from examples without local minima , 1989, Neural Networks.

[9]  Richard Durbin,et al.  The computing neuron , 1989 .

[10]  Jeffrey L. Elman,et al.  Finding Structure in Time , 1990, Cogn. Sci..

[11]  Michael I. Jordan,et al.  Advances in Neural Information Processing Systems 30 , 1995 .

[12]  Geoffrey E. Hinton,et al.  Discovering Viewpoint-Invariant Relationships That Characterize Objects , 1990, NIPS.

[13]  Nathan Intrator.  Exploratory Feature Extraction in Speech Signals , 1990, NIPS.

[14]  Peter Földiák,et al.  Learning Invariance from Transformation Sequences , 1991, Neural Comput..

[15]  Graeme Mitchison,et al.  Removing Time Variation with the Anti-Hebbian Differential Synapse , 1991, Neural Computation.

[16]  T. Leen.  Dynamics of learning in linear feature-discovery networks , 1991 .

[17]  S. Kung,et al.  Neural networks for extracting unsymmetric principal components , 1991, Neural Networks for Signal Processing Proceedings of the 1991 IEEE Workshop.

[18]  Geoffrey E. Hinton,et al.  Self-organizing neural network that discovers surfaces in random-dot stereograms , 1992, Nature.