Competitive Learning: From Interactive Activation to Adaptive Resonance

Functional and mechanistic comparisons are made between several network models of cognitive processing: competitive learning, interactive activation, adaptive resonance, and back propagation. The starting point of this comparison is the article of Rumelhart and Zipser (1985) on feature discovery through competitive learning. All the models which Rumelhart and Zipser (1985) have described were shown in Grossberg (1976b) to exhibit a type of learning which is temporally unstable. Competitive learning mechanisms can be stabilized in response to an arbitrary input environment by being supplemented with mechanisms for learning top-down expectancies, or templates; for matching bottom-up input patterns with the top-down expectancies; and for releasing orienting reactions in a mismatch situation, thereby updating short-term memory and searching for another internal representation. Network architectures which embody all of these mechanisms were called adaptive resonance models by Grossberg (1976c) . Self-stabilizing learning models are candidates for use in real-world applications where unpredictable changes can occur in complex input environments. Competitive learning postulates are inconsistent with the postulates of the interactive activation model of McClelland and Rumelhart (1981) , and suggest different levels of processing and interaction rules for the analysis of word recognition. Adaptive resonance models use these alternative levels and interaction rules. The selforganizing learning of an adaptive resonance model is compared and contrasted with the teacher-directed learning of a back propagation model. A number of criteria for evaluating real-time network models of cognitive processing are described and applied.

[1]  D. D. Wheeler.  Processes in word recognition , 1970 .

[2]  Teuvo Kohonen,et al.  Self-Organization and Associative Memory , 1988 .

[3]  M. Posner,et al.  Retention of Abstract Ideas. , 1970 .

[4]  S. Grossberg,et al.  Neural dynamics of attention switching and temporal-order information in short-term memory , 1988, Memory & cognition.

[5]  S. Grossberg.  The Adaptive Self-Organization of Serial Order in Behavior: Speech, Language, And Motor Control , 1987 .

[6]  S. Grossberg,et al.  Neural dynamics of decision making under risk: affective balance and cognitive-emotional interactions. , 1988, Psychological review.

[7]  S. Grossberg.  How does a brain build a cognitive code , 1980 .

[8]  J. C. Johnston,et al.  Reply to Matthei: we really is worse than you or them, and so are Ma and Pa. , 1983, Journal of experimental psychology. Human perception and performance.

[9]  Roman Bek,et al.  Discourse on one way in which a quantum-mechanics language on the classical logical base can be built up , 1978, Kybernetika.

[10]  S. Grossberg.  A neural model of attention, reinforcement and discrimination learning. , 1975, International review of neurobiology.

[11]  S. Grossberg.  Studies of mind and brain : neural principles of learning, perception, development, cognition, and motor control , 1982 .

[12]  Michael I. Posner,et al.  Cognition: An Introduction , 1973 .

[13]  W. Estes.  Memory storage and retrieval processes in category learning. , 1986 .

[14]  S. Grossberg,et al.  Neural dynamics of attentionally modulated Pavlovian conditioning: blocking, interstimulus interval, and secondary reinforcement. , 1987, Applied optics.

[15]  S. Grossberg,et al.  Neural dynamics of word recognition and recall: attentional priming, learning, and resonance. , 1986 .

[16]  A. A. Mullin,et al.  Principles of neurodynamics , 1962 .

[17]  James L. McClelland,et al.  An interactive activation model of context effects in letter perception: Part 2. The contextual enhancement effect and some tests and extensions of the model. , 1982, Psychological review.

[18]  James L. McClelland,et al.  Distributed memory and the representation of general and specific information. , 1985, Journal of experimental psychology. General.

[19]  J. C. Johnston,et al.  Length effects in word perception: we is better than i but worse than you or them. , 1982 .

[20]  M. Posner,et al.  On the genesis of abstract ideas. , 1968, Journal of experimental psychology.

[21]  S. Grossberg.  Associative and Competitive Principles of Learning and Development: The Temporal Unfolding and Stability of Stm and Ltm Patterns , 1987 .

[22]  S. Grossberg,et al.  Probing cognitive processes through the structure of event-related potentials during learning: an experimental and theoretical analysis. , 1987, Applied optics.

[23]  J.A. Anderson,et al.  Theory of categorization based on distributed memory storage. , 1984 .

[24]  A. Salasoo,et al.  Building permanent memory codes: codification and repetition effects in word identification. , 1985, Journal of experimental psychology. General.

[25]  S. Grossberg,et al.  Neural Dynamics of Category Learning and Recognition: Attention, Memory Consolidation, and Amnesia , 1987 .

[26]  S Grossberg,et al.  Masking fields: a massively parallel neural architecture for learning, recognizing, and predicting multiple groupings of patterned data. , 1987, Applied optics.

[27]  Terrence J. Sejnowski,et al.  NETtalk: a parallel network that learns to read aloud , 1988 .

[28]  Geoffrey E. Hinton,et al.  Learning internal representations by error propagation , 1986 .

[29]  Stephen Grossberg,et al.  A Theory of Human Memory: Self-Organization and Performance of Sensory-Motor Codes, Maps, and Plans , 1982 .

[30]  James L. McClelland,et al.  An interactive activation model of context effects in letter perception: I. An account of basic findings. , 1981 .

[31]  Stephen Grossberg,et al.  A massively parallel architecture for a self-organizing neural pattern recognition machine , 1988, Comput. Vis. Graph. Image Process..

[32]  S. Grossberg.  Processing of Expected and Unexpected Events During Conditioning and Attention: A Psychophysiological Theory , 1982 .

[33]  David Zipser,et al.  Feature Discovery by Competive Learning , 1986, Cogn. Sci..

[34]  Stephen Grossberg,et al.  Neural dynamics of speech and language coding: developmental programs, perceptual grouping, and competition for short-term memory. , 1986, Human neurobiology.

[35]  J J Hopfield,et al.  Neural networks and physical systems with emergent collective computational abilities. , 1982, Proceedings of the National Academy of Sciences of the United States of America.

[36]  S. Grossberg.  On learning and energy-entropy dependence in recurrent and nonrecurrent signed networks , 1969 .

[37]  James L. McClelland.  Putting Knowledge in its Place: A Scheme for Programming Parallel Processing Structures on the Fly , 1988, Cogn. Sci..

[38]  Stephen Grossberg,et al.  Associative Learning, Adaptive Pattern Recognition, And Cooperative-Competitive Decision Making By Neural Networks , 1986, Other Conferences.

[39]  E. Bienenstock,et al.  Theory for the development of neuron selectivity: orientation specificity and binocular interaction in visual cortex , 1982, The Journal of neuroscience : the official journal of the Society for Neuroscience.