Abstract A major problem with connectionist networks is that newly-learned information may completely destroy previously-learned information unless the network is continually retrained on the old information. This phenomenon, known as catastrophic forgetting, is unacceptable both for practical purposes and as a model of mind. This paper advances the claim that catastrophic forgetting is in part the result of the overlap of system's distributed representations and can be reduced by reducing this overlap. A simple algorithm, called activation sharpening, is presented that allows a standard feed-forward backpropagation network to develop semi-distributed representations, thereby reducing the problem of catastrophic forgetting. Activation sharpening is discussed in tight of recent work done by other researchers who have experimented with this and other techniques for reducing catastrophic forgetting.
[1]
Pentti Kanerva,et al.
Sparse Distributed Memory
,
1988
.
[2]
Teuvo Kohonen,et al.
The self-organizing map
,
1990
.
[3]
R. Nosofsky.
American Psychological Association, Inc. Choice, Similarity, and the Context Theory of Classification
,
2022
.
[4]
Michael McCloskey,et al.
Catastrophic Interference in Connectionist Networks: The Sequential Learning Problem
,
1989
.
[5]
R Ratcliff,et al.
Connectionist models of recognition memory: constraints imposed by learning and forgetting functions.
,
1990,
Psychological review.
[6]
Robert M. French,et al.
Using Semi-Distributed Representations to Overcome Catastrophic Forgetting in Connectionist Networks
,
1991
.