The recent introduction of compact large capacity memories has opened up possibilities for more aggressive use of data in learning systems. Instead of using complex, global models for the data we investigate the use of well-known but modified and extended local non-parametric methods for learning. Our focus is on learning problems where a large set of data is available to the system designer. Such applications include speech recognition, character recognition and local weather prediction. The general system we present, called an Adaptive Memory, (AM) is adaptive in the sense that some part of the sample data is stored and a local non- parametric model is updated when new training data becomes available. This makes training possible throughout the usable lifetime of the system, in contrast with many popular learning algorithms like neural networks and other parametric methods that have a distinctive learning phase. In the past designers of learning systems have been reluctant to store data samples in memory because of the inherent slowness of searching and storing. However, with the advent of parallel searching algorithms and high speed large memories the AM approach is competitive with parametric methods and may ultimately exceed their performance for a large class of problems.
[1]
Elie Bienenstock,et al.
Neural Networks and the Bias/Variance Dilemma
,
1992,
Neural Computation.
[2]
Keinosuke Fukunaga,et al.
Estimation of Classification Error
,
1970,
IEEE Transactions on Computers.
[3]
Jon Louis Bentley,et al.
An Algorithm for Finding Best Matches in Logarithmic Expected Time
,
1977,
TOMS.
[4]
T. Kohonen,et al.
Appendix 2.4 Stopping Rule 2.3 Fine Tuning Using the Basic Lvq1 or Lvq2.1 Lvq Pak: a Program Package for the Correct Application of Learning Vector Quantization Algorithms
,
1992
.
[5]
Stephen Grossberg,et al.
Art 2: Self-Organization Of Stable Category Recognition Codes For Analog Input Patterns
,
1988,
Other Conferences.
[6]
Patti Price,et al.
The DARPA 1000-word resource management database for continuous speech recognition
,
1988,
ICASSP-88., International Conference on Acoustics, Speech, and Signal Processing.
[7]
Yuchun Lee,et al.
Handwritten Digit Recognition Using K Nearest-Neighbor, Radial-Basis Function, and Backpropagation Neural Networks
,
1991,
Neural Computation.