Neural network architectures for learning, prediction, and probability estimation

A new neural network architecture, called ARTMAP, is developed for incremental, nonparametric, supervised learning of recognition categories, multidimensional maps, and probability estimates. ARTMAP extends Adaptive Resonance Theory (ART) into the domain of supervised learning by incorporating environmental feedback and associative learning into the ART learning processes. Tested on benchmark classification problems such as distinguishing poisonous and edible mushrooms based on their visual features, ARTMAP outperforms a variety of systems in terms of speed, accuracy, and code compression. ARTMAP is also successfully applied to the incremental approximation of piecewise continuous functions, and to three probability estimation problems. The ARTMAP network includes a pair of Adaptive Resonance Theory modules, ART$\sb{a}$ and ART$\sb{b}.$ During training, input patterns are presented to ART$\sb{a}$ and output patterns are presented to ART$\sb{b}.$ During testing, input patterns are presented alone to ART$\sb{a}$ and ART$\sb{b}$ generates the system's predictions. ART$\sb{a}$ and ART$\sb{b}$ are linked by an associative learning network and an internal controller that conjointly maximize predictive generalization and minimize predictive error. When a predictive error occurs, the system's internal category structure is expanded by the minimum amount needed to correct the error, through an automatically controlled hypothesis testing process. ARTMAP is hereby a type of self-organizing expert system that calibrates the selectivity of its hypotheses based upon predictive success. As a result, rare but important events can be quickly and sharply distinguished even if they are similar to frequent events with different consequences. Because ARTMAP learning is self-stabilizing, it can continue learning one or more databases, without degrading its corpus of memories, until its full memory capacity is utilized.