Quasi-Optimum Combination of Multilayer Perceptrons for Adaptive Multiclass Pattern Recognition

Standard multiclass pattern recognition requires frequent re-learning stages when the set of categories of interest evolves in time. In order to minimize the computation costs of class incorporation and removal, we divide the global multiclass recognizer into a collection of class pairwise neural dichotomizers. When a new class appears, an adequate set of dichotomizers is created and trained to discriminate the new class from the rest. If a class disappears, its associated dichotomizers are eliminated In both cases previously learned knowledge is not disturbed. The properties of neural recognizers and pairwise modularization allow an analytic quasi-optimum method for combining network outputs to obtain the global multiclass response. An incremental and distributed pattern recognition architecture is presented and its performance experimentally evaluated, obtaining better error rates and learning times than conventional multiclass recognizers using similar resources. The design is highly parallel and asyncronous, adequate for dynamic real time applications.

[1]  Eric A. Wan,et al.  Neural network classification: a Bayesian interpretation , 1990, IEEE Trans. Neural Networks.

[2]  William G. Baxt,et al.  Improving the Accuracy of an Artificial Neural Network Using Multiple Differently Trained Networks , 1992, Neural Computation.

[3]  Anders Krogh,et al.  Introduction to the theory of neural computation , 1994, The advanced book program.

[4]  Fumitaka Kimura,et al.  Handwritten numerical recognition based on multiple algorithms , 1991, Pattern Recognit..

[5]  Keinosuke Fukunaga,et al.  Introduction to Statistical Pattern Recognition , 1972 .

[6]  Leonid I. Perlovsky,et al.  Maximum likelihood neural networks for sensor fusion and adaptive classification , 1991, Neural Networks.

[7]  Athanasios Papoulis,et al.  Probability, Random Variables and Stochastic Processes , 1965 .

[8]  Adam Krzyżak,et al.  Methods of combining multiple classifiers and their applications to handwriting recognition , 1992, IEEE Trans. Syst. Man Cybern..

[9]  Michael I. Jordan,et al.  Task Decomposition Through Competition in a Modular Connectionist Architecture: The What and Where Vision Tasks , 1990, Cogn. Sci..

[10]  Kurt Hornik,et al.  Multilayer feedforward networks are universal approximators , 1989, Neural Networks.

[11]  R. Palmer,et al.  Introduction to the theory of neural computation , 1994, The advanced book program.

[12]  Richard O. Duda,et al.  Pattern classification and scene analysis , 1974, A Wiley-Interscience publication.

[13]  Sargur N. Srihari,et al.  Decision Combination in Multiple Classifier Systems , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[14]  C.Y. Suen,et al.  Associative switch for combining multiple classifiers , 1991, IJCNN-91-Seattle International Joint Conference on Neural Networks.

[15]  Hervé Bourlard,et al.  Factoring Networks by a Statistical Method , 1992, Neural Computation.

[16]  Kurt Hornik,et al.  FEED FORWARD NETWORKS ARE UNIVERSAL APPROXIMATORS , 1989 .

[17]  Yann LeCun,et al.  Transforming Neural-Net Output Levels to Probability Distributions , 1990, NIPS.