Decision templates for multiple classifier fusion: an experimental comparison

Multiple classi"er fusion may generate more accurate classi"cation than each of the constituent classi"ers. Fusion is often based on "xed combination rules like the product and average. Only under strict probabilistic conditions can these rules be justi"ed. We present here a simple rule for adapting the class combiner to the application. c decision templates (one per class) are estimated with the same training set that is used for the set of classi"ers. These templates are then matched to the decision pro"le of new incoming objects by some similarity measure. We compare 11 versions of our model with 14 other techniques for classi"er fusion on the Satimage and Phoneme datasets from the database ELENA. Our results show that decision templates based on integral type measures of similarity are superior to the other schemes on both data sets. ( 2000 Pattern Recognition Society. Published by Elsevier Science Ltd. All rights reserved.

[1]  J. Kacprzyk,et al.  The Ordered Weighted Averaging Operators: Theory and Applications , 1997 .

[2]  Michael I. Jordan,et al.  Local linear perceptrons for classification , 1996, IEEE Trans. Neural Networks.

[3]  Robert P. W. Duin,et al.  Classifier Conditional Posterior Probabilities , 1998, SSPR/SPR.

[4]  Jon Atli Benediktsson,et al.  Consensus theoretic classification methods , 1992, IEEE Trans. Syst. Man Cybern..

[5]  L. Kuncheva.  An application of OWA operators to the aggregation of multiple classification decisions , 1997 .

[6]  Hsin-Chia Fu,et al.  A divide-and-conquer methodology for modular supervised neural network design , 1994, Proceedings of 1994 IEEE International Conference on Neural Networks (ICNN'94).

[7]  Heekuck Oh,et al.  Neural Networks for Pattern Recognition , 1993, Adv. Comput..

[8]  B.V. Dasarathy,et al.  A composite classifier system design: Concepts and methodology , 1979, Proceedings of the IEEE.

[9]  Harris Drucker,et al.  Boosting and Other Ensemble Methods , 1994, Neural Computation.

[10]  Antonio Bellacicco,et al.  Handbook of statistics 2: Classification, pattern recognition and reduction of dimensionality: P.R. KRISHNAIAH and L.N. KANAL (Eds.) North-Holland, Amsterdam, 1982, xxii + 903 pages, Dfl.275.00 , 1984 .

[11]  Adam Krzyżak,et al.  Methods of combining multiple classifiers and their applications to handwriting recognition , 1992, IEEE Trans. Syst. Man Cybern..

[12]  Roberto Battiti,et al.  Democracy in neural nets: Voting schemes for classification , 1994, Neural Networks.

[13]  Volker Tresp,et al.  Combining Estimators Using Non-Constant Weighting Functions , 1994, NIPS.

[14]  Kevin W. Bowyer,et al.  Combination of multiple classifiers using local accuracy estimates , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[15]  Ching Y. Suen,et al.  A Method of Combining Multiple Experts for the Recognition of Unconstrained Handwritten Numerals , 1995, IEEE Trans. Pattern Anal. Mach. Intell..

[16]  Ching Y. Suen,et al.  Application of majority voting to pattern recognition: an analysis of its behavior and performance , 1997, IEEE Trans. Syst. Man Cybern. Part A.

[17]  Galina L. Rogova,et al.  Combining the results of several neural network classifiers , 1994, Neural Networks.

[18]  Frank J. Smieja,et al.  The pandemonium system of reflective agents , 1996, IEEE Trans. Neural Networks.

[19]  Ching Y. Suen,et al.  Optimal combinations of pattern classifiers , 1995, Pattern Recognit. Lett..

[20]  Eros Gian Alessandro Pasero,et al.  Multi-layer perceptron ensembles for increased performance and fault-tolerance in pattern recognition tasks , 1994, Proceedings of 1994 IEEE International Conference on Neural Networks (ICNN'94).

[21]  Geoffrey E. Hinton,et al.  Adaptive Mixtures of Local Experts , 1991, Neural Computation.

[22]  James M. Keller,et al.  Fuzzy Models and Algorithms for Pattern Recognition and Image Processing , 1999 .

[23]  Jiri Matas,et al.  On Combining Classifiers , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[24]  Isabelle Bloch.  Information combination operators for data fusion: a comparative review with classification , 1996, IEEE Trans. Syst. Man Cybern. Part A.

[25]  Ching Y. Suen,et al.  A method of combining multiple classifiers-a neural network approach , 1994, Proceedings of the 12th IAPR International Conference on Pattern Recognition, Vol. 3 - Conference C: Signal Processing (Cat. No.94CH3440-5).

[26]  Sung-Bae Cho,et al.  Combining multiple neural networks by fuzzy integral for robust classification , 1995, IEEE Trans. Syst. Man Cybern..

[27]  Janusz Kacprzyk,et al.  The Ordered Weighted Averaging Operators , 1997 .

[28]  P. Gader,et al.  Advances in fuzzy integration for pattern recognition , 1994, CVPR 1994.

[29]  Laveen N. Kanal,et al.  Classification, Pattern Recognition and Reduction of Dimensionality , 1982, Handbook of Statistics.

[30]  R. Bordley.  A Multiplicative Formula for Aggregating Probability Assessments , 1982 .

[31]  J. R. Sveinsson,et al.  Multistage classifiers optimized by neural networks and genetic algorithms , 1997 .

[32]  Robert P. W. Duin,et al.  PRTools - Version 3.0 - A Matlab Toolbox for Pattern Recognition , 2000 .

[33]  Sherif Hashem,et al.  Optimal Linear Combinations of Neural Networks , 1997, Neural Networks.

[34]  Ludmila I. Kuncheva,et al.  'Change-glasses' approach in pattern recognition , 1993, Pattern Recognit. Lett..

[35]  Robert A. Jacobs,et al.  Methods For Combining Experts' Probability Assessments , 1995, Neural Computation.

[36]  Richard O. Duda,et al.  Pattern classification and scene analysis , 1974, A Wiley-Interscience publication.

[37]  BattitiRoberto,et al.  Democracy in neural nets , 1994 .

[38]  Geoffrey E. Hinton,et al.  Evaluation of Adaptive Mixtures of Competing Experts , 1990, NIPS.

[39]  Kevin W. Bowyer,et al.  Combination of Multiple Classifiers Using Local Accuracy Estimates , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[40]  Michel Grabisch,et al.  Classification by fuzzy integral: performance and tests , 1994, CVPR 1994.

[41]  Paul D. Gader,et al.  Fusion of handwritten word classifiers , 1996, Pattern Recognit. Lett..

[42]  Ludmila I. Kuncheva.  On Combining Multiple Classifiers , 1998 .

[43]  Michael I. Jordan,et al.  Advances in Neural Information Processing Systems 30 , 1995 .

[44]  Robert P. W. Duin,et al.  Expected classification error of the Fisher linear classifier with pseudo-inverse covariance matrix , 1998, Pattern Recognit. Lett..

[45]  Sung-Bae Cho,et al.  Multiple network fusion using fuzzy logic , 1995, IEEE Trans. Neural Networks.

[46]  Peter E. Hart,et al.  Pattern classification and scene analysis , 1974, A Wiley-Interscience publication.

[47]  D Wang,et al.  Use of fuzzy-logic-inspired features to improve bacterial recognition through classifier fusion , 1998, IEEE Trans. Syst. Man Cybern. Part B.

[48]  Michael I. Jordan,et al.  Convergence results for the EM approach to mixtures of experts architectures , 1995, Neural Networks.

[49]  Didier Dubois,et al.  Fuzzy sets and systems ' . Theory and applications , 2007 .

[50]  Bruce W. Schmeiser,et al.  Optimal linear combinations of neural networks: an overview , 1994, Proceedings of 1994 IEEE International Conference on Neural Networks (ICNN'94).

[51]  J. A. Anderson,et al.  7 Logistic discrimination , 1982, Classification, Pattern Recognition and Reduction of Dimensionality.

[52]  B. Ripley,et al.  Pattern Recognition , 1968, Nature.

[53]  Keung-Chi Ng,et al.  Consensus diagnosis: a simulation study , 1992, IEEE Trans. Syst. Man Cybern..