On Combining Classifiers

We develop a common theoretical framework for combining classifiers which use distinct pattern representations and show that many existing schemes can be considered as special cases of compound classification where all the pattern representations are used jointly to make a decision. An experimental comparison of various classifier combination schemes demonstrates that the combination rule developed under the most restrictive assumptions-the sum rule-outperforms other classifier combinations schemes. A sensitivity analysis of the various schemes to estimation errors is carried out to show that this finding can be justified theoretically.

[1]  Josef Kittler,et al.  Pattern recognition : a statistical approach , 1982 .

[2]  Marek Kurzynski.  On the identity of optimal strategies for multistage classifiers , 1989, Pattern Recognit. Lett..

[3]  Hisham El-Shishiny,et al.  A multistage algorithm for fast classification of patterns , 1989, Pattern Recognit. Lett..

[4]  Michael C. Fairhurst,et al.  An interactive two-level architecture for a memory network pattern classifier , 1990, Pattern Recognit. Lett..

[5]  Lars Kai Hansen,et al.  Neural Network Ensembles , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[6]  Fumitaka Kimura,et al.  Handwritten numerical recognition based on multiple algorithms , 1991, Pattern Recognit..

[7]  Léon Bottou,et al.  Local Learning Algorithms , 1992, Neural Computation.

[8]  J. Kittler,et al.  Multistage pattern recognition with reject option , 1992, Proceedings., 11th IAPR International Conference on Pattern Recognition. Vol.II. Conference B: Pattern Recognition Methodology and Systems.

[9]  J. Franke,et al.  A comparison of two approaches for combining the votes of cooperating classifiers , 1992, Proceedings., 11th IAPR International Conference on Pattern Recognition. Vol.II. Conference B: Pattern Recognition Methodology and Systems.

[10]  David H. Wolpert,et al.  Stacked generalization , 1992, Neural Networks.

[11]  Adam Krzyżak,et al.  Methods of combining multiple classifiers and their applications to handwriting recognition , 1992, IEEE Trans. Syst. Man Cybern..

[12]  Léon Bottou,et al.  Local Algorithms for Pattern Recognition and Dependencies Estimation , 1993, Neural Computation.

[13]  Dmitry A. Denisov,et al.  Model-based chromosome recognition via hypotheses construction/verification , 1994, Pattern Recognit. Lett..

[14]  A. J. Elms.  A connected character recogniser using level building of HMMs , 1994, Proceedings of the 12th IAPR International Conference on Pattern Recognition, Vol. 3 - Conference C: Signal Processing (Cat. No.94CH3440-5).

[15]  Sargur N. Srihari,et al.  Decision Combination in Multiple Classifier Systems , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[16]  Hsi-Jian Lee,et al.  Multi-stage pre-candidate selection in handwritten chinese character recognition systems , 1994, Pattern Recognit..

[17]  Jiangying Zhou,et al.  Discrimination of characters by a multi-stage recognition process , 1994, Pattern Recognit..

[18]  Volker Tresp,et al.  Combining Estimators Using Non-Constant Weighting Functions , 1994, NIPS.

[19]  Anders Krogh,et al.  Neural Network Ensembles, Cross Validation, and Active Learning , 1994, NIPS.

[20]  Galina L. Rogova,et al.  Combining the results of several neural network classifiers , 1994, Neural Networks.

[21]  Nikhil R. Pal,et al.  A multistage generalization of the rank nearest neighbor classification rule , 1995, Pattern Recognit. Lett..

[22]  Josef Kittler,et al.  Constraining Probabilistic Relaxation with Symbolic Attributes , 1995, CAIP.

[23]  Bruce W. Schmeiser,et al.  Improving model accuracy using optimal linear combinations of trained neural networks , 1995, IEEE Trans. Neural Networks.

[24]  Ching Y. Suen,et al.  A Method of Combining Multiple Experts for the Recognition of Unconstrained Handwritten Numerals , 1995, IEEE Trans. Pattern Anal. Mach. Intell..

[25]  Sung-Bae Cho,et al.  Combining multiple neural networks by fuzzy integral for robust classification , 1995, IEEE Trans. Syst. Man Cybern..

[26]  Sung-Bae Cho,et al.  Multiple network fusion using fuzzy logic , 1995, IEEE Trans. Neural Networks.

[27]  Kagan Tumer,et al.  Classifier Combining: Analytical Results and Implications , 1995 .

[28]  Josef Kittler,et al.  Combining symbolic with numeric attributes in multi-class object recognition problems , 1995, Proceedings., International Conference on Image Processing.

[29]  Kamal A. Ali,et al.  On the Link between Error Correlation and Error Reduction in Decision Tree Ensembles , 1995 .

[30]  Majid Ahmadi,et al.  Recognition of handwritten numerals with multiple feature and multistage classifier , 1995, Pattern Recognit..

[31]  Kagan Tumer,et al.  Analysis of decision boundaries in linearly combined neural classifiers , 1996, Pattern Recognit..

[32]  Josef Kittler,et al.  Combining classifiers , 1996, Proceedings of 13th International Conference on Pattern Recognition.

[33]  Yoav Freund,et al.  Experiments with a New Boosting Algorithm , 1996, ICML.

[34]  Kevin W. Bowyer,et al.  Combination of multiple classifiers using local accuracy estimates , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[35]  Robert P. W. Duin,et al.  Stabilizing classifiers for very small sample sizes , 1996, Proceedings of 13th International Conference on Pattern Recognition.

[36]  Gérard Chollet,et al.  Combining methods to improve speaker verification decision , 1996, Proceeding of Fourth International Conference on Spoken Language Processing. ICSLP '96.

[37]  Josef Kittler,et al.  Weighting Factors in Multiple Expert Fusion , 1997, BMVC.

[38]  Josef Kittler,et al.  Strategies for combining classifiers employing shared and distinct pattern representations , 1997, Pattern Recognit. Lett..

[39]  Luc Vandendorpe,et al.  Profile Authentication Using a Chamfer Matching Algorithm , 1997, AVBPA.

[40]  Yoav Freund,et al.  Boosting the margin: A new explanation for the effectiveness of voting methods , 1997, ICML.

[41]  Luc Vandendorpe,et al.  The M2VTS Multimodal Face Database (Release 1.00) , 1997, AVBPA.

[42]  Jiri Matas,et al.  Combining evidence in personal identity verification systems , 1997, Pattern Recognit. Lett..

[43]  Jiri Matas,et al.  Fast face localisation and verification , 1999, Image Vis. Comput..

[44]  Takeo Kanade,et al.  A statistical approach to 3d object detection applied to faces and cars , 2000 .