Information Analysis of Multiple Classifier Fusion

We consider a general scheme of parallel classifier combinations in the framework of statistical pattern recognition. Each statistical classifier defines a set of output variables in terms of a posteriori probabilities, i.e. it is used as a feature extractor. Unlike usual combining schemes the output vectors of classifiers are combined in parallel. The statistical Shannon information is used as a criterion to compare different combining schemes from the point of view of the theoretically available decision information. By means of relatively simple arguments we derive a theoretical hierarchy between different schemes of classifier fusion in terms of information inequalities.

[1]  Lars Kai Hansen,et al.  Neural Network Ensembles , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[2]  Galina L. Rogova,et al.  Combining the results of several neural network classifiers , 1994, Neural Networks.

[3]  Ching Y. Suen,et al.  A Method of Combining Multiple Experts for the Recognition of Unconstrained Handwritten Numerals , 1995, IEEE Trans. Pattern Anal. Mach. Intell..

[4]  Josef Kittler,et al.  Combining Multiple Classifiers in Probabilistic Neural Networks , 2000, Multiple Classifier Systems.

[5]  Sung-Bae Cho,et al.  Multiple network fusion using fuzzy logic , 1995, IEEE Trans. Neural Networks.

[6]  Igor Vajda,et al.  About the maximum information and maximum likelihood principles , 1998, Kybernetika.

[7]  Jiri Matas,et al.  On Combining Classifiers , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[8]  Ching Y. Suen,et al.  A method of combining multiple classifiers-a neural network approach , 1994, Proceedings of the 12th IAPR International Conference on Pattern Recognition, Vol. 3 - Conference C: Signal Processing (Cat. No.94CH3440-5).

[9]  Sung-Bae Cho,et al.  Combining multiple neural networks by fuzzy integral for robust classification , 1995, IEEE Trans. Syst. Man Cybern..

[10]  David H. Wolpert,et al.  Stacked generalization , 1992, Neural Networks.

[11]  R. Casey,et al.  Advances in Pattern Recognition , 1971 .

[12]  Françoise Fogelman-Soulié,et al.  Multi-Modular Neural Network Architectures: Applications in Optical Character and Human Face Recognition , 1993, Int. J. Pattern Recognit. Artif. Intell..

[13]  Pavel Pudil,et al.  On Virtually Binary Nature of Probabilistic Neural Networks , 1998, SSPR/SPR.

[14]  Chuanyi Ji,et al.  Combinations of Weak Classifiers , 1996, NIPS.

[15]  Ching Y. Suen,et al.  A theoretical analysis of the application of majority voting to pattern recognition , 1994, Proceedings of the 12th IAPR International Conference on Pattern Recognition, Vol. 3 - Conference C: Signal Processing (Cat. No.94CH3440-5).