A maximum entropy approach to multiple classifiers combination

In this paper,we present amaximumentropy (maxent) approach to the fusion of experts opinions, or classifiers outputs, problem. Themaxent approach is quite versatile and allows us to express in a clear, rigorous,way the a priori knowledge that is available on the problem. For instance, our knowledge about the reliability of the experts and the correlations between these experts can be easily integrated: Each piece of knowledge is expressed in the form of a linear constraint. An iterative scaling algorithm is used in order to compute the maxent solution of the problem. The maximum entropy method seeks the joint probability density of a set of random variables that has maximum entropy while satisfying the constraints. It is therefore the “most honest” characterization of our knowledge given the available facts (constraints). In the case of conflicting constraints, we propose to minimise the “lack of constraints satisfaction” or to relax some constraints and recompute the maximum entropy solution. The maxent fusion rule is illustrated by some simulations.

[1]  George J. Klir,et al.  Fuzzy sets, uncertainty and information , 1988 .

[2]  T. Modis,et al.  Experts in uncertainty , 1993 .

[3]  Solomon Kullback,et al.  Approximating discrete probability distributions , 1969, IEEE Trans. Inf. Theory.

[4]  Michel Grabisch,et al.  Assessing the value of a candidate: Comparing belief function and possibility theories , 1999, UAI.

[5]  J. N. Kapur,et al.  Entropy optimization principles with applications , 1992 .

[6]  Hiremaglur K. Kesavan,et al.  The generalized maximum entropy principle (with applications) , 1987 .

[7]  S. Katti.  Operational Subjective Statistical Methods , 1998 .

[8]  Douglas J. Miller,et al.  Maximum entropy econometrics: robust estimation with limited data , 1996 .

[9]  William B. Levy,et al.  Maximum entropy aggregation of individual opinions , 1994, IEEE Trans. Syst. Man Cybern..

[10]  Christopher J. Merz,et al.  Using Correspondence Analysis to Combine Classifiers , 1999, Machine Learning.

[11]  Xiuzhen Cheng,et al.  An asymptotic analysis of some expert fusion methods , 2001, Pattern Recognit. Lett..

[12]  Christian Genest,et al.  Combining Probability Distributions: A Critique and an Annotated Bibliography , 1986 .

[13]  Lotfi A. Zadeh,et al.  Fuzzy Sets , 1996, Inf. Control..

[14]  Robert A. Jacobs,et al.  Methods For Combining Experts' Probability Assessments , 1995, Neural Computation.

[15]  Josef Kittler,et al.  Sum Versus Vote Fusion in Multiple Classifier Systems , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[16]  Frederick Jelinek,et al.  Statistical methods for speech recognition , 1997 .

[17]  Jiri Matas,et al.  On Combining Classifiers , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[18]  I. J. Myung,et al.  Maximum Entropy Aggregation of Expert Predictions , 1996 .