Data Fusion with Entropic Priors

In classification problems, lack of knowledge of the prior distribution may make the application of Bayes' rule inadequate. Uniform or arbitrary priors may often provide classification answers that, even in simple examples, may end up contradicting our common sense about the problem. Entropic priors, determined via the application of the maximum entropy principle, seem to provide a much better answer and can be easily derived and applied to classification tasks when no more than the likelihood functions are available. In this paper we present an example in which the use of the entropic priors is compared to the results of the application of Dempster-Shafer theory.

[1]  Tilman Neumann.  Bayesian Inference Featuring Entropic Priors , 2007 .

[2]  Philippe Smets,et al.  Belief functions: The disjunctive rule of combination and the generalized Bayesian theorem , 1993, Int. J. Approx. Reason..

[3]  R. Baierlein.  Probability Theory: The Logic of Science , 2004 .

[4]  Thomas Lukasiewicz.  MAXIMUM ENTROPY , 2000 .

[5]  P. Smets,et al.  Target classification approach based on the belief function theory , 2005, IEEE Transactions on Aerospace and Electronic Systems.

[6]  C. C. Rodriguez.  Entropic priors for discrete probabilistic networks and for mixtures of Gaussians models , 2002, physics/0201016.

[7]  Glenn Shafer,et al.  A Mathematical Theory of Evidence , 2020, A Mathematical Theory of Evidence.

[8]  Dsto PO Box,et al.  Belief function theory on the continuous space with an application to model based classification , 2006 .

[9]  E. Jaynes.  Information Theory and Statistical Mechanics , 1957 .

[10]  Philippe Smets,et al.  Decision making in the TBM: the necessity of the pignistic transformation , 2005, Int. J. Approx. Reason..

[11]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems , 1988 .

[12]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[13]  R. Preuss,et al.  Maximum entropy and Bayesian data analysis: Entropic prior distributions. , 2003, Physical review. E, Statistical, nonlinear, and soft matter physics.

[14]  Daniel Kahneman,et al.  Probabilistic reasoning , 1993 .

[15]  Ariel Caticha.  Maximum entropy, fluctuations and priors , 2001 .

[16]  Ariel Caticha,et al.  Updating Probabilities with Data and Moments , 2007, ArXiv.

[17]  Thomas M. Cover,et al.  Elements of Information Theory (Wiley Series in Telecommunications and Signal Processing) , 2006 .