MAXIMUM ENTROPY

We apply the principle of maximum entropy to select a unique j oint probability distribution from the set of all joint probability distribu tions specified by a credal network. In detail, we start by showing that the unique joint distribu t on of a Bayesian tree coincides with the maximum entropy model of its conditional distribut ions. This result, however, does not hold anymore for general Bayesian networks. We thus present a new kind of maximum entropy models, which are computed sequentially. We th en show that for all general Bayesian networks, the sequential maximum entropy model co incides with the unique joint distribution. Moreover, we apply the new principle of seque ntial maximum entropy to interval Bayesian networks and more generally to credal netwo rks. We especially show that this application is equivalent to a number of small local ent ropy maximizations. 1Institut und Ludwig Wittgenstein Labor für Informationss y teme, Technische Universität Wien, Favoritenstraße 9-11, A-1040 Vienna, Austria. E-mail: lukasiewicz@kr.tuwien.ac.at. Acknowledgements: I am very grateful to Fabio Gagliardi Cozman, Gabriele Kern -Isberner, and Richard Neapolitan for their useful comments on an earlier v ersion of this paper. This work has been partially supported by a DFG grant and the Austrian Scie nce Fund Project N Z29-INF. Copyright c 2000 by the authors 2 INFSYS RR 1843-00-03

[1]  L. M. M.-T..  Theory of Probability , 1929, Nature.

[2]  H. Jeffreys.  Logical Foundations of Probability , 1952, Nature.

[3]  E. Jaynes.  Information Theory and Statistical Mechanics , 1957 .

[4]  Rodney W. Johnson,et al.  Axiomatic derivation of the principle of maximum entropy and the principle of minimum cross-entropy , 1980, IEEE Trans. Inf. Theory.

[5]  E. T. Jaynes,et al.  Papers on probability, statistics and statistical physics , 1983 .

[6]  Peter C. Cheeseman,et al.  A Method of Computing Generalized Bayesian Probability Values for Expert Systems , 1983, IJCAI.

[7]  Henry E. Kyburg,et al.  Bayesian and Non-Bayesian Evidential Updating , 1987, Artificial Intelligence.

[8]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[9]  Daniel Hunter,et al.  Causality and maximum entropy updating , 1989, Int. J. Approx. Reason..

[10]  Jeff B. Paris,et al.  A note on the inevitability of maximum entropy , 1990, Int. J. Approx. Reason..

[11]  P. Walley.  Statistical Reasoning with Imprecise Probabilities , 1990 .

[12]  John S. Breese,et al.  Decision making with interval influence diagrams , 1990, UAI.

[13]  A. Hasman,et al.  Probabilistic reasoning in intelligent systems: Networks of plausible inference , 1991 .

[14]  Bjørnar Tessem,et al.  Interval probability propagation , 1992, Int. J. Approx. Reason..

[15]  David Poole,et al.  Probabilistic Horn Abduction and Bayesian Networks , 1993, Artif. Intell..

[16]  Joseph Y. Halpern,et al.  Random Worlds and Maximum Entropy , 1994, J. Artif. Intell. Res..

[17]  Peter Haddawy,et al.  Anytime Deduction for Probabilistic Logic , 1994, Artif. Intell..

[18]  Andrés Cano,et al.  Convex Sets Of Probabilities Propagation By Simulated Annealing , 1994 .

[19]  Peter Haddawy,et al.  Generating Bayesian Networks from Probablity Logic Knowledge Bases , 1994, UAI.

[20]  Wilhelm Rödder,et al.  Coherent Knowledge Processing at Maximum Entropy by SPIRIT , 1996, UAI.

[21]  Joseph Y. Halpern,et al.  From Statistical Knowledge Bases to Degrees of Belief , 1996, Artif. Intell..

[22]  Joseph Y. Halpern,et al.  Irrelevance and Conditioning in First-Order Probabilistic Logic , 1996, AAAI/IAAI, Vol. 1.

[23]  Andrés Cano,et al.  A Genetic algorithm to approximate convex sets of probabilities , 1996 .

[24]  Werner Kießling,et al.  Increased robustness of Bayesian networks through probability intervals , 1997, Int. J. Approx. Reason..

[25]  Jeff B. Paris,et al.  In defense of the maximum entropy inference process , 1997, Int. J. Approx. Reason..

[26]  Gabriele Kern-Isberner.  A Logically Sound Method for Uncertain Reasoning with Quantified Conditionals , 1997, ECSQARU-FAPR.

[27]  Enrico Fagiuoli,et al.  2U: An Exact Interval Propagation Algorithm for Polytrees with Binary Variables , 1998, Artif. Intell..

[28]  Thomas Lukasiewicz,et al.  Probabilistic Deduction with Conditional Constraints over Basic Events , 2011, KR.

[29]  Fábio Gagliardi Cozman.  Irrelevance and Independence Relations in quasi-Bayesian Networks , 1998, UAI.

[30]  Gabriele Kern-Isberner,et al.  Characterizing the Principle of Minimum Cross-Entropy Within a Conditional-Logical Framework , 1998, Artif. Intell..

[31]  Fábio Gagliardi Cozman.  Irrelevance and Independence Axioms in Quasi-Bayesian Theory , 1999, ESCQARU.

[32]  Wolfgang Ertel,et al.  Combining Data and Knowledge by MaxEnt-Optimization of Probability Distributions , 1999, PKDD.

[33]  Thomas Lukasiewicz,et al.  Probalilistic Logic Programming under Maximum Entropy , 1999, ESCQARU.

[34]  Thomas Lukasiewicz,et al.  Local probabilistic deduction from taxonomic and probabilistic knowledge-bases over conjunctive events , 1999, Int. J. Approx. Reason..

[35]  W. Ertel,et al.  Reasoning with Probabilities and Maximum Entropy: The System PIT and its Application in LEXMED , 2000 .