Propagating Imprecise Probabilities in Bayesian Networks

Abstract Often experts are incapable of providing “exact” probabilities; likewise, samples on which the probabilities in networks are based must often be small and preliminary. In such cases the probabilities in the networks are imprecise. The imprecision can be handled by second order probability distributions. It is convenient to use beta or Dirichlet distributions to express the uncertainty about probabilities. The problem of how to propagate point probabilities in a Bayesian network now is transformed into the problem of how to propagate Dirichlet distributions in Bayesian networks. It is shown that the propagation of Dirichlet distributions in Bayesian networks with incomplete data results in a system of probability mixtures of beta-binomial and Dirichlet distributions. Approximate first order probabilities and their second order probability density functions are obtained by stochastic simulation. A number of properties of the propagation of imprecise probabilities are discussed by the use of examples. An important property is that the imprecision of inferences increases rapidly as new premises are added to an argument. The imprecision can be used as a pruning criterion in a network to keep the number of variables involved in an inferential argument small. Thus, imprecision may be used as an Ockam's razor in Bayesian networks.

[1]  Stephen E. Fienberg,et al.  Discrete Multivariate Analysis: Theory and Practice , 1976 .

[2]  Andrew R. Runnalls A Survey of Sampling Methods for Inference on Directed Graphs , 1994 .

[3]  Gernot D. Kleiter,et al.  Bayesian Diagnosis in Expert Systems , 1992, Artif. Intell..

[4]  Wray L. Buntine Operations for Learning with Graphical Models , 1994, J. Artif. Intell. Res..

[5]  Gerhard H. Fischer,et al.  "Contributions to Mathematical Psychology, Psychometrics, and Methodology" , 1993 .

[6]  S. Shen,et al.  The statistical analysis of compositional data , 1983 .

[7]  David Maxwell Chickering,et al.  Learning Bayesian networks: The combination of knowledge and statistical data , 1995, Mach. Learn..

[8]  Wai Lam,et al.  LEARNING BAYESIAN BELIEF NETWORKS: AN APPROACH BASED ON THE MDL PRINCIPLE , 1994, Comput. Intell..

[9]  Eric R. Ziecel Selecting Models From Data , 1995 .

[10]  Roderick J. A. Little,et al.  Statistical Analysis with Missing Data , 1988 .

[11]  John Aitchison,et al.  Statistical Prediction Analysis , 1975 .

[12]  R. F.,et al.  Mathematical Statistics , 1944, Nature.

[13]  Andrew P. Sage,et al.  Uncertainty in Artificial Intelligence , 1987, IEEE Transactions on Systems, Man, and Cybernetics.

[14]  Gregory F. Cooper,et al.  A randomized approximation algorithm for probabilistic inference on bayesian belief networks , 1990, Networks.

[15]  P. Walley Statistical Reasoning with Imprecise Probabilities , 1990 .

[16]  Richard E. Neapolitan,et al.  Probabilistic reasoning in expert systems - theory and algorithms , 2012 .

[17]  G. Oehlert A note on the delta method , 1992 .

[18]  Ronald R. Yager,et al.  Uncertainty in Knowledge Bases , 1990, Lecture Notes in Computer Science.

[19]  John S. Breese,et al.  Interval Influence Diagrams , 1989, UAI.

[20]  Giulianella Coletti,et al.  Conditional Events with Vague Information in Expert Systems , 1990, IPMU.

[21]  Gerhard Paass Second order probabilities for uncertain and conflicting evidence , 1990, UAI.

[22]  Norman L. Johnson,et al.  Urn models and their application , 1977 .

[23]  Richard E. Neapolitan,et al.  An Implementation of a Method for Computing the Uncertainty in Inferred Probabilities in Belief Networks , 1993, UAI.

[24]  David J. Spiegelhalter,et al.  Assessment, Criticism and Improvement of Imprecise Subjective Probabilities for a Medical Expert System , 2013, UAI.

[25]  Gernot D. Kleiter,et al.  A Bayesian approach to imprecision in belief nets , 1995 .

[26]  Petr Hájek,et al.  Uncertain information processing in expert systems , 1992 .

[27]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems , 1988 .

[28]  J. Keynes A Treatise on Probability. , 1923 .

[29]  Gernot D. Kleiter,et al.  THE PRECISION OF BAYESIAN CLASSIFICATION: THE MULTIVARIATE NORMAL CASE , 1993 .

[30]  Wray L. Buntine A Guide to the Literature on Learning Probabilistic Networks from Data , 1996, IEEE Trans. Knowl. Data Eng..

[31]  Ronald R. Yager,et al.  Uncertainty in Intelligent Systems , 1993 .

[32]  Richard E. Neapolitan,et al.  Investigation of Variances in Belief Networks , 1991, UAI.

[33]  J. N. R. Jeffers,et al.  Graphical Models in Applied Multivariate Statistics. , 1990 .

[34]  N. Wermuth,et al.  Graphical Models for Associations between Variables, some of which are Qualitative and some Quantitative , 1989 .

[35]  Adrian F. M. Smith,et al.  Sampling-Based Approaches to Calculating Marginal Densities , 1990 .

[36]  John Aitchison,et al.  The Statistical Analysis of Compositional Data , 1986 .

[37]  Tomas Hrycej,et al.  Gibbs Sampling in Bayesian Networks , 1990, Artif. Intell..

[38]  David J. Spiegelhalter,et al.  Bayesian analysis in expert systems , 1993 .

[39]  David J. Spiegelhalter,et al.  Probabilistic Reasoning in Predictive Expert Systems , 1985, UAI.

[40]  D. Edwards Introduction to graphical modelling , 1995 .

[41]  Gernot D. Kleiter,et al.  Expressing imprecision in probabilistic knowledge , 1993 .

[42]  Upendra Dave,et al.  Probabilistic Reasoning and Bayesian Belief Networks , 1996 .

[43]  Gernot D. Kleiter,et al.  Natural Sampling: Rationality without Base Rates , 1994 .

[44]  D. Rubin,et al.  Statistical Analysis with Missing Data , 1988 .

[45]  David J. Spiegelhalter,et al.  A Unified Approach to Imprecision and Sensitivity of Beliefs in Expert Systems , 1987, Conference on Uncertainty in Artificial Intelligence.