The impact of overconfidence bias on practical accuracy of Bayesian network models: an empirical study

In this paper, we examine the influence of overconfidence in parameter specification on the performance of a Bayesian network model in the context of HEPAR II, a sizeable Bayesian network model for diagnosis of liver disorders. We enter noise in the parameters in such a way that the resulting distributions become biased toward extreme probabilities. We believe that this offers a systematic way of modeling expert overconfidence in probability estimates. It appears that the diagnostic accuracy of HEPAR II is less sensitive to overconfidence in probabilities than it is to underconfidence and to random noise, especially when noise is very large.

[1]  D. Heckerman,et al.  Probabilistic diagnosis using a reformulation of the INTERNIST-1/QMR knowledge base. II. Evaluation of diagnostic performance. , 1991, Methods of information in medicine.

[2]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[3]  Marek J. Druzdzel,et al.  Are Bayesian Networks Sensitive to Precision of Their Parameters , 2008 .

[4]  Frank Rijmen,et al.  Bayesian networks with a logistic regression model for the conditional probabilities , 2008, Int. J. Approx. Reason..

[5]  Eric Horvitz,et al.  Decision Analysis and Expert Systems , 1991, AI Mag..

[6]  Silja Renooij,et al.  Analysing Sensitivity Data from Probabilistic Networks , 2001, UAI.

[7]  Linda C. van der Gaag,et al.  Properties of Sensitivity Analysis of Bayesian Belief Networks , 2002, Annals of Mathematics and Artificial Intelligence.

[8]  Max Henrion,et al.  Uncertainty: A Guide to Dealing with Uncertainty in Quantitative Risk and Policy Analysis , 1990 .

[9]  D. Heckerman,et al.  ,81. Introduction , 2022 .

[10]  Marek J. Druzdzel,et al.  Learning Bayesian network parameters from small data sets: application of Noisy-OR gates , 2001, Int. J. Approx. Reason..

[11]  Gregory M. Provan,et al.  The Sensitivity of Belief Networks to Imprecise Probabilities: An Experimental Investigation , 1996, Artif. Intell..

[12]  Marek J. Druzdzel,et al.  An Experimental Comparison of Methods for Handling Incomplete Data in Learning Parameters of Bayesian Networks , 2002, Intelligent Information Systems.

[13]  Russell G. Almond,et al.  Modeling Diagnostic Assessments with Bayesian Networks , 2007 .

[14]  Oscar Kipersztok,et al.  Another look at sensitivity of Bayesian networks to imprecise probabilities , 2001, AISTATS.

[15]  Hanna Wasyluk,et al.  Extension of the HEPAR II Model to Multiple-Disorder Diagnosis , 2000, Intelligent Information Systems.

[16]  A. Onisko.  Effect of Imprecision in Probabilities on Bayesian Network Models : An Empirical Study , 2003 .

[17]  Russell G. Almond,et al.  Models for Conditional Probability Tables in Educational Assessment , 2001, AISTATS.