Impact of precision of Bayesian network parameters on accuracy of medical diagnostic systems

OBJECTIVE One of the hardest technical tasks in employing Bayesian network models in practice is obtaining their numerical parameters. In the light of this difficulty, a pressing question, one that has immediate implications on the knowledge engineering effort, is whether precision of these parameters is important. In this paper, we address experimentally the question whether medical diagnostic systems based on Bayesian networks are sensitive to precision of their parameters. METHODS AND MATERIALS The test networks include Hepar II, a sizeable Bayesian network model for diagnosis of liver disorders and six other medical diagnostic networks constructed from medical data sets available through the Irvine Machine Learning Repository. Assuming that the original model parameters are perfectly accurate, we lower systematically their precision by rounding them to progressively courser scales and check the impact of this rounding on the models' accuracy. RESULTS Our main result, consistent across all tested networks, is that imprecision in numerical parameters has minimal impact on the diagnostic accuracy of models, as long as we avoid zeroes among parameters. CONCLUSION The experiments' results provide evidence that as long as we avoid zeroes among model parameters, diagnostic accuracy of Bayesian network models does not suffer from decreased precision of their parameters.

[1]  Riccardo Bellazzi,et al.  Dynamic Bayesian Networks in Modelling Cellular Systems: a Critical Appraisal on Simulated Data , 2006, 19th IEEE Symposium on Computer-Based Medical Systems (CBMS'06).

[2]  D. Ayres-de- Campos,et al.  SisPorto 2.0: a program for automated analysis of cardiotocograms. , 2000, The Journal of maternal-fetal medicine.

[3]  Linda C. van der Gaag,et al.  Making Sensitivity Analysis Computationally Efficient , 2000, UAI.

[4]  Ivan Bratko,et al.  ASSISTANT 86: A Knowledge-Elicitation Tool for Sophisticated Users , 1987, EWSL.

[5]  D. Heckerman,et al.  Toward Normative Expert Systems: Part I The Pathfinder Project , 1992, Methods of Information in Medicine.

[6]  R. Likert.  “Technique for the Measurement of Attitudes, A” , 2022, The SAGE Encyclopedia of Research Design.

[7]  Peter J. F. Lucas,et al.  A probabilistic and decision-theoretic approach to the management of infectious disease at the ICU , 2000, Artif. Intell. Medicine.

[8]  José Mira Mira,et al.  NasoNet, modeling the spread of nasopharyngeal cancer with networks of probabilistic events in discrete time , 2002, Artif. Intell. Medicine.

[9]  Niels Peek,et al.  Using sensitivity analysis for efficient quantification of a belief network , 1999, Artif. Intell. Medicine.

[10]  G. Gorry,et al.  Experience with a model of sequential diagnosis. , 2011, Computers and biomedical research, an international journal.

[11]  Lukasz A. Kurgan,et al.  Knowledge discovery approach to automated cardiac SPECT diagnosis , 2001, Artif. Intell. Medicine.

[12]  Tze-Yun Leong,et al.  Multiple Perspective Dynamic Decision Making , 1998, Artif. Intell..

[13]  Marek J. Druzdzel,et al.  SMILE: Structural Modeling, Inference, and Learning Engine and GeNIE: A Development Environment for Graphical Decision-Theoretic Models , 1999, AAAI/IAAI.

[14]  J. Czerniak,et al.  Application of rough sets in the presumptive diagnosis of urinary system diseases , 2003 .

[15]  Max Henrion,et al.  Uncertainty: A Guide to Dealing with Uncertainty in Quantitative Risk and Policy Analysis , 1990 .

[16]  Marek J. Druzdzel,et al.  The impact of overconfidence bias on practical accuracy of Bayesian network models: an empirical study , 2008, BMA.

[17]  Gregory M. Provan,et al.  The Sensitivity of Belief Networks to Imprecise Probabilities: An Experimental Investigation , 1996, Artif. Intell..

[18]  D. Heckerman,et al.  ,81. Introduction , 2022 .

[19]  Peter J. F. Lucas,et al.  Dynamic Bayesian networks as prognostic models for clinical patient management , 2008, J. Biomed. Informatics.

[20]  Eric Horvitz,et al.  Decision Analysis and Expert Systems , 1991, AI Mag..

[21]  Cungen Cao,et al.  Dynamic decision analysis in medicine: a data-driven approach , 1998, Int. J. Medical Informatics.

[22]  Silja Renooij,et al.  Analysing Sensitivity Data from Probabilistic Networks , 2001, UAI.

[23]  R S LEDLEY,et al.  Reasoning foundations of medical diagnosis; symbolic logic, probability, and value theory aid our understanding of how physicians reason. , 1959, Science.

[24]  Gregory F. Cooper,et al.  NESTOR: A Computer-Based Medical Diagnostic Aid That Integrates Causal and Probabilistic Knowledge. , 1984 .

[25]  Andrew W. Moore,et al.  Efficient Algorithms for Minimizing Cross Validation Error , 1994, ICML.

[26]  Silja Renooij,et al.  Probabilities for a probabilistic network: a case study in oesophageal cancer , 2002, Artif. Intell. Medicine.

[27]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[28]  Hanna Wasyluk,et al.  Extension of the HEPAR II Model to Multiple-Disorder Diagnosis , 2000, Intelligent Information Systems.

[29]  Gregory F. Cooper,et al.  The ALARM Monitoring System: A Case Study with two Probabilistic Inference Techniques for Belief Networks , 1989, AIME.

[30]  Bruce G. Buchanan,et al.  The MYCIN Experiments of the Stanford Heuristic Programming Project , 1985 .

[31]  Steen Andreassen,et al.  MUNIN - A Causal Probabilistic Network for Interpretation of Electromyographic Findings , 1987, IJCAI.

[32]  Frank Rijmen,et al.  Bayesian networks with a logistic regression model for the conditional probabilities , 2008, Int. J. Approx. Reason..

[33]  A. Tversky,et al.  Judgment under Uncertainty: Heuristics and Biases , 1974, Science.

[34]  Peter J. F. Lucas,et al.  Bayesian networks in biomedicine and health-care , 2004, Artif. Intell. Medicine.

[35]  D. Heckerman,et al.  Probabilistic diagnosis using a reformulation of the INTERNIST-1/QMR knowledge base. II. Evaluation of diagnostic performance. , 1991, Methods of information in medicine.

[36]  Linda C. van der Gaag,et al.  Properties of Sensitivity Analysis of Bayesian Belief Networks , 2002, Annals of Mathematics and Artificial Intelligence.

[37]  Marek J. Druzdzel,et al.  Verbal expressions of probability , 2009 .

[38]  Russell G. Almond,et al.  Modeling Diagnostic Assessments with Bayesian Networks , 2007 .

[39]  L. C. van der Gaag,et al.  Practicable sensitivity analysis of Bayesian belief networks , 1998 .

[40]  Adnan Darwiche,et al.  When do Numbers Really Matter? , 2001, UAI.

[41]  Pukelsheim Friedrich,et al.  On stationary multiplier methods for the rounding of probabilities and the limiting law of the Sainte-Laguë divergence , 2005 .

[42]  Igor Kononenko,et al.  Inductive and Bayesian learning in medical diagnosis , 1993, Appl. Artif. Intell..

[43]  Frederick Mosteller,et al.  The Distribution of Sums of Rounded Percentages , 1967 .

[44]  José Mira Mira,et al.  DIAVAL, a Bayesian expert system for echocardiography , 1997, Artif. Intell. Medicine.

[45]  H. Peyton Young,et al.  Fair Representation: Meeting the Ideal of One Man, One Vote , 1982 .

[46]  Marek J. Druzdzel,et al.  Learning Bayesian network parameters from small data sets: application of Noisy-OR gates , 2001, Int. J. Approx. Reason..

[47]  Marek J. Druzdzel,et al.  An Experimental Comparison of Methods for Handling Incomplete Data in Learning Parameters of Bayesian Networks , 2002, Intelligent Information Systems.

[48]  Marek J. Druzdzel,et al.  Are Bayesian Networks Sensitive to Precision of Their Parameters , 2008 .

[49]  Russell G. Almond,et al.  Models for Conditional Probability Tables in Educational Assessment , 2001, AISTATS.

[50]  Edward H. Shortliffe,et al.  Rule Based Expert Systems: The Mycin Experiments of the Stanford Heuristic Programming Project (The Addison-Wesley series in artificial intelligence) , 1984 .

[51]  Evert de Jonge,et al.  Using hierarchical dynamic Bayesian networks to investigate dynamics of organ failure in patients in the Intensive Care Unit , 2010, J. Biomed. Informatics.