Medicine in words and numbers: a cross-sectional survey comparing probability assessment scales

BackgroundIn the complex domain of medical decision making, reasoning under uncertainty can benefit from supporting tools. Automated decision support tools often build upon mathematical models, such as Bayesian networks. These networks require probabilities which often have to be assessed by experts in the domain of application. Probability response scales can be used to support the assessment process. We compare assessments obtained with different types of response scale.MethodsGeneral practitioners (GPs) gave assessments on and preferences for three different probability response scales: a numerical scale, a scale with only verbal labels, and a combined verbal-numerical scale we had designed ourselves. Standard analyses of variance were performed.ResultsNo differences in assessments over the three response scales were found. Preferences for type of scale differed: the less experienced GPs preferred the verbal scale, the most experienced preferred the numerical scale, with the groups in between having a preference for the combined verbal-numerical scale.ConclusionWe conclude that all three response scales are equally suitable for supporting probability assessment. The combined verbal-numerical scale is a good choice for aiding the process, since it offers numerical labels to those who prefer numbers and verbal labels to those who prefer words, and accommodates both more and less experienced professionals.

[1]  Wibecke Brun,et al.  Verbal probabilities: Ambiguous, context-dependent, or both? , 1988 .

[2]  Silja Renooij,et al.  Talking probabilities: communicating probabilistic information with words and numbers , 1999, Int. J. Approx. Reason..

[3]  Thomas S. Wallsten,et al.  Base rate effects on the interpretations of probability and frequency expressions , 1986 .

[4]  A. Tversky,et al.  Support theory: A nonextensional representation of subjective probability. , 1994 .

[5]  Philip S. Yu.  Editorial: New AE Introduction , 2003, IEEE Trans. Knowl. Data Eng..

[6]  A. Hasman,et al.  Probabilistic reasoning in intelligent systems: Networks of plausible inference , 1991 .

[7]  Silja Renooij,et al.  Probability elicitation for belief networks: issues to consider , 2001, The Knowledge Engineering Review.

[8]  Steen Andreassen,et al.  MUNIN - A Causal Probabilistic Network for Interpretation of Electromyographic Findings , 1987, IJCAI.

[9]  Benjamin Kuipers,et al.  Critical Decisions under Uncertainty: Representation and Structure , 1990, Cogn. Sci..

[10]  Linda C. van der Gaag,et al.  Sensitivity analysis: an aid for belief-network quantification , 2000, The Knowledge Engineering Review.

[11]  L. C. van der Gaag,et al.  Building probabilistic networks: Where do the numbers come from? - a guide to the literature , 2000 .

[12]  B Fischhoff,et al.  A New Scale for Assessing Perceptions of Chance , 2000, Medical decision making : an international journal of the Society for Medical Decision Making.

[13]  Silja Renooij,et al.  Probabilities for a probabilistic network: a case study in oesophageal cancer , 2002, Artif. Intell. Medicine.

[14]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[15]  B. Fischhoff,et al.  Fault Trees: Sensitivity of Estimated Failure Probabilities to Problem Representation , 2005 .

[16]  O Manor,et al.  Probabilistic reasoning and clinical decision-making: do doctors overestimate diagnostic probabilities? , 2003, QJM : monthly journal of the Association of Physicians.

[17]  D. Budescu,et al.  Preferences and reasons for communicating probabilistic information in verbal or numerical terms , 1993 .

[18]  José Mira Mira,et al.  DIAVAL, a Bayesian expert system for echocardiography , 1997, Artif. Intell. Medicine.

[19]  B. Kuipers,et al.  Critical Decisions under Uncertainty: Representation and Structure , 1990, Cogn. Sci..

[20]  Yuval Rottenstreich,et al.  Typical versus atypical unpacking and superadditive probability judgment. , 2004, Journal of experimental psychology. Learning, memory, and cognition.

[21]  Marek J Druzdzel,et al.  Verbal Expressions of Probability in Informed Consent Litigation , 1991, Medical decision making : an international journal of the Society for Medical Decision Making.

[22]  Wibecke Brun,et al.  Verbal Expressions of Uncertainty and Probability , 2005 .

[23]  P. Lucas,et al.  Computer-based Decision Support in the Management of Primary Gastric non-Hodgkin Lymphoma , 1998, Methods of Information in Medicine.

[24]  I. Erev,et al.  Verbal versus numerical probabilities: Efficiency, biases, and the preference paradox☆ , 1990 .

[25]  Carla P. Gomes,et al.  Artificial intelligence and operations research: challenges and opportunities in planning and scheduling , 2000, The Knowledge Engineering Review.

[26]  J. Kassirer,et al.  The threshold approach to clinical decision making. , 1980, The New England journal of medicine.

[27]  L. Macchi,et al.  Thinking : psychological perspectives on reasoning, judgment and decision making , 2005 .

[28]  A. Tversky,et al.  Judgment under Uncertainty: Heuristics and Biases , 1974, Science.

[29]  Gregory M. Provan,et al.  Why is diagnosis using belief networks insensitive to imprecision in probabilities? , 1996, UAI.

[30]  Carl-Axel S. Staël von Holstein,et al.  Exceptional Paper---Probability Encoding in Decision Analysis , 1975 .

[31]  Helder Coelho,et al.  A multi-agent intelligent environment for medical knowledge , 2003, Artif. Intell. Medicine.

[32]  P J Lucas,et al.  Converting a rule-based expert system into a belief network. , 1993, Medical informatics = Medecine et informatique.

[33]  J Kievit,et al.  How do surgeons' probability estimates of operative mortality compare with a decision analytic model? , 1996, Acta psychologica.

[34]  Linda C. van der Gaag,et al.  Developing a Bayesian Network for Clinical Diagnosis in Veterinary Medicine : from the Individual to the Herd , .

[35]  Jeremy E. Oakley,et al.  Uncertain Judgements: Eliciting Experts' Probabilities , 2006 .

[36]  Peter J. F. Lucas,et al.  A dynamic Bayesian network for diagnosing ventilator-associated pneumonia in ICU patients , 2009, Expert Syst. Appl..

[37]  Silja Renooij,et al.  Evaluation of a verbal-numerical probability scale , 2003, Int. J. Approx. Reason..

[38]  D A Redelmeier,et al.  Probability judgement in medicine: discounting unspecified possibilities. , 1995, Medical decision making : an international journal of the Society for Medical Decision Making.

[39]  J. Blanc,et al.  Fusion of expert knowledge with data using belief functions: a case study in waste-water treatment , 2002, Proceedings of the Fifth International Conference on Information Fusion. FUSION 2002. (IEEE Cat.No.02EX5997).

[40]  大西 仁,et al.  Pearl, J. (1988, second printing 1991). Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference. Morgan-Kaufmann. , 1994 .