Improving Accuracy on Bayesian Inference Problems Using a Brief Tutorial

Research suggests that most people struggle when asked to interpret the outcomes of diagnostic tests such as those presented as Bayesian inference problems. To help people interpret these difficult problems, we created a brief tutorial, requiring less than 10 minutes, that guided participants through the creation of an aid (either graph or table) based on an example inference problem and then showed the correct way to calculate the positive predictive value of the problem (i.e., likelihood that positive tests correctly indicate presence of condition). Approximately 70% of those in each training condition found the correct response on at least one problem in the format for which they were trained. Just under 55% of those in the control condition (i.e., no training) were able to find the correct response on at least one table or graph problem. We demonstrated a relationship between numeracy and performance on both problem formats, although we did not find evidence for a relationship between graph literacy and performance for either problem format. Potential improvements to and applications of the tutorial are discussed. Copyright © 2016 John Wiley & Sons, Ltd.

[1]  Edward T. Cokely,et al.  Individual Differences in Graph Literacy: Overcoming Denominator Neglect in Risk Comprehension , 2012 .

[2]  Gerd Gigerenzer,et al.  How to Improve Bayesian Reasoning Without Instruction: Frequency Formats , 1995 .

[3]  C. K. Mertz,et al.  PSYCHOLOGICAL SCIENCE Research Article Numeracy and Decision Making , 2022 .

[4]  Edward T. Cokely,et al.  Science Current Directions in Psychological , 2010 .

[5]  P. Ubel,et al.  Measuring Numeracy without a Math Test: Development of the Subjective Numeracy Scale , 2007, Medical decision making : an international journal of the Society for Medical Decision Making.

[6]  M. Siegrist,et al.  How do people perceive graphical risk communication? The role of subjective numeracy , 2011 .

[7]  Gerd Gigerenzer,et al.  Teaching Bayesian reasoning in less than two hours. , 2001, Journal of experimental psychology. General.

[8]  Victor M Montori,et al.  Perspective: The Role of Numeracy in Health Care , 2008, Journal of health communication.

[9]  Eduardo Salas,et al.  The Transfer of Training: What Really Matters , 2011 .

[10]  Timothy T. Baldwin,et al.  Transfer of Training: A Meta-Analytic Review , 2010 .

[11]  P. Ubel,et al.  The impact of the format of graphical presentation on health-related knowledge and treatment choices. , 2008, Patient education and counseling.

[12]  Lisa M. Schwartz,et al.  PSYCHOLOGICAL SCIENCE IN THE PUBLIC INTEREST Helping Doctors and Patients Make Sense of Health Statistics , 2022 .

[13]  Olena A. Cherkasky,et al.  Should Health Numeracy Be Assessed Objectively or Subjectively? , 2016, Medical decision making : an international journal of the Society for Medical Decision Making.

[14]  P. Kooreman,et al.  Flu Shots, Mammograms, and the Perception of Probabilities , 2010, SSRN Electronic Journal.

[15]  P. MacEneaney,et al.  The meaning of diagnostic test results: a spreadsheet for swift data analysis. , 2000, Clinical radiology.

[16]  Beat Kleiner,et al.  A Mosaic of Television Ratings , 1984 .

[17]  V. Reyna,et al.  How numeracy influences risk comprehension and medical decision making. , 2009, Psychological bulletin.

[18]  Angela Fagerlin,et al.  Clinical Implications of Numeracy: Theory and Practice , 2008, Annals of behavioral medicine : a publication of the Society of Behavioral Medicine.

[19]  Masako Okamoto,et al.  Health numeracy in Japan: measures of basic numeracy account for framing bias in a highly numerate population , 2012, BMC Medical Informatics and Decision Making.

[20]  B. Rimer,et al.  General Performance on a Numeracy Scale among Highly Educated Samples , 2001, Medical decision making : an international journal of the Society for Medical Decision Making.

[21]  L. Cosmides,et al.  Are humans good intuitive statisticians after all? Rethinking some conclusions from the literature on judgment under uncertainty , 1996, Cognition.

[22]  Valerie F. Reyna,et al.  Improving communication of breast cancer recurrence risk , 2012, Breast Cancer Research and Treatment.

[23]  Miroslav Sirota,et al.  The effect of iconicity of visual displays on statistical reasoning: evidence in favor of the null hypothesis , 2014, Psychonomic bulletin & review.

[24]  Gerd Gigerenzer,et al.  Natural Frequencies Help Older Adults and People with Low Numeracy to Evaluate Medical Screening Tests , 2009, Medical decision making : an international journal of the Society for Medical Decision Making.

[25]  Ulrich Hoffrage,et al.  Teaching Bayesian reasoning: an evaluation of a classroom tutorial for medical students , 2002, Medical teacher.

[26]  W. H. Finch,et al.  Speaking and interruptions during primary care office visits. , 2001, Family medicine.

[27]  Ulrich Hoffrage,et al.  Visual representation of statistical information improves diagnostic inferences in doctors and their patients. , 2013, Social science & medicine.

[28]  G Gigerenzer,et al.  Using natural frequencies to improve diagnostic inferences , 1998, Academic medicine : journal of the Association of American Medical Colleges.

[29]  D. Eddy.  Judgment under uncertainty: Probabilistic reasoning in clinical medicine: Problems and opportunities , 1982 .

[30]  W. Gaissmaier,et al.  Numbers can be worth a thousand pictures: individual differences in understanding graphical and numerical representations of health-related information. , 2012, Health psychology : official journal of the Division of Health Psychology, American Psychological Association.

[31]  Schonlau Matthias,et al.  Comprehension of Graphs and Tables Depend on the Task: Empirical Evidence from Two Web-Based Studies , 2012 .

[32]  M. Galesic,et al.  Graph Literacy , 2011, Medical decision making : an international journal of the Society for Medical Decision Making.

[33]  M K Marvel,et al.  Soliciting the patient's agenda: have we improved? , 1999, JAMA.

[34]  Susan N. Friel,et al.  Making sense of graphs: Critical factors influencing comprehension and instructional implications. , 2001 .

[35]  G. Guyatt,et al.  Users' Guides to the Medical Literature: III. How to Use an Article About a Diagnostic Test: B. What Are the Results and Will They Help Me In Caring for My Patients? , 1994 .

[36]  Joachim Meyer,et al.  Multiple Factors that Determine Performance with Tables and Graphs , 1997, Hum. Factors.

[37]  Jessica S. Ancker,et al.  The Practice of Informatics: Design Features of Graphs in Health Risk Communication: A Systematic Review , 2006, J. Am. Medical Informatics Assoc..

[38]  Mirta Galesic,et al.  How to Reduce the Effect of Framing on Messages About Health , 2010, Journal of General Internal Medicine.

[39]  Heike Hofmann,et al.  Visualizing association rules with interactive mosaic plots , 2000, KDD '00.

[40]  Ellen Peters,et al.  Development and Testing of an Abbreviated Numeracy Scale: A Rasch Analysis Approach , 2012, Journal of behavioral decision making.