Evidence-Based Risk Communication

Shared decision making is a collaborative process that allows patients and medical professionals to consider the best scientific evidence available, along with patients values and preferences, to make health care decisions (1). A recent Institute of Medicine report concluded that although people desire a patient experience that includes deep engagement in shared decision making, there are gaps between what patients want and what they get (2). For patients to get the experience they want, providers must effectively communicate evidence about benefits and harms. To improve the decision-making process, the Institute of Medicine recommended development and dissemination of high-quality communication tools (2). New tools, however, must match patients numerical abilities, which are often limited. For example, in one study, as many as 40% of high school graduates could not perform basic numerical operations, such as converting 1% of 1000 to 10 of 1000. This collective statistical illiteracy is a major barrier to the interpretation of health statistics (3). Physicians may also find statistical information difficult to interpret and explain (4). Existing literature about methods of communicating benefits and harms is broad. One review, based on 19 studies, concluded that the choice of a specific graphic is not as important as whether the graphic frames the frequency of an event with a visual representation of the total population in which it occurs (5). Another review, involving a limited literature search, found that comprehension improved when using frequencies (such as 1 in 5) instead of event rates (such as 20%) and using absolute risk reductions (ARRs) instead of relative risk reductions (RRRs) (6). The review did not assess affective outcomes, such as patient satisfaction, and behavioral outcomes, such as changes in decision making. Yet another review identified strong evidence that patients misinterpret RRRs and supported the effectiveness of graphs in communicating harms (7). However, they did not examine the comparative effectiveness of such approaches. More narrowly focused Cochrane reviews examined the communication of risk specific to screening tests (8, 9); numerical presentations, such as ARRs, RRRs, and numbers needed to treat (NNTs) (10); and effects of decision aids (11). An expert commentary about effective risk communication recommended using plain language, icon arrays, and absolute risks and providing time intervals with risk information (12). A group of experts identified 11 key components of risk communication, including presenting numerical estimates in context with evaluative labels, conveying uncertainty, and tailoring estimates (13). The aim of this systematic review is to comprehensively examine the comparative effectiveness of all methods of communicating probabilistic information about benefits and harms to patients to maximize their understanding, satisfaction, and decision-making ability. Methods We developed and followed a plan for the review that included several searches and dual abstraction of study data using standardized abstraction forms. Data Sources and Study Selection We searched PubMed (1966 to March 2014), CINAHL, EMBASE, and the Cochrane Central Register of Controlled Trials (1966 to December 2011) using keywords and structured terms related to the concepts of patients; communication; riskbenefit; and outcomes, such as understanding or comprehension, preferences or satisfaction, and decision making. Supplement 1 shows the detailed search strategy. Supplement 1. Search Strategies We included cross-sectional or prospective, longitudinal trials that were published in English and had an active control group that recruited patients or healthy volunteers and compared any method of communicating probabilistic information with another method. We focused on different methods of communicating the same specific probabilities to eliminate any independent effects that could result from different probabilities being studied (for example, different magnitudes or directions of effect). Studies of personalized risks, which may vary from person to person, were included when participants were randomly assigned. When studies of personalized risks were not randomized, the risks were considered to differ between the groups and were excluded. No limits were placed on study size, location, or duration or on the nature of the communication method. When needed, we reviewed sources specified in the articles, such as Web sites, to directly review the interventions and determine whether probabilistic information was addressed. Studies of medical students, health professionals, and public health or mass media campaigns were excluded. One independent reviewer screened each title and abstract and excluded citations that were not original studies or were unrelated to probabilistic information. Two independent reviewers screened the full text of the remaining citations to identify eligible articles. Disagreements between the 2 reviewers were resolved by consensus, with a third reviewer arbitrating any unresolved disagreements. Data Extraction and Quality Assessment Two reviewers independently abstracted detailed information about the study population, interventions, primary outcomes, and risk of bias from each included study using a standardized abstraction form, which was developed a priori (Supplement 2). A third reviewer resolved any disagreements. We categorized outcomes in 1 of 3 domains: cognitive (or understanding, such as accuracy in answering questions related to probabilistic information, or general comprehension of the probabilistic information), affective (such as preferences for or satisfaction with the method of communicating probabilistic information), and behavioral (such as real or theoretical decision making). Supplement 2. Abstraction Form Risk of bias in randomized, controlled trials was assessed on the basis of adequacy of randomization, allocation concealment, similarity of study groups at baseline, blinding, equal treatment of groups throughout the study, completeness of follow-up, and intention to treat (participants analyzed in the groups to which they were randomly assigned) (14). Risk of bias in observational studies was assessed with a modified set of criteria adapted from the NewcastleOttawa Scale (15). Data Synthesis and Analysis Data were tabulated, and the frequency of all head-to-head comparisons in studies was assessed to identify clusters of comparisons. In many instances, several interventions were bundled in a single study group (such as event rate plus icon array, or event rate plus natural frequencies plus ARRs). Bundles were not separated or combined with similar interventions because it could not be determined which component of the bundle drove the intervention. Descriptive statistics were used. We decided a priori not to do meta-analysis because of study heterogeneity. We emphasized findings from randomized studies as well as nonrandomized studies when findings were supported by more than 1 study. Role of the Funding Source No funding supported this study. The authors participated within their role on the Evidence-Based Medicine Task Force of the Society of General Internal Medicine. Results The initial search through December 2011 retrieved 22103 citations (16661 from PubMed, 1194 from CINAHL, 2861 from the Cochrane Central Register of Controlled Trials, and 1387 from EMBASE), and 20076 remained after removing duplicates. We updated the PubMed search through 30 March 2014, yielding 6529 additional citations; 5970 remained after removing duplicates, for a total of 26046 citations for review. A total of 630 articles were selected for full-text review and 84 were included, representing 91 unique studies (1699). Reasons for exclusion are noted in Figure 1, and study details are provided in Supplement 3. Figure 1. Summary of evidence search and selection. Supplement 3. Details of All Included Studies Seventy-four (81.3%) of the 91 included studies were randomized trials, most with cross-sectional designs. The median number of participants in randomized trials was 268 (range, 31 to 4685), and the median in all studies was 268 (range, 24 to 16133). Thirty-three studies (36.3%) included patients at specific risk for the target condition of interest. Forty-eight studies (52.7%) presented probabilistic data about benefits of a therapy or intervention (with 7 [14.6%] also presenting harms), 21 (23.1%) presented data only on harms, and 9 (10%) involved screening tests. Forty-nine studies (54.4%) delivered interventions on paper and 39 (42.9%) on a computer, typically over the Internet. The characteristics of study participants are presented in Tables 1 and 2. Table 1. Characteristics of Study Participants Table 2. Proportion of Studies Including Participants at Risk Versus Not at Risk for Target Condition Risk of bias for the included randomized trials was moderate (Figure 2). Randomization was adequate in 32 trials (42.7%), inadequate in 3 (4.0%), and unclear in 40 (53.3%). Allocation concealment was not stated in 55 trials (73.3%). Similarity of groups at baseline was adequate in 37 trials (49.3%) and unclear in 32 (42.7%). Blinding, equal treatment, and intention-to-treat items were similarly difficult to assess from reported information. Figure 2. Risk of bias for randomized, controlled trials (n = 74). Adapted from reference 100. Study Interventions and Comparators A frequency table (heat map) of all study intervention comparisons was created to identify clusters of comparisons (Supplement 4). The heat map represents study group comparisons, so one study may contribute several comparisons. The most commonly studied numerical presentations of data were natural frequencies, defined as the numbers of persons with events juxtaposed with a baseline denominator of persons (for example, 4 out of 100 persons had the outcome); event rates, defined as the proportions of persons wi