The Role of Numeracy in Understanding the Benefit of Screening Mammography

Patients are increasingly being exposed to quantitative information about risks for disease and benefits of treatment. Many authors [1-6] believe that the use of such information is an important component of informed decision making; others claim that only patients given such information can make truly informed choices. This position is reflected in the recent decision of the National Institutes of Health Consensus Panel not to make a recommendation about screening mammography for women aged 40 to 49 years but to advocate that these women make their own decisions about screening on the basis of their personal evaluation of risks and benefits [7]. This strategy, however, is based on the assumption that patients understand quantitative information. Research in this area has largely focused on how simple changes in the format of numerical information can influence choices, a concept referred to as framing [8-20]. But just as it is premature to worry about wording before knowing whether patients can read, it may be premature to worry about framing before knowing whether patients can understand and manipulate numbers. In fact, evidence suggests that many persons do not work well with numbers [21, 22]. It is likely that quantitative information is only meaningful to the extent that patients have some facility with basic probability and numerical concepts, a construct called numeracy. To learn more about the role of numeracy in communicating information about risk, we studied women's comprehension of messages about mammography. Our goal was to understand how numeracy affects women's ability to gauge the benefit of mammography after receiving quantitative information. We provided women with typical risk reduction expressions about mammography, framed in relative and absolute terms, to study how well they could make use of such information. We hypothesized that the ability to use quantitative risk information would be related to the level of numeracy. Methods Study Design and Sample In December 1995, we drew a simple random sample of 500 women from a registry of female veterans maintained at the Department of Veterans Affairs Medical Center, White River Junction, Vermont. These women were mailed one of four questionnaires, which differed only in how the same information on average risk reduction with mammography was presented. Reminder letters were sent to nonrespondents after 2 weeks, and new copies of the questionnaire were mailed after 4 weeks. The study was closed in February 1996. Because of errors in the registry, 26 entries in our sample were not usable (10 of the listed persons were deceased, and 16 either were not female veterans or were listed twice). Thus, our possible respondent pool totaled 474, of which 302 (64%) returned the questionnaire. The returned surveys sometimes contained unanswered questions. Such blanks may represent information about the respondent's abilities, or they may be a marker of an unmotivated respondent or one who has become fatigued from the survey burden. To remove those who were unmotivated or fatigued, we required completion of four of the five questions on the last page of the survey for inclusion in the analyses. Only 15 women did not meet this criterion. Thus, our final sample comprised 287 women, for a 61% completion rate. Unanswered numeracy questions were considered wrong answers. Restricting the sample to women who answered these questions yielded results similar to those seen with the entire sample. Questionnaire Design Assessment of Numeracy Numeracy was assessed with three questions and was scored as the total number of correct responses. The first question assessed basic familiarity with probability: Imagine that we flip a fair coin 1,000 times. What is your best guess about how many times the coin would come up heads in 1,000 flips? ____times out of 1,000. The second question asked respondents to convert a percentage (1%) to a proportion (10 in 1000): In the BIG BUCKS LOTTERY, the chance of winning a $10 prize is 1%. What is your best guess about how many people would win a $10 prize if 1000 people each buy a single ticket to BIG BUCKS?____person(s) out of 1,000. The third question reversed this task, asking the respondent to convert a proportion (1 in 1000) to a percentage (0.1%): In ACME PUBLISHING SWEEPSAKES, the chance of winning a car is 1 in 1,000. What percent of tickets to ACME PUBLISHING SWEEPSAKES win a car?____%. Presentation of Risk Reduction Data We randomly assigned each of the 500 women to receive one of four questionnaires; the questionnaires differed only in how the information on risk reduction was framed (Figure 1). Risk reduction data were framed as a 33% relative risk reduction, a 33% relative risk reduction together with the baseline risk for death from breast cancer in the next 10 years, a 4 in 1000 absolute risk reduction, or a 4 in 1000 absolute risk reduction together with the baseline risk for death from breast cancer in the next 10 years. Figure 1. Overview of the task presented to a woman completing the survey and our measures of her ability to accurately apply risk reduction information. Assessment of Perceived Benefit To assess their perceived risk for death from breast cancer with and without mammography (Figure 1), women were asked to do the following: Imagine 1,000 women exactly like you. Of these women, what is your best guess about how many will die from breast cancer during the next 10 years if they are not screened every year for breast cancer by mammogram _____out of 1000 they are screened every year for breast cancer by mammogram _____out of 1000 Calculation of Accuracy To see how accurately respondents applied the risk reduction data that they were given, we compared their perceived risk for death from breast cancer with mammography with their perceived risk for death without mammography (Figure 1). Accuracy was judged by the ability to adjust perceived risk in accordance with the risk reduction data presented. Accuracy was judged solely by the change between perceived risk with mammography and perceived risk without mammography, which we calculated from the responses to these two questions. Thus, women could grossly overestimate their risk while still accurately applying the risk reduction information. In fact, many women did overestimate their perceived risk without mammography, as observed in previous work [23]. For the two groups presented with the absolute risk reduction, we subtracted each woman's perceived risk with mammography from her perceived risk without mammography. For example, a woman who perceived her risk to be 100 out of 1000 without mammography and 96 out of 1000 with mammography indicated an absolute risk reduction of 4 out of 1000. Because the women in these groups were told that mammography decreases the risk for death from breast cancer by 4 in 1000, only women whose responses indicated an absolute risk reduction of 4 in 1000 were judged to be accurate. For the two groups presented with the relative risk reduction, we determined the percentage reduction between the perceived risk without mammography and the perceived risk with mammography. For example, a woman with a perceived risk of 100 out of 1000 without mammography and 67 out of 1000 with mammography indicated a 33% risk reduction. The women in these groups were told that mammography decreases the risk for death from breast cancer by 33%. To allow for rounding off, women whose responses indicated a relative risk reduction between 30% and 40% were judged to be accurate. Statistical Analysis For the assessment of both numeracy and accuracy, the percentage of the sample that had correct answers for each measure was calculated as the number of respondents with correct responses divided by the total number of respondents (n = 287). Chi-square tests and Kruskal-Wallis tests were used to compare participant characteristics across the four groups. All comparisons were two-sided and were considered statistically significant at a P value of less than 0.05. We used multiple logistic regression to assess the relation between accuracy in applying the risk reduction information (dependent variable) and numeracy (independent variable) after adjusting for age, income, education, and the framing of the information provided. Role of the Funding Sponsor This research was funded by the Department of Veterans Affairs Fellowship Program in Ambulatory Care and the Department of Defense's Breast Cancer Research Program. Neither department had any role in the gathering, analysis, or interpretation of the data or in deciding whether to submit the report for publication. Results Study Sample Overall, the study sample consisted of older female veterans, almost all of whom had at least a high school education. Most reported having annual household incomes of less than $25 000, and fewer than one fourth were currently employed (Table 1). Most reported having had at least one mammogram, and 9% had a history of breast cancer. No significant differences were seen in any characteristic across the four groups. Table 1. Characteristics of the Study Sample* Assessment of Numeracy Almost half of the women (46%) answered the coin-flip question (which asked how many times a coin would come up heads in 1000 flips) incorrectly, raising questions about basic understanding of probability. Incorrect answers ranged from 0 to 800 and were largely underestimates (one third of the sample thought that 1000 flips of a fair coin would result in <300 heads). The most common incorrect answers were 25, 50, and 250. Women also had trouble converting between percentages and probability expressions. The difficulty was greater for the conversion of 1 in 1000 to 0.1% than for the conversion of 1% to 10 in 1000. Although 46% of 287 respondents were unable to convert 1% to 10 in 1000, 80% were unable to convert 1 in 1000 to 0.1%. The total number of correct responses to these three simple tasks were as follows