Development and Testing of an Abbreviated Numeracy Scale: A Rasch Analysis Approach

ABSTRACT Research has demonstrated that individual differences in numeracy may have important consequences for decision making. In the present paper, we develop a shorter, psychometrically improved measure of numeracy—the ability to understand, manipulate, and use numerical information, including probabilities. Across two large independent samples that varied widely in age and educational level, participants completed 18 items from existing numeracy measures. In Study 1, we conducted a Rasch analysis on the item pool and created an eight‐item numeracy scale that assesses a broader range of difficulty than previous scales. In Study 2, we replicated this eight‐item scale in a separate Rasch analysis using data from an independent sample. We also found that the new Rasch‐based numeracy scale, compared with previous measures, could predict decision‐making preferences obtained in past studies, supporting its predictive validity. In Study, 3, we further established the predictive validity of the Rasch‐based numeracy scale. Specifically, we examined the associations between numeracy and risk judgments, compared with previous scales. Overall, we found that the Rasch‐based scale was a better linear predictor of risk judgments than prior measures. Moreover, this study is the first to present the psychometric properties of several popular numeracy measures across a diverse sample of ages and educational level. We discuss the usefulness and the advantages of the new scale, which we feel can be used in a wide range of subject populations, allowing for a more clear understanding of how numeracy is associated with decision processes. Copyright © 2012 John Wiley & Sons, Ltd.

[1]  S. Gosling,et al.  Should we trust web-based studies? A comparative analysis of six preconceptions about internet questionnaires. , 2004, The American psychologist.

[2]  J. M. Cortina,et al.  What Is Coefficient Alpha? An Examination of Theory and Applications , 1993 .

[3]  R. Thaler,et al.  Libertarian Paternalism , 2019, Encyclopedia of Law and Economics.

[4]  Peter Williams,et al.  New Rules of Measurement , 2015 .

[5]  P. Slovic,et al.  Is the informed-choice policy approach appropriate for Medicare beneficiaries? , 2001, Health affairs.

[6]  Paul Slovic,et al.  The Use of Narrative Evidence and Explicit Likelihood by Decision Makers Varying in Numeracy , 2009 .

[7]  Edward T. Cokely,et al.  Measuring Risk Literacy: The Berlin Numeracy Test , 2012, Judgment and Decision Making.

[8]  Lisa M. Schwartz,et al.  The Role of Numeracy in Understanding the Benefit of Screening Mammography , 1997, Annals of Internal Medicine.

[9]  J. Suls,et al.  Flawed Self-Assessment , 2004, Psychological science in the public interest : a journal of the American Psychological Society.

[10]  G. McCarthy,et al.  The Development of a Semi‐Structured Interview to Investigate the Attachment‐Related Experiences of Adults with Learning Disabilities , 1996 .

[11]  Judith H Hibbard,et al.  Development and testing of a short form of the patient activation measure. , 2005, Health services research.

[12]  P. Slovic,et al.  Numeracy skill and the communication, comprehension, and use of risk-benefit information. , 2007, Health affairs.

[13]  P. Ubel,et al.  Validation of the Subjective Numeracy Scale: Effects of Low Numeracy on Comprehension of Risk Communications and Utility Elicitations , 2007, Medical decision making : an international journal of the Society for Medical Decision Making.

[14]  M. R. Novick,et al.  Statistical Theories of Mental Test Scores. , 1971 .

[15]  Kristopher J Preacher,et al.  On the practice of dichotomization of quantitative variables. , 2002, Psychological methods.

[16]  Improvement,et al.  Adult literacy in America : a first look at the results of the National Adult Literacy Survey , 1993 .

[17]  Natalie A. Obrecht,et al.  An encounter frequency account of how experience affects likelihood estimation , 2009, Memory & cognition.

[18]  P. Ubel,et al.  Measuring Numeracy without a Math Test: Development of the Subjective Numeracy Scale , 2007, Medical decision making : an international journal of the Society for Medical Decision Making.

[19]  Jürgen Unützer,et al.  Long-term effectiveness and cost of a systematic care program for bipolar disorder. , 2006, Archives of general psychiatry.

[20]  S. Epstein,et al.  Conflict between intuitive and rational processing: when people behave against their better judgment. , 1994, Journal of personality and social psychology.

[21]  C. K. Mertz,et al.  PSYCHOLOGICAL SCIENCE Research Article Numeracy and Decision Making , 2022 .

[22]  Lisa M. Schwartz,et al.  The value of benefit data in direct-to-consumer drug ads. , 2004, Health affairs.

[23]  V. Reyna,et al.  Individual Differences in Numeracy and Cognitive Reflection, with Implications for Biases and Fallacies in Probability Judgment. , 2012, Journal of behavioral decision making.

[24]  S. Embretson The new rules of measurement. , 1996 .

[25]  Georg Rasch,et al.  Probabilistic Models for Some Intelligence and Attainment Tests , 1981, The SAGE Encyclopedia of Research Design.

[26]  J. Alonso,et al.  Classical test theory versus Rasch analysis for quality of life questionnaire reduction , 2003, Health and Quality of Life Outcomes.

[27]  A. Tversky,et al.  Prospect theory: an analysis of decision under risk — Source link , 2007 .

[28]  E. Peters Beyond Comprehension , 2012 .

[29]  V. Reyna,et al.  How numeracy influences risk comprehension and medical decision making. , 2009, Psychological bulletin.

[30]  K. Stanovich,et al.  The Cognitive Reflection Test as a predictor of performance on heuristics-and-biases tasks , 2011, Memory & cognition.

[31]  Melissa L. Finucane,et al.  Developing a tool for measuring the decision-making competence of older adults. , 2010, Psychology and aging.

[32]  A. Tversky,et al.  Prospect Theory : An Analysis of Decision under Risk Author ( s ) : , 2007 .

[33]  S. Briggs,et al.  The role of factor analysis in the development and evaluation of personality scales , 1986 .

[34]  C. K. Mertz,et al.  Less Is More in Presenting Quality Information to Consumers , 2007, Medical care research and review : MCRR.

[35]  John Sabatini,et al.  Basic Reading Skills and the Literacy of America's Least Literate Adults: Results from the 2003 National Assessment of Adult Literacy (NAAL) Supplemental Studies. NCES 2009-481. , 2009 .

[36]  J. Burkell,et al.  What are the chances? Evaluating risk and benefit information in consumer health materials. , 2004, Journal of the Medical Library Association : JMLA.

[37]  B. Rimer,et al.  General Performance on a Numeracy Scale among Highly Educated Samples , 2001, Medical decision making : an international journal of the Society for Medical Decision Making.

[38]  D. McCarthy,et al.  On the sins of short-form development. , 2000, Psychological assessment.

[39]  C. Estrada,et al.  Health literacy and numeracy. , 1999, JAMA.

[40]  A. Kaufman,et al.  Development and validation of a Rasch-derived CES-D short form. , 2004, Psychological assessment.

[41]  L. Cronbach,et al.  Construct validity in psychological tests. , 1955, Psychological bulletin.

[42]  K. Stanovich,et al.  On the relative independence of thinking biases and cognitive ability. , 2008, Journal of personality and social psychology.

[43]  Edward T. Cokely,et al.  Cognitive abilities and superior decision making under risk: A protocol analysis and process model evaluation , 2009, Judgment and Decision Making.

[44]  P. Slovic,et al.  The Use of Narrative Evidence and Explicit Likelihood by Decisionmakers Varying in Numeracy , 2009, Risk analysis : an official publication of the Society for Risk Analysis.

[45]  W. Velicer,et al.  Comparison of five rules for determining the number of components to retain. , 1986 .

[46]  Ian J. Bateman,et al.  The Affect Heuristic and the Attractiveness of Simple Gambles , 2007 .

[47]  Jacob Cohen,et al.  A power primer. , 1992, Psychological bulletin.

[48]  S. Frederick Journal of Economic Perspectives—Volume 19, Number 4—Fall 2005—Pages 25–42 Cognitive Reflection and Decision Making , 2022 .

[49]  Mark A. Kutner,et al.  The Health Literacy of America's Adults: Results from the 2003 National Assessment of Adult Literacy. NCES 2006-483. , 2006 .

[50]  Paul Slovic,et al.  The affect heuristic , 2007, Eur. J. Oper. Res..

[51]  Ellen Peters,et al.  Breast Cancer Patients’ Treatment Expectations after Exposure to the Decision Aid Program Adjuvant Online: The Influence of Numeracy , 2010, Medical decision making : an international journal of the Society for Medical Decision Making.

[52]  R. Devellis Classical Test Theory , 2006, Medical care.

[53]  C. K. Mertz,et al.  Bringing meaning to numbers: the impact of evaluative categories on decisions. , 2009, Journal of experimental psychology. Applied.

[54]  S. Dehaene,et al.  The Number Sense: How the Mind Creates Mathematics. , 1998 .

[55]  D. Watson,et al.  Constructing validity: Basic issues in objective scale development , 1995 .

[56]  A. Tversky,et al.  Prospect theory: analysis of decision under risk , 1979 .