ESTABLISHING THE VALIDITY OF THE LOCUS ASSESSMENTS THROUGH AN EVIDENCED-CENTERED DESIGN APPROACH

This paper presents the systematic process utilized by the Levels of Conceptual Understanding in Statistics (LOCUS) project to establish content validity for assessments measuring students’ statistical understanding in grades 6-12 (ages 11-18). Evidence Centered Design (ECD) was used to develop assessments aligned with the United States’ Common Core State Standards in Mathematics (CCSSM) as well as the Guidelines for Assessment and Instruction in Statistics Education (GAISE). The ECD process began with a domain analysis based on CCSSM, GAISE, and learning trajectories from statistics education research and subsequently added layers articulating claims about student proficiency and observable evidence to support those claims. The ECD approach formalized the evidentiary reasoning by which performance on LOCUS can be used to support valid inferences about the larger domain of statistical understanding. BACKGROUND Purpose of the LOCUS Assessments Over the past 25 years, inclusion of statistics at the school level and recognition of the importance of statistical literacy have been gaining momentum in the United States. These ongoing efforts to promote statistical literacy are exemplified by the American Statistical Association’s (ASA) Guidelines for Assessment and Instruction in Statistics Education (GAISE) (Franklin, et al., 2007) which identify three levels of statistical understanding (Levels A, B, and C) ideally achieved at the school level. Influenced by the GAISE document, the widely adopted Common Core State Standards in Mathematics (CCSSM) (NGACBP & CCSSO, 2010) have considerably raised the expectations for statistics learning in grades 6-12 (ages 11-18) across the country. However, inclusion in the standards does not guarantee that statistics will be taught at a level sufficient to produce statistically literate citizens. Gal & Garfield (1997) note that teaching of statistics often reflects the way it is assessed on large-scale assessments, with an emphasis on procedural knowledge rather than conceptual understanding. Further, increased emphasis on statistics in the precollege curriculum warrants research investigating students’ understanding of statistics and the effectiveness of instructional interventions, purposes for which existing large-scale assessments and instructor-designed exams are typically ill-suited (Gal & Garfield, 1997). Continued progress in the field of statistics education demands a valid and reliable assessment of statistical understanding The Levels of Conceptual Understanding in Statistics (LOCUS) project, funded by the National Science Foundation (DRL-1118168), is addressing these assessment issues by addressing three broad goals: • Develop instruments to assess levels of statistical understanding as initially defined in the GAISE framework and the CCSSM • Provide a characterization of grade 6-12 students’ current level of statistical understanding • Provide a tool for researchers and teachers to assess growth in statistical understanding LOCUS differs from traditional large-scale assessments in a few critical ways. First, LOCUS intends to measure conceptual understanding at all stages of the statistical problem-solving process (Franklin, et al., 2007), de-emphasizing rote calculations. Second, LOCUS takes into consideration the distinction between mathematical and statistical reasoning, noting that statistical reasoning is inextricably linked to context and must at every stage account for variability (Cobb & Moore, 1997; delMas, 2005).