Development of the Concise Data Processing Assessment

The Concise Data Processing Assessment (CDPA) was developed to probe student abilities related to the nature of measurement and uncertainty and to handling data. The diagnostic is a ten question, multiple-choice test that can be used as both a pre-test and post-test. A key component of the development process was interviews with students, which were used to both uncover common modes of student thinking and validate item wording. To evaluate the reliability and discriminatory power of this diagnostic, we performed statistical tests focusing on both item analysis (item difficulty index, item discrimination index, and point-biserial coefficient) and on the entire test (test reliability and Ferguson’s delta). Scores on the CDPA range from chance (for novices) to about 80% (for experts), indicating that it possesses good dynamic range. Overall, the results indicate that the CDPA is a reliable assessment tool for measuring targeted abilities in undergraduate physics students.

[1]  P. Alberto,et al.  Literacy , 1983, Language in Society.

[2]  Saalih Allie,et al.  The development of first year physics students' ideas about measurement in terms of point and set paradigms , 2001 .

[3]  Michael Pignone,et al.  Numeracy and the medical student's ability to interpret data. , 2002, Effective clinical practice : ECP.

[4]  Hans Niedderer,et al.  SURVEY 2: STUDENTS’ IMAGES OF SCIENCE AS THEY RELATE TO LABWORK LEARNING , 1998 .

[5]  R. D'Agostino,et al.  The 27 Percent Rule Revisited , 1975 .

[6]  Robert J. Beichner,et al.  Evaluating an electricity and magnetism assessment tool: Brief electricity and magnetism assessment , 2006 .

[7]  David Rosengrant,et al.  Multiple-choice test of energy and momentum concepts , 2003, 1602.06497.

[8]  Paul Kline,et al.  A Handbook of Test Construction : Introduction to Psychometric Design , 1987 .

[9]  Arnold Zellner,et al.  Introduction to Measurement with Theory , 2009 .

[10]  Ronald K. Thornton,et al.  Assessing student learning of Newton’s laws: The Force and Motion Conceptual Evaluation and the Evaluation of Active Learning Laboratory and Lecture Curricula , 1998 .

[11]  R. Bell,et al.  Test Length and Validity , 1980 .

[12]  John P. Campbell,et al.  Measurement Theory for the Behavioral Sciences. , 1983 .

[13]  Lisa M. Schwartz,et al.  The Role of Numeracy in Understanding the Benefit of Screening Mammography , 1997, Annals of Internal Medicine.

[14]  L. McDermott,et al.  Resource Letter: PER-1: Physics Education Research , 1999 .

[15]  Literacy, numeracy and employability: evidence form the British birth cohort studies , 2004 .

[16]  David P Maloney,et al.  Surveying students’ conceptual knowledge of electricity and magnetism , 2001 .

[17]  Educational Evaluation.  Standards for Educational and Psychological Testing , 1999 .

[18]  C. E. Wieman,et al.  Design and Validation of the Quantum Mechanics Conceptual Survey. , 2010, 1007.2015.

[19]  Rosedith Sitgreaves Bowker.  Measurement Theory for the Behavioral Sciences , 1983 .

[20]  Wendy K. Adams,et al.  Development and Validation of Instruments to Measure Learning of Expert‐Like Thinking , 2011 .

[21]  Genaro Zavala,et al.  Understanding and interpreting calculus graphs: Refining an instrument , 2010 .

[22]  Robert J. Beichner,et al.  Students’ understanding of direct current resistive electrical circuits , 2004 .

[23]  G. A. Ferguson,et al.  On the theory of test discrimination , 1949, Psychometrika.

[24]  C. E. Wieman,et al.  AIP Conference Proceedings: Exploring Student Understanding of Energy through the Quantum Mechanics Conceptual Survey , 2006 .

[25]  Lillian C. McDermott,et al.  Student difficulties in connecting graphs and physics: Examples from kinematics , 1987 .

[26]  B. Simon,et al.  What is the Value of Course-Specific Learning Goals?. , 2009 .

[27]  Robert J. Beichner,et al.  Testing student interpretation of kinematics graphs , 1994 .

[28]  Marie-Geneviève Séré,et al.  Pupils' Reasoning and Practice during Hands-on Activities in the Measurement Phase. , 1998 .

[29]  Duane L. Deardorff,et al.  Introductory physics students' treatment of measurement uncertainty , 2001 .

[30]  Rebecca Lippmann Kung,et al.  University students’ ideas about data processing and data comparison in a physics laboratory course , 2012 .

[31]  Rodney L. Doran,et al.  Basic measurement and evaluation of science instruction , 1980 .

[32]  Matthias Burisch,et al.  Test length and validity revisited , 1997 .

[33]  Marie-Geneviève Séré,et al.  Learning the statistical analysis of measurement errors , 1993 .

[34]  L. Crocker,et al.  Introduction to Classical and Modern Test Theory , 1986 .

[35]  Rebecca Faith Lippmann.  Students' understanding of measurement and uncertainty in the physics laboratory: Social construction, underlying concepts, and quantitative analysis , 2003 .

[36]  D. Hestenes,et al.  Force concept inventory , 1992 .

[37]  Eric Schulman,et al.  Misconceptions about astronomical magnitudes , 1997 .