The Statistics Concepts Inventory: Developing A Valid And Reliable Instrument

The Statistics Concepts Inventory (SCI) is currently under development at the University of Oklahoma. This paper documents the early stages of assessing the validity, reliability, and discriminatory power of a cognitive assessment instrument for statistics. The evolution of test items on the basis of validity, reliability, and discrimination is included. The instrument has been validated on the basis of content validity through the use of focus groups and faculty surveys. Concurrent validity is measured by correlating SCI scores with course grades. The SCI currently attains concurrent validity for Engineering Statistics courses, but fails to do so for Mathematics Statistics courses. Because the SCI is targeted at Engineering departments, this is a good starting point, but the researchers hope to improve the instrument so that it has applicability across disciplines. The test is shown to be reliable in terms of coefficient alpha for most populations. This paper also describes how specific questions have changed as a result of answer distribution analysis, reliability, discrimination, and focus group comments. Four questions are analyzed in detail: 1) one that was thrown out, 2) one that underwent major revisions, 3) one that required only minor changes, and 4) one that required no changes.

[1]  A. Tversky,et al.  Subjective Probability: A Judgment of Representativeness , 1972 .

[2]  Robert V. Brill,et al.  Applied Statistics and Probability for Engineers , 2004, Technometrics.

[3]  M. W. Richardson,et al.  The theory of the estimation of test reliability , 1937 .

[4]  J. Novak,et al.  Educational Psychology: A Cognitive View , 1969 .

[5]  T. Reed-Rhoads,et al.  Progress on concept inventory assessment tools , 2003, 33rd Annual Frontiers in Education, 2003. FIE 2003..

[6]  Albert C. Oosterhof,et al.  Developing and using classroom assessments , 1996 .

[7]  S. D. Lima,et al.  Concept or computation: Students' understanding of the mean , 1981 .

[8]  R. Hake.  Interactive-engagement vs Traditional Methods in Mechanics Instruction* , 1998 .

[9]  David S. Moore,et al.  The active practice of statistics : a textbook for multimedia learning , 1997 .

[10]  R. L. Ebel,et al.  Procedures for the Analysis of Classroom Tests , 1954 .

[11]  T. L. Kelley.  The selection of upper and lower groups for the validation of test items. , 1939 .

[12]  Kirk Allen,et al.  The statistics concept inventory: a pilot study , 2003, 33rd Annual Frontiers in Education, 2003. FIE 2003..

[13]  L. Cronbach.  Coefficient alpha and the internal structure of tests , 1951 .

[14]  Alexander Pollatsek,et al.  Inconsistencies in Students' Reasoning about Probability. , 1993 .

[15]  Joan Garfield,et al.  Difficulties in Learning Basic Concepts in Probability and Statistics: Implications for Research. , 1988 .

[16]  Ibrahim A. Halloun,et al.  The initial knowledge state of college physics students , 1985 .

[17]  Clifford Konold,et al.  Issues in Assessing Conceptual Understanding in Probability and Statistics , 1995 .

[18]  D. Hestenes,et al.  Force concept inventory , 1992 .