Number Correct Scoring: Comparison between Classical True Score Theory and Multidimensional Item Response Theory.

This study investigated the number-correct scoring method based on different theories (classical true-score theory and multidimensional item response theory) when a standardized test requires more than one ability for an examinee to get a correct response. The number-correct scoring procedure that is widely used is the one that is defined in classical true-score theory (CTT). In CTT, a test score is equal to the number of items an examinee answered, so that all items are weighted "one." It is also possible to use a form of number-correct scoring in which the weights of items are different. In this study, the accuracy of estimated number-correct scores relative to true number-correct scores under CTT, multidimensional item response theory (MIRT) and both MIRT and CTT were studied using simulated data for a standardized test in which true scores and estimated scores were known. A method in which item weights were based on MIRT and test scores based on CTT (MIX method) was found to be the most accurate method used to estimate the true score on an examinee. This MIX method was also significantly different from the other three scoring methods using the bootstrap analysis. An appendix contains definitions of the notations representing the various parameters and approaches. (Contains 20 references.) (SLD) Reproductions supplied by EDRS are the best that can be made from the original document. CO 0) NC4 O Number Correct Scoring: Comparison between Classical True Score Theory and Multidimensional Item Response Theory Ourania Rotou, Patricia B. Elmore & Todd C. Headrick Southern Illinois University at Carbondale Department of Educational Psychology and Special Education Carbondale, IL 62901-4618 U.S. DEPARTMENT OF EDUCATION Of O .ce of Educational Research and Improvement E CATIONAL RESOURCES INFORMATION CENTER (ERIC) This document has been reproduced as received from the person or organizaiion originating it. Minor changes have been made to improve reproduction quality. Points of view or opinions stated in this document do not necessarily represent official OERI position or policy. 1 PERMISSION TO REPRODUCE AND DISSEMINATE THIS MATERIAL HAS BEEN GRANTED BY TO THE EDUCATIONAL RESOURCES INFORMATION CENTER (ERIC) Paper presented at the annual meeting of the American Educational Research Association, April, 2001, Seattle.