Conditional standard errors of measurement for scale scores using binomial and compund binomial assu

Standard errors of measurement of scale scores by score level (conditional standard errors of measurement) can be valuable to users of test results. In addition, the Standards for Educational and Psychological Testing (AERA, APA, & NCME, 1985) recommends that conditional standard errors be reported by test developers. Although a variety of procedures are available for estimating conditional standard errors of measurement for raw scores, few procedures exist for estimating conditional standard errors of measurement for scale scores from a single test administration. In this article, a procedure is described for estimating the reliability and conditional standard errors of measurement of scale scores. This method is illustrated using a strong true score model. Practical applications of this methodology are given. These applications include a procedure for constructing score scales that equalize standard errors of measurement along the score scale. Also included are examples of the effects of various nonlinear raw-to-scale score transformations on scale score reliability and conditional standard errors of measurement. These illustrations examine the effects on scale score reliability and conditional standard errors of measurement of (a) the different types of raw-to-scale score transformations (e.g., normalizing scores), (b) the number of scale score points used, and (c) the transformation used to equate alternate forms of a test. All the illustrations use data from the ACT Assessment testing program.

[1]  Frederic M. Lord,et al.  A theoretical distribution for mental test scores , 1962 .

[2]  F. Lord Applications of Item Response Theory To Practical Testing Problems , 1980 .

[3]  Robert L. Brennan,et al.  A Variance Components Model for Measurement Procedures Associated with a Table of Specifications , 1982 .

[4]  Audrey L. Quails-Payne A Comparison of Score Level Estimates of the Standard Error of Measurement , 1992 .

[5]  Ronald A. Thisted,et al.  Elements of Statistical Computing: Numerical Computation. , 1991 .

[6]  L. S. Feldt Some Relationships between the Binomial Error Model and Classical Test Theory , 1984 .

[7]  B. Hanson Method of Moments Estimates for the Four-Parameter Beta Compound Binomial Model and the Calculation of Classification Consistency Indexes , 1991 .

[8]  J. Tukey,et al.  Transformations Related to the Angular and the Square Root , 1950 .

[9]  Frederic M. Lord,et al.  A strong true-score theory, with applications , 1965 .

[10]  F. Lord DO TESTS OF THE SAME LENGTH HAVE THE SAME STANDARD ERROR OF MEASUREMENT , 1956 .

[11]  F. Lord Estimating true-score distributions in psychological testing (an empirical bayes estimation problem) , 1969 .

[12]  Tony Scallan,et al.  Elements of Statistical Computing: Numerical Computation , 1988 .

[13]  M. J. Kolen Smoothing Methods for Estimating Test Score Distributions , 1991 .

[14]  F. F. Stephan,et al.  Statistical Procedures and their Mathematical Bases. , 1936 .

[15]  M. J. Kolen Defining Score Scales in Relation to Measurement Error , 1988 .