Psychometric properties of item response theory proficiency estimates are considered in this paper. Proficiency estimators based on summed scores and pattern scores include non-Bayes maximum likelihood and test characteristic curve estimators and Bayesian estimators. The psychometric properties investigated include reliability, conditional standard errors of measurement, and score distributions. Four real-data examples include (a) effects of choice of estimator on score distributions and percent proficient, (b) effects of the prior distribution on score distributions and percent proficient, (c) effects of test length on score distributions and percent proficient, and (d) effects of proficiency estimator on growth-related statistics for a vertical scale. The examples illustrate that the choice of estimator influences score distributions and the assignment of examinee to proficiency levels. In particular, for the examples studied, the choice of Bayes versus non-Bayes estimators had a more serious practical effect than the choice of summed versus pattern scoring.
[1]
F. Samejima.
Estimation of latent ability using a response pattern of graded scores
,
1968
.
[2]
Frederic M. Lord,et al.
Comparison of IRT True-Score and Equipercentile Observed-Score "Equatings"
,
1984
.
[3]
Frederic M. Lord.
MAXIMUM LIKELIHOOD AND BAYESIAN PARAMETER ESTIMATION IN ITEM RESPONSE THEORY
,
1986
.
[4]
E. Muraki.
A Generalized Partial Credit Model: Application of an EM Algorithm
,
1992
.
[5]
Mary Pommerich,et al.
Item Response Theory for Scores on Tests Including Polytomous Items with Ordered Responses
,
1995
.
[6]
R. Brennan,et al.
Test Equating, Scaling, and Linking
,
2004
.
[7]
Michael J. Kolen,et al.
Comparisons of Methodologies and Results in Vertical Scaling for Educational Achievement Tests
,
2007
.