Investigating the Impact of Uncertainty About Item Parameters on Ability Estimation

Asymptotic expansions of the maximum likelihood estimator (MLE) and weighted likelihood estimator (WLE) of an examinee’s ability are derived while item parameter estimators are treated as covariates measured with error. The asymptotic formulae present the amount of bias of the ability estimators due to the uncertainty of item parameter estimators. A numerical example is presented to illustrate how to apply the formulae to evaluate the impact of uncertainty about item parameters on ability estimation and the appropriateness of estimating ability using the regular MLE or WLE method.

[1]  Raymond J. Carroll,et al.  Covariate Measurement Error in Logistic Regression , 1985 .

[2]  Ing Rj Ser.  Approximation Theorems of Mathematical Statistics , 1980 .

[3]  Robert J. Mislevy,et al.  BILOG 3 : item analysis and test scoring with binary logistic models , 1990 .

[4]  Alexander Kukush,et al.  Measurement Error Models , 2011, International Encyclopedia of Statistical Science.

[5]  Hua-Hua Chang,et al.  The asymptotic posterior normality of the latent trait in an IRT model , 1993 .

[6]  Frederic M. Lord,et al.  Unbiased estimators of ability parameters, of their variance, and of their parallel-forms reliability , 1983 .

[7]  Raymond J. Carroll,et al.  Measurement error in nonlinear models: a modern perspective , 2006 .

[8]  Charles Lewis.  Expected Response Functions , 2001 .

[9]  B. Efron.  The jackknife, the bootstrap, and other resampling plans , 1987 .

[10]  Robert J. Mislevy,et al.  DEALING WITH UNCERTAINTY ABOUT ITEM PARAMETERS: EXPECTED RESPONSE FUNCTIONS , 1994 .

[11]  F. Lord.  Applications of Item Response Theory To Practical Testing Problems , 1980 .

[12]  R. Tsutakawa,et al.  The effect of uncertainty of item parameter estimation on ability estimates , 1990 .

[13]  Anne Boomsma,et al.  Essays on Item Response Theory , 2000 .

[14]  J. Norris.  Appendix: probability and measure , 1997 .

[15]  B. Efron,et al.  The Jackknife: The Bootstrap and Other Resampling Plans. , 1983 .

[16]  D. Hall.  Measurement Error in Nonlinear Models: A Modern Perspective , 2008 .

[17]  T. A. Warm.  Weighted likelihood estimation of ability in item response theory , 1989 .