Making the Most of What We Have: A Practical Application of Multidimensional Item Response Theory in Test Scoring

This article proposes a practical method that capitalizes on the availability of information from multiple tests measuring correlated abilities given in a single test administration. By simultaneously estimating different abilities with the use of a hierarchical Bayesian framework, more precise estimates for each ability dimension are obtained. The efficiency of the proposed method is most pronounced when highly correlated abilities are estimated from multiple short tests. Employing Markov chain Monte Carlo techniques allows for straightforward estimation of model parameters.

[1]  David Thissen,et al.  Item Response Theory for Items Scored in Two Categories , 2001 .

[2]  Richard J. Patz,et al.  A Straightforward Approach to Markov Chain Monte Carlo Methods for Item Response Models , 1999 .

[3]  Noel J. Purcell,et al.  A Biometrics Invited Paper. Estimation for Small Domains , 1979 .

[4]  Fumiko Samejima,et al.  A Use of the Information Function in Tailored Testing , 1977 .

[5]  David J. Spiegelhalter,et al.  Introducing Markov chain Monte Carlo , 1995 .

[6]  G. Casella,et al.  Explaining the Gibbs Sampler , 1992 .

[7]  Daniel O. Segall,et al.  Multidimensional adaptive testing , 1996 .

[8]  L. Tierney.  Markov Chains for Exploring Posterior Distributions , 1994 .

[9]  S. Chib,et al.  Understanding the Metropolis-Hastings Algorithm , 1995 .

[10]  D. Rubin,et al.  On Jointly Estimating Parameters and Missing Data by Maximizing the Complete-Data Likelihood , 1983 .

[11]  Brian W. Junker,et al.  Applications and Extensions of MCMC in IRT: Multiple Item Types, Missing Data, and Rated Responses , 1999 .

[12]  F. Lord.  Applications of Item Response Theory To Practical Testing Problems , 1980 .

[13]  R. Hambleton,et al.  Item Response Theory , 1984, The History of Educational Measurement.

[14]  Noel J. Purcell,et al.  Estimation for Small Domains , 1979 .

[15]  R. D. Bock,et al.  Marginal maximum likelihood estimation of item parameters: Application of an EM algorithm , 1981 .

[16]  Howard Wainer,et al.  Augmented Scores-"Borrowing Strength" to Compute Scores Based on Small Numbers ofltems , 2001 .

[17]  Eugene G. Johnson.  The NAEP 1992 technical report , 1994 .

[18]  Sylvia Richardson,et al.  Markov Chain Monte Carlo in Practice , 1997 .

[19]  J. S. Roberts,et al.  A General Item Response Theory Model for Unfolding Unidimensional Polytomous Responses , 2000 .

[20]  R. Hambleton,et al.  Handbook of Modern Item Response Theory , 1997 .

[21]  R. Mislevy.  Estimating latent distributions , 1984 .

[22]  W. Alan Nicewander,et al.  Ability estimation for conventional tests , 1993 .

[23]  R. D. Bock,et al.  Marginal maximum likelihood estimation of item parameters , 1982 .