Multidimensional Scoring of Abilities: The Ordered Polytomous Response Case:

Recent work has shown that multidimensionally scoring responses from different tests can provide better ability estimates. For educational assessment data, applications of this approach have been limited to binary scores. Of the different variants, the de la Torre and Patz model is considered more general because implementing the scoring procedure does not require prior knowledge of the correlational structure of the abilities, and existing item parameter estimates from traditional item response theory models can be utilized. This article extends the application of this method to data scored on ordered polytomous scales. A simulation study systematically examines how improvement in ability estimates is affected by factors such as the number of score categories, number of tests, test length, and correlation between abilities. Application of the method is illustrated using real data. Estimates of the abilities and the correlational structure are obtained using the Markov chain Monte Carlo method.

[1]  F. Samejima.  Estimation of latent ability using a response pattern of graded scores , 1968 .

[2]  Noel J. Purcell,et al.  Estimation for Small Domains , 1979 .

[3]  Noel J. Purcell,et al.  A Biometrics Invited Paper. Estimation for Small Domains , 1979 .

[4]  R. D. Bock,et al.  Marginal maximum likelihood estimation of item parameters , 1982 .

[5]  D. Rubin,et al.  On Jointly Estimating Parameters and Missing Data by Maximizing the Complete-Data Likelihood , 1983 .

[6]  R. Mislevy.  Estimating latent distributions , 1984 .

[7]  Robert J. Mislevy.  Exploiting auxiliary information about examinees in the estimation of item parameters , 1986 .

[8]  E. Muraki.  A Generalized Partial Credit Model: Application of an EM Algorithm , 1992 .

[9]  G. Casella,et al.  Explaining the Gibbs Sampler , 1992 .

[10]  Fritz Drasgow,et al.  Distinguishing Among Paranletric item Response Models for Polychotomous Ordered Data , 1994 .

[11]  L. Tierney.  Markov Chains for Exploring Posterior Distributions , 1994 .

[12]  David B. Dunson,et al.  Bayesian Data Analysis , 2010 .

[13]  David J. Spiegelhalter,et al.  Introducing Markov chain Monte Carlo , 1995 .

[14]  S. Chib,et al.  Understanding the Metropolis-Hastings Algorithm , 1995 .

[15]  Daniel O. Segall,et al.  Multidimensional adaptive testing , 1996 .

[16]  Raymond J. Adams,et al.  The Multidimensional Random Coefficients Multinomial Logit Model , 1997 .

[17]  Dani Gamerman,et al.  Markov Chain Monte Carlo: Stochastic Simulation for Bayesian Inference , 1997 .

[18]  Mark D. Reckase,et al.  A Linear Logistic Multidimensional Model for Dichotomous Item Response Data , 1997 .

[19]  Brian W. Junker,et al.  Applications and Extensions of MCMC in IRT: Multiple Item Types, Missing Data, and Rated Responses , 1999 .

[20]  Wen-Chung Wang,et al.  Implementation and Measurement Efficiency of Multidimensional Computerized Adaptive Testing , 2004 .

[21]  Wen-Chung Wang,et al.  Improving measurement precision of test batteries using multidimensional item response models. , 2004, Psychological methods.

[22]  Richard J. Patz,et al.  Making the Most of What We Have: A Practical Application of Multidimensional Item Response Theory in Test Scoring , 2005 .

[23]  William D. Schafer,et al.  Trait Parameter Recovery Using Multidimensional Computerized Adaptive Testing in Reading and Mathematics , 2005 .

[24]  Cees A. W. Glas,et al.  Application of Multidimensional Item Response Theory Models to Longitudinal Data , 2006 .