Assessing Item Fit for Unidimensional Item Response Theory Models Using Residuals from Estimated Item Response Functions

Residual analysis (e.g. Hambleton & Swaminathan, Item response theory: principles and applications, Kluwer Academic, Boston, 1985; Hambleton, Swaminathan, & Rogers, Fundamentals of item response theory, Sage, Newbury Park, 1991) is a popular method to assess fit of item response theory (IRT) models. We suggest a form of residual analysis that may be applied to assess item fit for unidimensional IRT models. The residual analysis consists of a comparison of the maximum-likelihood estimate of the item characteristic curve with an alternative ratio estimate of the item characteristic curve. The large sample distribution of the residual is proved to be standardized normal when the IRT model fits the data. We compare the performance of our suggested residual to the standardized residual of Hambleton et al. (Fundamentals of item response theory, Sage, Newbury Park, 1991) in a detailed simulation study. We then calculate our suggested residuals using data from an operational test. The residuals appear to be useful in assessing the item fit for unidimensional IRT models.

[1]  Clement A. Stone,et al.  Assessing Goodness of Fit of Item Response Theory Models: A Comparison of Traditional and Alternative Procedures , 2003 .

[2]  Shelby J. Haberman,et al.  A Stabilized Newton-Raphson Algorithm for Log-Linear Models for Frequency Tables Derived by Indirect Observation , 1988 .

[3]  G. Masters.  A rasch model for partial credit scoring , 1982 .

[4]  M. Reckase.  The Past and Future of Multidimensional Item Response Theory , 1997 .

[5]  T. Louis.  Finding the Observed Information Matrix When Using the EM Algorithm , 1982 .

[6]  W. M. Yen.  Using Simulation Results to Choose a Latent Trait Model , 1981 .

[7]  Hamzeh Dodeen.  The Relationship Between Item Parameters and Item Fit , 2004 .

[8]  Taehoon Kang,et al.  An Investigation of the Performance of the Generalized S-X2 Item-Fit Index for Polytomous IRT Models , 2007 .

[9]  Paul W. Holland.  The Dutch Identity: A new tool for the study of item response models , 1990 .

[10]  R. Hambleton,et al.  Handbook of Modern Item Response Theory , 1997 .

[11]  ResidPlots-2 , 2009 .

[12]  Alija Kulenović,et al.  Standards for Educational and Psychological Testing , 1999 .

[13]  Sandip Sinharay,et al.  How Often Do Subscores Have Added Value? Results from Operational and Simulated Data , 2010 .

[14]  R. Hambleton,et al.  Item Response Theory , 1984, The History of Educational Measurement.

[15]  J. Naylor,et al.  Applications of a Method for the Efficient Computation of Posterior Distributions , 1982 .

[16]  R. D. Bock,et al.  Marginal maximum likelihood estimation of item parameters , 1982 .

[17]  C. R. Rao,et al.  Linear Statistical Inference and its Applications , 1968 .

[18]  R. Hambleton,et al.  Item Response Theory: Principles and Applications , 1984 .

[19]  Howard Wainer,et al.  Estimating Ability With the Wrong Model , 1987 .

[20]  Shelby J. Haberman,et al.  USE OF GENERALIZED RESIDUALS TO EXAMINE GOODNESS OF FIT OF ITEM RESPONSE MODELS , 2009 .

[21]  George E. P. Box,et al.  Empirical Model‐Building and Response Surfaces , 1988 .

[22]  E. Muraki.  A Generalized Partial Credit Model , 1997 .

[23]  Cornelis A.W. Glas,et al.  A Comparison of Item-Fit Statistics for the Three-Parameter Logistic Model , 2003 .

[24]  R. D. Bock,et al.  Marginal maximum likelihood estimation of item parameters: Application of an EM algorithm , 1981 .

[25]  Thomas R. Boucher,et al.  Test Equating, Scaling, and Linking: Methods and Practices , 2007 .

[26]  R. Brennan,et al.  Test Equating, Scaling, and Linking , 2004 .

[27]  Calyampudi R. Rao,et al.  Linear Statistical Inference and Its Applications. , 1975 .

[28]  Shelby J. Haberman,et al.  Log-Linear Models and Frequency Tables with Small Expected Cell Counts , 1977 .

[29]  Shelby J. Haberman,et al.  Maximum Likelihood Estimates in Exponential Response Models , 1977 .

[30]  A. Rupp,et al.  Performance of the S − χ2 Statistic for Full-Information Bifactor Models , 2011 .

[31]  S. Haberman.  Analysis of qualitative data , 1978 .

[32]  Shelby J. Haberman,et al.  ADAPTIVE QUADRATURE FOR ITEM RESPONSE MODELS , 2006 .

[33]  Sandip Sinharay,et al.  Assessing Fit of Unidimensional Item Response Theory Models Using a Bayesian Approach. , 2005 .

[34]  R. Hambleton,et al.  Fundamentals of Item Response Theory , 1991 .

[35]  Sandip Sinharay.  Bayesian item fit analysis for unidimensional item response theory models. , 2006, The British journal of mathematical and statistical psychology.

[36]  S. Haberman.  Adaptive Quadrature for Item Response Models. Research Report. ETS RR-06-29. , 2006 .

[37]  D. Thissen,et al.  Likelihood-Based Item-Fit Indices for Dichotomous Item Response Theory Models , 2000 .

[38]  Stephen B. Dunbar,et al.  A Comparison of Item Fit Statistics for Mixed IRT Models , 2010 .