Investigation and Treatment of Missing Item Scores in Test and Questionnaire Data

This article first discusses a statistical test for investigating whether or not the pattern of missing scores in a respondent-by-item data matrix is random. Since this is an asymptotic test, we investigate whether it is useful in small but realistic sample sizes. Then, we discuss two known simple imputation methods, person mean (PM) and two-way (TW) imputation, and we propose two new imputation methods, response-function (RF) and mean response-function (MRF) imputation. These methods are based on few assumptions about the data structure. An empirical data example with simulated missing item scores shows that the new method RF was superior to the methods PM, TW, and MRF in recovering from incomplete data several statistical properties of the original complete data. Methods TW and RF are useful both when item score missingness is ignorable and nonignorable.

[1]  Mark D. Reckase,et al.  Item Response Theory: Parameter Estimation Techniques , 1998 .

[2]  D. Altman,et al.  Missing data , 2007, BMJ : British Medical Journal.

[3]  L. A. van der Ark,et al.  The effect of missing data imputation on Mokken scale analysis , 2005 .

[4]  K. Koehler,et al.  An Empirical Investigation of Goodness-of-Fit Statistics for Sparse Multinomials , 1980 .

[5]  M. Petersen,et al.  Introduction to Nonparametric Item Response Theory , 2005, Quality of Life Research.

[6]  David E. Booth,et al.  Analysis of Incomplete Multivariate Data , 2000, Technometrics.

[7]  I. W. Molenaar,et al.  Rasch models: foundations, recent developments and applications , 1995 .

[8]  Timothy R. C. Read,et al.  Multinomial goodness-of-fit tests , 1984 .

[9]  Arnold L. van den Wollenberg,et al.  Two new test statistics for the rasch model , 1982 .

[10]  Jae-On Kim,et al.  The Treatment of Missing Data in Multivariate Analysis , 1977 .

[11]  Donald B. Rubin,et al.  EM and beyond , 1991 .

[12]  Harrie C. M. Vorst,et al.  Alternative Missing Data Techniques to Grade Point Average: Imputing Unavailable Grades , 2002 .

[13]  Nicole A. Lazar,et al.  Statistical Analysis With Missing Data , 2003, Technometrics.

[14]  Christine E. DeMars,et al.  Item Response Theory , 2010, Assessing Measurement Invariance for Applied Research.

[15]  Mark Huisman,et al.  Item Nonresponse: Occurence, Causes, and Imputation of Missing Answers to Test Items , 1999 .

[16]  Cees A. W. Glas,et al.  Testing the Rasch Model , 1995 .

[17]  Georg Rasch,et al.  Probabilistic Models for Some Intelligence and Attainment Tests , 1981, The SAGE Encyclopedia of Research Design.

[18]  Brian W. Junker,et al.  Latent and Manifest Monotonicity in Item Response Models , 2000 .

[19]  Robert J. Mokken,et al.  A Theory and Procedure of Scale Analysis. , 1973 .

[20]  L. Cronbach.  Coefficient alpha and the internal structure of tests , 1951 .

[21]  Klaas Sijtsma,et al.  Factor analysis of multidimensional polytomous item response data suffering from ignorable item nonresponse. , 1999 .

[22]  Clifford C. Clogg,et al.  Handbook of Statistical Modeling for the Social and Behavioral Sciences. , 1995 .

[23]  M. R. Novick,et al.  Statistical Theories of Mental Test Scores. , 1971 .

[24]  Klaas Sijtsma,et al.  MSP5 for Windows User's Manual , 2000 .

[25]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[26]  W. Wong,et al.  The calculation of posterior distributions by data augmentation , 1987 .

[27]  Charles Lewis,et al.  A Nonparametric Approach to the Analysis of Dichotomous Item Responses , 1982 .

[28]  K Sijtsma,et al.  Influence of Imputation and EM Methods on Factor Analysis when Item Nonresponse in Questionnaire Data is Nonignorable , 2000, Multivariate behavioral research.

[29]  B. Junker.  Conditional association, essential independence and monotone unidimensional Item response models , 1993 .

[30]  Mark Huisman,et al.  Imputation of Missing Scale Data with Item Response Models , 2001 .

[31]  Brian W. Junker,et al.  Stochastic ordering using the latent trait and the sum score in polytomous IRT models , 1997 .

[32]  Peter J. Cameron,et al.  Rank three permutation groups with rank three subconstituents , 1985, J. Comb. Theory, Ser. B.