Checking Dimensionality in Item Response Models With Principal Component Analysis on Standardized Residuals

Dimensionality is an important assumption in item response theory (IRT). Principal component analysis on standardized residuals has been used to check dimensionality, especially under the family of Rasch models. It has been suggested that an eigenvalue greater than 1.5 for the first eigenvalue signifies a violation of unidimensionality when there are 500 persons and 30 items. The cut-point of 1.5 is often used beyond this specific condition of sample size and test length. This study argues that a fixed cut-point is not applicable because the distribution of eigenvalues or their ratios depends on sample size and test length, just like other statistics. The authors conducted a series of simulations to verify this argument. They then proposed three chi-square statistics for multivariate independence to test the correlation matrix obtained from the standardized residuals. Through simulations, it was found that Steiger’s statistic behaved fairly like a chi-square distribution, when its degrees of freedom were adjusted.

[1]  P. Fayers.  Item Response Theory for Psychologists , 2004, Quality of Life Research.

[2]  John Hattie,et al.  Methodology Review: Assessing Unidimensionality of Tests and ltenls , 1985 .

[3]  Dirk L. Knol,et al.  On the Assessment of Dimensionality in Multidimensional Item Response Theory Models. Research Report 90-8. , 1990 .

[4]  E. Muraki.  A GENERALIZED PARTIAL CREDIT MODEL: APPLICATION OF AN EM ALGORITHM , 1992 .

[5]  Terry A. Ackerman.  Unidimensional IRT Calibration of Compensatory and Noncompensatory Multidimensional Items , 1989 .

[6]  D. Andrich.  A rating formulation for ordered response categories , 1978 .

[7]  J H Steiger,et al.  Testing Pattern Hypotheses On Correlation Matrices: Alternative Statistics And Some Empirical Results. , 1980, Multivariate behavioral research.

[8]  Roderick P. McDonald,et al.  The dimensionality of tests and items , 1981 .

[9]  J. H. Steiger.  Tests for comparing elements of a correlation matrix. , 1980 .

[10]  J. Linacre.  Detecting multidimensionality: which residual data-type works best? , 1998, Journal of outcome measurement.

[11]  Richard M. Smith.  A comparison of methods for determining dimensionality in Rasch measurement , 1996 .

[12]  F. Lord.  Applications of Item Response Theory To Practical Testing Problems , 1980 .

[13]  R. Hambleton,et al.  Item Response Theory , 1984, The History of Educational Measurement.

[14]  E. Muraki,et al.  Full-Information Item Factor Analysis , 1988 .

[15]  Georg Rasch,et al.  Probabilistic Models for Some Intelligence and Attainment Tests , 1981, The SAGE Encyclopedia of Research Design.

[16]  W. Venables,et al.  An analysis of correlation matrices: Equal correlations , 1984 .

[17]  Everett V. Smith.  Detecting and evaluating the impact of multidimensionality using item fit statistics and principal component analysis of residuals. , 2002, Journal of applied measurement.

[18]  G. Masters.  A rasch model for partial credit scoring , 1982 .

[19]  H. Kaiser.  The Application of Electronic Computers to Factor Analysis , 1960 .

[20]  G. Box,et al.  A general distribution theory for a class of likelihood criteria. , 1949, Biometrika.

[21]  Wendy M. Yen,et al.  Effects of Local Item Dependence on the Fit and Equating Performance of the Three-Parameter Logistic Model , 1984 .

[22]  R. Cattell.  The Scree Test For The Number Of Factors. , 1966, Multivariate behavioral research.

[23]  Ilker Yalcin,et al.  Nonlinear factor analysis , 1995 .

[24]  J. H. Steiger,et al.  Tests of Multivariate Independence: A Critical Analysis of "A Monte Carlo Study of Testing the Significance of Correlation Matrices" by Silver and Dunlap , 1993 .

[25]  M. Bartlett.  TESTS OF SIGNIFICANCE IN FACTOR ANALYSIS , 1950 .

[26]  G Karabatsos.  A critique of Rasch residual fit statistics. , 2000, Journal of applied measurement.

[27]  Identifiers California,et al.  Annual Meeting of the National Council on Measurement in Education , 1998 .

[28]  M. Reckase.  Unifactor Latent Trait Models Applied to Multifactor Tests: Results and Implications , 1979 .

[29]  M. Gessaroli,et al.  Using an Approximate Chi-Square Statistic to Test the Number of Dimensions Underlying the Responses to a Set of Items , 1996 .

[30]  M. Bartlett,et al.  A note on the multiplying factors for various chi square approximations , 1954 .

[31]  F. Samejima.  Estimation of latent ability using a response pattern of graded scores , 1968 .

[32]  Melvin R. Novick,et al.  Some latent train models and their use in inferring an examinee's ability , 1966 .