Three Approaches to Determining the Dimensionality of Binary Items

A monte carlo investigation of three approaches to assessing the dimensionality of binary items used a population model that allowed sampling of items and examinees and provided for variation and con trol of important parameters. The model was realistic of performance of binary items in current tests of cognitive abilities. Three indices were inves tigated : one based on the property of local indepen dence of unidimensional tests (the independence index), one based on patterns of second factor loadings derived from simplex theory (the pattern index), and one that reflects the shape of the curve of successive eigenvalues (the ratio of differences index). The last index was used for matrices of phi coefficients, tetrachoric correlations, and variances covariances. The local independence index reported here was the most accurate dimensionality index. The pattern index was accurate under many combi nations of parameters, but decreased substantially at the highest level of factor correlations and the widest dispersion of item difficulties. None of the eigenvalue indices produced satisfactory accuracy, except under the most favorable combinations of parameters. Nonetheless, the eigenvalues of variance-covariance matrices provided a more accurate basis for dimensionality decisions than tetrachoric correlations, which have been the statistic of choice of many investigators. Recommendations for use are also given.

[1]  Roderick P. McDonald,et al.  The dimensionality of tests and items , 1981 .

[2]  Arnold L. van den Wollenberg,et al.  Two new test statistics for the rasch model , 1982 .

[3]  F. Lord.  Applications of Item Response Theory To Practical Testing Problems , 1980 .

[4]  C. Parsons,et al.  Application of Unidimensional Item Response Theory Models to Multidimensional Data , 1983 .

[5]  L. Tucker,et al.  Evaluation of factor analytic research procedures by means of simulated correlation matrices , 1969 .

[6]  David A. Harrison,et al.  Robustness of Irt Parameter Estimation to Violations of The Unidimensionality Assumption , 1986 .

[7]  Lloyd G. Humphreys,et al.  Latent roots of random data correlation matrices with squared multiple correlations on the diagonal: A monte carlo study , 1976 .

[8]  Roderick P. McDonald,et al.  Factor Analysis and Related Methods , 1985 .

[9]  William Stout,et al.  A nonparametric approach for assessing latent trait unidimensionality , 1987 .

[10]  R. Wherry,et al.  Factor pattern of test items and tests as a function of the correlation coefficient: Content, difficulty, and constant error factors , 1944 .

[11]  Paul F. Lazarsfeld,et al.  Latent Structure Analysis. , 1969 .

[12]  Fritz Drasgow,et al.  Item response theory : application to psychological measurement , 1983 .

[13]  Richard G. Montanelli,et al.  An Investigation of the Parallel Analysis Criterion for Determining the Number of Common Factors , 1975 .

[14]  F. Krauss.  Latent Structure Analysis , 1980 .

[15]  William F. Strout.  A new item response theory modeling approach with applications to unidimensionality assessment and ability estimation , 1990 .

[16]  M. R. Novick,et al.  Statistical Theories of Mental Test Scores. , 1971 .

[17]  Rebecca Zwick,et al.  Assessing the Dimensionality of NAEP Reading Data , 1987 .

[18]  P. Rosenbaum.  Testing the conditional independence and monotonicity assumptions of item response theory , 1984 .

[19]  J. Tukey,et al.  Multiple-Factor Analysis , 1947 .

[20]  Mark D. Reckase,et al.  The Difficulty of Test Items That Measure More Than One Ability , 1985 .

[21]  John Hattie,et al.  Methodology Review: Assessing Unidimensionality of Tests and ltenls , 1985 .