Unidimensional Calibrations and Interpretations of Composite Traits for Multidimensional Tests

A two-stage process that considers the multi dimensionality of tests under the framework of unidimensional item response theory (IRT) is described and evaluated. In the first stage, items are clustered in a multidimensional latent space with respect to their direction of maximum dis crimination. The separate item clusters are subsequently calibrated using a unidimensional IRT model to provide item parameter and trait estimates for composite traits in the context of the multidimensional trait space. This application is proposed as a workable compromise to some of the estimation, indeterminacy, and interpretation problems that affect the direct use of multi dimensional IRT procedures for item calibration and trait estimation. The findings of a study based on simulated multidimensional data indicate that there are identifiable gains in estimation robustness and score interpretation with almost no sacrifice in goodness-of-fit using this two-stage approach to modeling composite latent traits.

[1]  Robert J. Mislevy,et al.  BILOG 3 : item analysis and test scoring with binary logistic models , 1990 .

[2]  Michael V. LeVine,et al.  Appropriateness measurement: Review, critique and validating studies , 1982 .

[3]  George Engelhard,et al.  Full-Information Item Factor Analysis: Applications of EAP Scores , 1985 .

[4]  Fritz Drasgow,et al.  Appropriateness measurement with polychotomous item response models and standardized indices , 1984 .

[5]  G. Freytag.  [CORRELATION AND CAUSALITY]. , 1964, Psychiatrie, Neurologie, und medizinische Psychologie.

[6]  E. B. Andersen,et al.  Modern factor analysis , 1961 .

[7]  William F. Strout.  A new item response theory modeling approach with applications to unidimensionality assessment and ability estimation , 1990 .

[8]  R. Luecht,et al.  Item Selection Using an Average Growth Approximation of Target Information Functions , 1992 .

[9]  F. Drasgow.  An Evaluation of Marginal Maximum Likelihood Estimation for the Two-Parameter Logistic Model , 1989 .

[10]  R. D. Bock,et al.  Marginal maximum likelihood estimation of item parameters: Application of an EM algorithm , 1981 .

[11]  Robert A. Forsyth,et al.  An Examination of the Characteristics of Unidimensional IRT Parameter Estimates Derived From Two-Dimensional Data , 1985 .

[12]  Roderick P. McDonald,et al.  Linear Versus Models in Item Response Theory , 1982 .

[13]  Terry A. Ackerman.  An Examination of the Effect of Multidimensionality on Parallel Forms Construction. , 1991 .

[14]  Timothy R. Miller,et al.  Cluster Analysis of Angular Data in Applications of Multidimensional Item-Response Theory , 1992 .

[15]  Mark D. Reckase,et al.  The Difficulty of Test Items That Measure More Than One Ability , 1985 .

[16]  Walter D. Way,et al.  THE FEASIBILITY OF MODELING SECONDARY TOEFL ABILITY DIMENSIONS USING MULTIDIMENSIONAL IRT MODELS , 1992 .

[17]  B. Muthén.  Latent variable structural equation modeling with categorical data , 1983 .

[18]  B. Wright,et al.  Best test design , 1979 .

[19]  Marek Libura,et al.  Multidimensional Item Response Theory Estimation: A computer program , 1987 .

[20]  Robert J. Mislevy,et al.  Bayes modal estimation in item response models , 1986 .

[21]  Mark D. Reckase,et al.  The Discriminating Power of Items That Measure More Than One Dimension , 1991 .

[22]  Richard M. Luecht,et al.  Computerized Test Construction Using an Average Growth Approximation of Target Information Functions. , 1990 .

[23]  R. Cranley,et al.  Multivariate Analysis—Methods and Applications , 1985 .

[24]  William Stout,et al.  A nonparametric approach for assessing latent trait unidimensionality , 1987 .

[25]  Fumiko Samejima,et al.  Normal ogive model on the continuous response level in the multidimensional latent space , 1974 .