A Mixture Group Bifactor Model for Binary Responses

The presence of nuisance dimensionality is a potential threat to the accuracy of results for tests calibrated using a measurement model such as a factor analytic model or an item response theory model. This article describes a mixture group bifactor model to account for the nuisance dimensionality due to a testlet structure as well as the dimensionality due to differences in patterns of responses. The model can be used for testing whether or not an item functions differently across latent groups in addition to investigating the differential effect of local dependency among items within a testlet. An example is presented comparing test speededness results from a conventional factor mixture model, which ignores the testlet structure, with results from the mixture group bifactor model. Results suggested the 2 models treated the data somewhat differently. Analysis of the item response patterns indicated that the 2-class mixture bifactor model tended to categorize omissions as indicating speededness. With the mixture group bifactor model, more local dependency was present in the speeded than in the nonspeeded class. Evidence from a simulation study indicated the Bayesian estimation method used in this study for the mixture group bifactor model can successfully recover generated model parameters for 1- to 3-group models for tests containing testlets.

[1]  Allan S. Cohen,et al.  A Mixture Model Analysis of Differential Item Functioning , 2005 .

[2]  Richard R. Reilly,et al.  A STUDY OF SPEEDEDNESS AS A SOURCE OF TEST BIAS1 , 1972 .

[3]  Philip Heidelberger,et al.  Simulation Run Length Control in the Presence of an Initial Transient , 1983, Oper. Res..

[4]  The Precision of Simulation Study Results , 2001 .

[5]  Allan S. Cohen,et al.  A Method for Maintaining Scale Stability in the Presence of Test Speededness , 2003 .

[6]  Identifiers California,et al.  Annual Meeting of the National Council on Measurement in Education , 1998 .

[7]  Howard Wainer,et al.  Testlet Response Theory: An Analog for the 3PL Model Useful in Testlet-Based Adaptive Testing , 2000 .

[8]  T. C. Oshima.  The Effect of Speededness on Parameter Estimation in Item Response Theory , 1994 .

[9]  Andrew Gelman,et al.  General methods for monitoring convergence of iterative simulations , 1998 .

[10]  Allan S. Cohen,et al.  A Multilevel Mixture IRT Model With an Application to DIF , 2010 .

[11]  Sylvia Frühwirth-Schnatter,et al.  Finite Mixture and Markov Switching Models , 2006 .

[12]  D. Rubin,et al.  Inference from Iterative Simulation Using Multiple Sequences , 1992 .

[13]  C. Robert.  Mixtures of Distributions: Inference and Estimation , 1996 .

[14]  Robert J. Mislevy,et al.  Modeling item responses when different subjects employ different solution strategies , 1990 .

[15]  H. Akaike.  A new look at the statistical model identification , 1974 .

[16]  Allan S. Cohen,et al.  A Mixture Item Response Model for Multiple-Choice Data , 2001 .

[17]  George B. Macready,et al.  The Use of Loglinear Models for Assessing Differential Item Functioning Across Manifest and Latent Examinee Groups , 1990 .

[18]  Bradley P. Carlin,et al.  Bayesian measures of model complexity and fit , 2002 .

[19]  C. Robert,et al.  Estimation of Finite Mixture Distributions Through Bayesian Sampling , 1994 .

[20]  David Thissen,et al.  Trace Lines for Testlets: A Use of Multiple-Categorical-Response Models. , 1989 .

[21]  Richard J. Patz,et al.  A Straightforward Approach to Markov Chain Monte Carlo Methods for Item Response Models , 1999 .

[22]  S. Richardson,et al.  Mixtures of distributions: inference and estimation , 1995 .

[23]  Matthias von Davier,et al.  PARTIALLY OBSERVED MIXTURES OF IRT MODELS: AN EXTENSION OF THE GENERALIZED PARTIAL CREDIT MODEL , 2003 .

[24]  N. E. Day.  Estimating the components of a mixture of normal distributions , 1969 .

[25]  Donald Hedeker,et al.  Full-information item bi-factor analysis , 1992 .

[26]  David G. Stork,et al.  Pattern Classification (2nd ed.) , 1999 .

[27]  Martha L. Stocking,et al.  Developing a Common Metric in Item Response Theory , 1982 .

[28]  A. Cohen,et al.  The Role of Extended Time and Item Content on a High–Stakes Mathematics Test , 2005 .

[29]  Frank Rijmen,et al.  Formal Relations and an Empirical Comparison among the Bi‐Factor, the Testlet, and a Second‐Order Multidimensional IRT Model , 2010 .

[30]  Allan S. Cohen,et al.  Model Selection Methods for Mixture Dichotomous IRT Models , 2009 .

[31]  J. Albert.  Bayesian Estimation of Normal Ogive Item Response Curves Using Gibbs Sampling , 1992 .

[32]  Jürgen Rost,et al.  Rasch Models in Latent Classes: An Integration of Two Approaches to Item Analysis , 1990 .

[33]  Wendy M. Yen,et al.  Effects of Local Item Dependence on the Fit and Equating Performance of the Three-Parameter Logistic Model , 1984 .

[34]  J. Rost,et al.  Applications of Latent Trait and Latent Class Models in the Social Sciences , 1998 .

[35]  Jun Yu,et al.  Deviance Information Criterion for Comparing Stochastic Volatility Models , 2002 .

[36]  Kentaro Yamamoto,et al.  ESTIMATING THE EFFECTS OF TEST LENGTH AND TEST TIME ON PARAMETER ESTIMATION USING THE HYBRID MODEL , 1995 .

[37]  Daniel M. Bolt,et al.  A Comparison of Alternative Models for Testlets , 2006 .

[38]  Kentaro Yamamoto,et al.  A Model That Combines IRT and Latent Class Models , 1987 .

[39]  M. Stephens.  Dealing with label switching in mixture models , 2000 .

[40]  Isaac I. Bejar.  Test Speededness under Number-Right Scoring: An Analysis of the Test of English as a Foreign Language. , 1985 .

[41]  Richard R. Reilly,et al.  A STUDY OF SPEEDEDNESS AS A SOURCE OF TEST BIAS , 1971 .

[42]  Eric T. Bradlow,et al.  A Bayesian random effects model for testlets , 1999 .

[43]  Robert D. Ankenmann,et al.  Comparison of Dichotomous and Polytomous Item Response Models in Equating Scores from Tests Composed of Testlets , 2001 .

[44]  Howard Wainer,et al.  How Is Reliability Related to the Quality of Test Scores? What Is the Effect of Local Dependence on Reliability? , 1998 .

[45]  M. R. Novick,et al.  Statistical Theories of Mental Test Scores. , 1971 .

[46]  K. Samuelsen.  Examining Differential Item Functioning From A Latent Class Perspective , 2005 .

[47]  G. Schwarz.  Estimating the Dimension of a Model , 1978 .

[48]  Allan S. Cohen,et al.  Item Parameter Estimation Under Conditions of Test Speededness: Application of a Mixture Rasch Model With Ordinal Constraints , 2002 .

[49]  A. Raftery,et al.  How Many Iterations in the Gibbs Sampler , 1991 .