Detecting Intervention Effects Using a Multilevel Latent Transition Analysis with a Mixture IRT Model

A multilevel latent transition analysis (LTA) with a mixture IRT measurement model (MixIRTM) is described for investigating the effectiveness of an intervention. The addition of a MixIRTM to the multilevel LTA permits consideration of both potential heterogeneity in students’ response to instructional intervention as well as a methodology for assessing stage sequential change over time at both student and teacher levels. Results from an LTA–MixIRTM and multilevel LTA–MixIRTM were compared in the context of an educational intervention study. Both models were able to describe homogeneities in problem solving and transition patterns. However, ignoring a multilevel structure in LTA–MixIRTM led to different results in group membership assignment in empirical results. Results for the multilevel LTA–MixIRTM indicated that there were distinct individual differences in the different transition patterns. The students receiving the intervention treatment outscored their business as usual (i.e., control group) counterparts on the curriculum-based Fractions Computation test. In addition, 27.4 % of the students in the sample moved from the low ability student-level latent class to the high ability student-level latent class. Students were characterized differently depending on the teacher-level latent class.

[1]  L. Collins,et al.  Latent Class Models for Stage-Sequential Dynamic Latent Variables , 1992 .

[2]  Chih-Chien Yang,et al.  Evaluating latent class analysis models in qualitative phenotype identification , 2006, Comput. Stat. Data Anal..

[3]  Sophia Rabe-Hesketh,et al.  Generalized latent variable models: multilevel, longitudinal, and structural equation models , 2004 .

[4]  Kentaro Yamamoto,et al.  Partially Observed Mixtures of IRT Models: An Extension of the Generalized Partial-Credit Model , 2003 .

[5]  Ronald C. Serlin,et al.  Shrinking Achievement Differences With Anchored Math Problems , 2007 .

[6]  J. Vermunt,et al.  Latent class and finite mixture models for multilevel data sets , 2008, Statistical methods in medical research.

[7]  Sophia Rabe-Hesketh,et al.  Alternating imputation posterior estimation of models with crossed random effects , 2011, Comput. Stat. Data Anal..

[8]  D. Ball.  Prospective Elementary and Secondary Teachers' Understanding of Division. , 1990 .

[9]  John D. Bransford,et al.  The Jasper Project: Lessons in Curriculum, Instruction, Assessment, and Professional Development , 1997 .

[10]  Michael W. Arthur,et al.  Using Multilevel Mixtures to Evaluate Intervention Effects in Group Randomized Trials , 2008, Multivariate behavioral research.

[11]  Matthias von Davier,et al.  Measuring Growth in a Longitudinal Large-Scale Assessment with a General Latent Variable Model , 2011 .

[12]  Erling B. Andersen,et al.  Estimating latent correlations between repeated testings , 1985 .

[13]  Daniel T. Hickey,et al.  The Motivational and Academic Consequences of Elementary Mathematics Environments: Do Constructivist Innovations and Reforms Make a Difference? , 2001 .

[14]  H.L.J. van der Maas,et al.  An IRT Model with a Parameter-Driven Process for Change , 2005 .

[15]  Allan S. Cohen,et al.  Markov chain Monte Carlo estimation of a mixture item response theory model , 2013 .

[16]  Kristopher J Preacher,et al.  Measuring change for a multidimensional test using a generalized explanatory longitudinal item response model. , 2013, The British journal of mathematical and statistical psychology.

[17]  Paul De Boeck,et al.  A relation between a between-item multidimensional IRT model and the mixture rasch model , 2005 .

[18]  H. Bozdogan.  Model selection and Akaike's Information Criterion (AIC): The general theory and its analytical extensions , 1987 .

[19]  J. Graham,et al.  Modeling transitions in latent stage-sequential processes: a substance use prevention example. , 1991, Journal of consulting and clinical psychology.

[20]  Anthony S. Bryk,et al.  Hierarchical Linear Models: Applications and Data Analysis Methods , 1992 .

[21]  Tihomir Asparouhov,et al.  Item response mixture modeling: application to tobacco dependence criteria. , 2006, Addictive behaviors.

[22]  N. Cole.  Conceptions of Educational Achievement , 1990 .

[23]  Katherine E. Masyn,et al.  General growth mixture modeling for randomized preventive interventions. , 2001, Biostatistics.

[24]  S. Embretson,et al.  Item response theory for psychologists , 2000 .

[25]  Brian A. Bottge,et al.  Effects of Contextualized Math Instruction on Problem Solving of Average and Below-Average Achieving Students , 1999 .

[26]  Geoffrey J. McLachlan,et al.  Finite Mixture Models , 2019, Annual Review of Statistics and Its Application.

[27]  Brian A. Bottge,et al.  Teaching Mathematical Problem Solving to Middle School Students in Math, Technology Education, and Special Education Classrooms , 2004 .

[28]  T. Heinen,et al.  Latent Class and Discrete Latent Trait Models: Similarities and Differences , 1996 .

[29]  Allan S. Cohen,et al.  Latent Transition Analysis With a Mixture Item Response Theory Measurement Model , 2010 .

[30]  Bengt Muthén,et al.  Multilevel Latent Class Analysis: An Application of Adolescent Smoking Typologies With Individual and Contextual Predictors , 2010, Structural equation modeling : a multidisciplinary journal.

[31]  H. Akaike,et al.  Information Theory and an Extension of the Maximum Likelihood Principle , 1973 .

[32]  G. Schwarz.  Estimating the Dimension of a Model , 1978 .

[33]  Allan S. Cohen,et al.  A Multilevel Mixture IRT Model With an Application to DIF , 2010 .

[34]  B. Muthén,et al.  Deciding on the Number of Classes in Latent Class Analysis and Growth Mixture Modeling: A Monte Carlo Simulation Study , 2007 .

[35]  Sun-Joo Cho,et al.  Explanatory Secondary Dimension Modeling of Latent Differential Item Functioning , 2011 .

[36]  Mary Heinrichs,et al.  Weighing the Benefits of Anchored Math Instruction for Students with Disabilities in General Education Classes , 2002 .

[37]  Jürgen Rost,et al.  Rasch Models in Latent Classes: An Integration of Two Approaches to Item Analysis , 1990 .

[38]  C. Stein,et al.  Structural equation modeling. , 2012, Methods in molecular biology.

[39]  Allan S. Cohen,et al.  Model Selection Methods for Mixture Dichotomous IRT Models , 2009 .

[40]  R. P. McDonald,et al.  Nonlinear factor analysis. , 1967 .

[41]  Douglas Steinley,et al.  Examining Factor Score Distributions to Determine the Nature of Latent Spaces , 2007, Multivariate behavioral research.

[42]  Jeroen K. Vermunt,et al.  7. Multilevel Latent Class Models , 2003 .

[43]  Multilevel Mixture Models , 2006 .

[44]  M. Meulders,et al.  A conceptual and psychometric framework for distinguishing categories and dimensions. , 2005, Psychological review.

[45]  Matthias von Davier,et al.  PARTIALLY OBSERVED MIXTURES OF IRT MODELS: AN EXTENSION OF THE GENERALIZED PARTIAL CREDIT MODEL , 2003 .

[46]  Sylvia Frühwirth-Schnatter,et al.  Finite Mixture and Markov Switching Models , 2006 .