Exploiting TIMSS & PIRLS combined data: multivariate multilevel modelling of student achievement

We exploit a multivariate multilevel model for the analysis of the Italian sample of the TIMSS\&PIRLS 2011 Combined International Database on fourth grade students. The multivariate approach jointly considers educational achievement on Reading, Mathematics and Science, thus allowing us to test for differential associations of the covariates with the three outcomes, and to estimate the residual correlations between pairs of outcomes at student and class levels. Multilevel modelling allows us to disentangle student and contextual factors affecting achievement. We also account for territorial differences in wealth by means of an index from an external source. The model residuals point out classes with high or low performance. As educational achievement is measured by plausible values, the estimates are obtained through multiple imputation formulas. The results, while confirming the role of traditional student and contextual factors, reveal interesting patterns of achievement in Italian primary schools.

[1]  Juliette F Mendelovits,et al.  Reading for change : performance and engagement across countries : results of PISA 2000 , 2003 .

[2]  Leonardo Grilli,et al.  Differential Variability of Test Scores among Schools: A Multilevel Analysis of the Fifth-Grade INVALSI Test Using Heteroscedastic Random Effects. , 2011 .

[3]  J. Schafer.  Multiple Imputation in Multivariate Problems When the Imputation and Analysis Models Differ , 2003 .

[4]  Margaret Wu.  The Role of Plausible Values in Large-Scale Surveys. , 2005 .

[5]  Martin Hecht,et al.  Nested multiple imputation in large-scale assessments , 2014, Large-scale Assessments in Education.

[6]  Matthias von Davier,et al.  International Large-Scale Assessment Data , 2010 .

[7]  Tom Schuller,et al.  Understanding the Social Outcomes of Learning , 2007 .

[8]  Robert S. Stawski,et al.  Multilevel Analysis: An Introduction to Basic and Advanced Multilevel Modeling (2nd Edition) , 2013 .

[9]  C. McCulloch,et al.  Misspecifying the Shape of a Random Effects Distribution: Why Getting It Wrong May Not Matter , 2011, 1201.1980.

[10]  Donald B. Rubin,et al.  Significance levels from repeated p-values with multiply imputed data , 1991 .

[11]  Harvey Goldstein,et al.  Fitting multilevel multivariate models with missing data in responses and covariates that may include interactions and non‐linear terms , 2014 .

[12]  S. Rabe-Hesketh,et al.  Multilevel modelling of complex survey data , 2006 .

[13]  Lena Osterhagen,et al.  Multiple Imputation For Nonresponse In Surveys , 2016 .

[14]  H. Goldstein,et al.  Multivariate multilevel analyses of examination results , 2002 .

[15]  Anthony S. Bryk,et al.  Hierarchical Linear Models: Applications and Data Analysis Methods , 1992 .

[16]  John Jerrim,et al.  Socioeconomic Gradients in Children's Cognitive Skills: Are Cross-Country Comparisons Robust to Who Reports Family Background? , 2014, SSRN Electronic Journal.

[17]  J. Reeve,et al.  What teachers say and do to support students' autonomy during a learning activity. , 2006 .

[18]  H. Goldstein.  Multilevel Statistical Models , 2006 .

[19]  Svend Kreiner,et al.  Analyses of Model Fit and Robustness. A New Look at the PISA Scaling Model Underlying Ranking of Countries According to Reading Literacy , 2014, Psychometrika.

[20]  Carla Rampichini,et al.  Specification of random effects in multilevel models: a review , 2015 .

[21]  Sophia Rabe-Hesketh,et al.  Multilevel and Longitudinal Modeling Using Stata , 2005 .

[22]  Roger A. Sugden,et al.  Multiple Imputation for Nonresponse in Surveys , 1988 .

[23]  Francesco Bartolucci,et al.  Assessment of School Performance Through a Multilevel Latent Markov Rasch Model , 2009, 0909.4961.

[24]  Harvey Goldstein,et al.  International comparisons of student attainment: some issues arising from the PISA study , 2004 .

[25]  J. Berkhof,et al.  Diagnostic Checks for Multilevel Models , 2008 .

[26]  Ina V. S. Mullis,et al.  TIMSS and PIRLS 2011: Relationships among Reading, Mathematics, and Science Achievement at the Fourth Grade--Implications for Early Learning. , 2013 .

[27]  Roel Bosker,et al.  Multilevel analysis : an introduction to basic and advanced multilevel modeling , 1999 .

[28]  Robert J. Mislevy,et al.  Randomization-based inference about latent variables from complex samples , 1991 .

[29]  Leslie Rutkowski,et al.  Handbook of International Large-Scale Assessment : Background, Technical Issues, and Methods of Data Analysis , 2013 .

[30]  Donia Smaali Bouhlila,et al.  Multiple imputation using chained equations for missing data in TIMSS: a case study , 2013, Large-scale Assessments in Education.