The Effects of Ignoring a Level in Multilevel Analysis

Ignoring a level can have a substantial impact on the conclusions of a multilevel analysis. For intercept-only models and for balanced data, we derive these effects analytically. For more complex random intercept models or for unbalanced data, a simulation study is performed. Most important effects concern estimates and corresponding standard errors of the variance parameters at adjacent levels and of the coefficients of the predictors at the ignored and bordering levels. Therefore, we conclude that if the researcher is interested in a specific level, she/he should account for both the upper and lower level. Conclusions are illustrated using empirical data from educational research.

[1]  Joop J. Hox,et al.  Applied Multilevel Analysis. , 1995 .

[2]  Karl J. Friston,et al.  Variance Components , 2003 .

[3]  P. Mortimore,et al.  School Matters: The Junior Years , 1995 .

[4]  Kenneth J. Rowe,et al.  Multilevel Modelling in School Effectiveness Research. , 1996 .

[5]  G. A. Marcoulides.  Multilevel Analysis Techniques and Applications , 2002 .

[6]  Mirjam Moerbeek,et al.  The Consequence of Ignoring a Level of Nesting in Multilevel Analysis , 2004, Multivariate behavioral research.

[7]  Johannes Berkhof.  Specification methods for the multilevel model , 2000 .

[8]  Anthony S. Bryk,et al.  Hierarchical Linear Models: Applications and Data Analysis Methods , 1992 .

[9]  Mattei Dogan,et al.  Quantitative Ecological Analysis in the Social Sciences. , 1969 .

[10]  Risto Lethonen.  Multilevel Statistical Models (3rd ed.) , 2005 .

[11]  Geert Molenberghs,et al.  Linear Mixed Models in Practice , 1997 .

[12]  Patrick Onghena,et al.  A New Study on Educational Effectiveness in Secondary Schools in Flanders: An Introduction , 2002 .

[13]  Anne Konu,et al.  Evaluation of Well-Being in Schools – A Multilevel Analysis of General Subjective Well-Being , 2002 .

[14]  Kosuke Imai,et al.  Survey Sampling , 1998, Nov/Dec 2017.

[15]  H. Alker.  A typology of ecological fallacies , 1969 .

[16]  Patrick Onghena,et al.  The Effect of Schools and Classes on Mathematics Achievement , 2002 .

[17]  I. Kreft.  Are multilevel techniques necessary?: An overview, including simulation studies , 2005 .

[18]  R. Brennan.  Elements of generalizability theory , 1983 .

[19]  Michael P. Cohen.  Determining sample size for surveys with data analyzed by hierarchical linear models , 1998 .

[20]  Patrick Onghena,et al.  The effect of schools and classes on language achievement , 2003 .

[21]  Harry J M Hüttner,et al.  The Multilevel Design: A Guide with an Annotated Bibliography, 1980-1993 , 1995 .

[22]  Roel Bosker,et al.  Multilevel analysis : an introduction to basic and advanced multilevel modeling , 1999 .

[23]  René Veenstra,et al.  Leerlingen – klassen – scholen: Prestaties en vorderingen van leerlingen in het voortgezet onderwijs , 1999 .

[24]  Geert Molenberghs,et al.  Linear Mixed Models in Practice: A SAS-Oriented Approach , 1997 .

[25]  Anthony S. Bryk,et al.  A Hierarchical Model for Studying School Effects , 1986 .

[26]  Timothy J. Robinson,et al.  Multilevel Analysis: Techniques and Applications , 2002 .

[27]  Marie-Christine Opdenakker,et al.  The Importance of Identifying Levels in Multilevel Analysis: An Illustration of the Effects of Ignoring the Top or Intermediate Levels in School Effectiveness Research , 2000 .