Doubly-Latent Models of School Contextual Effects: Integrating Multilevel and Structural Equation Approaches to Control Measurement and Sampling Error

This article is a methodological-substantive synergy. Methodologically, we demonstrate latent-variable contextual models that integrate structural equation models (with multiple indicators) and multilevel models. These models simultaneously control for and unconfound measurement error due to sampling of items at the individual (L1) and group (L2) levels and sampling error due the sampling of persons in the aggregation of L1 characteristics to form L2 constructs. We consider a set of models that are latent or manifest in relation to sampling items (measurement error) and sampling of persons (sampling error) and discuss when different models might be most useful. We demonstrate the flexibility of these 4 core models by extending them to include random slopes, latent (single-level or cross-level) interactions, and latent quadratic effects. Substantively we use these models to test the big-fish-little-pond effect (BFLPE), showing that individual student levels of academic self-concept (L1-ASC) are positively associated with individual level achievement (L1-ACH) and negatively associated with school-average achievement (L2-ACH)—a finding with important policy implications for the way schools are structured. Extending tests of the BFLPE in new directions, we show that the nonlinear effects of the L1-ACH (a latent quadratic effect) and the interaction between gender and L1-ACH (an L1 × L1 latent interaction) are not significant. Although random-slope models show no significant school-to-school variation in relations between L1-ACH and L1-ASC, the negative effects of L2-ACH (the BFLPE) do vary somewhat with individual L1-ACH. We conclude with implications for diverse applications of the set of latent contextual models, including recommendations about their implementation, effect size estimates (and confidence intervals) appropriate to multilevel models, and directions for further research in contextual effect analysis.

[1]  H. Marsh,et al.  Long-Term Total Negative Effects of School-Average Ability on Diverse Educational Outcomes , 2010 .

[2]  David M. Lahuis,et al.  The Accuracy of Significance Tests for Slope Variance Components in Multilevel Random Coefficient Models , 2009 .

[3]  Joseph Hilbe,et al.  Data Analysis Using Regression and Multilevel/Hierarchical Models , 2009 .

[4]  Susan J. Curry,et al.  Extended Generalized Linear Latent and Mixed Model , 2008 .

[5]  B. Muthén,et al.  The multilevel latent covariate model: a new, more reliable approach to group-level effects in contextual studies. , 2008, Psychological methods.

[6]  H. Marsh,et al.  Social Comparison and Big-Fish-Little-Pond Effects on Self-Concept and Other Self-Belief Constructs: Role of Generalized and Specific Others. , 2008 .

[7]  Snigdhansu Chatterjee,et al.  Structural Equation Modeling, A Bayesian Approach , 2008, Technometrics.

[8]  Ulrich Trautwein,et al.  The Big-fish–little-pond-effect Stands Up to Critical Scrutiny: Implications for Theory, Methodology, and Future Research , 2008 .

[9]  Pascal Huguet,et al.  In search of the big fish: investigating the coexistence of the big-fish-little-pond effect with the positive effects of upward comparisons. , 2008, The British journal of social psychology.

[10]  Andrew J. Martin,et al.  How we judge ourselves from different perspectives: Contextual influences on self-concept formation , 2008 .

[11]  Ann A. O'Connell,et al.  Multilevel modeling of educational data , 2008 .

[12]  Howard Eisner,et al.  Information Age , 2008, Wiley Encyclopedia of Computer Science and Engineering.

[13]  J. Milyo,et al.  Estimating the Impact of State Policies and Institutions with Mixed-Level Data , 2007, State Politics & Policy Quarterly.

[14]  James A. Bovaird,et al.  Unconstrained Structural Equation Models of Latent Interactions: Contrasting Residual- and Mean-Centered Approaches , 2007 .

[15]  Christopher Winship,et al.  Counterfactuals and Causal Inference: Methods and Principles for Social Research , 2007 .

[16]  Craig K. Enders,et al.  Centering predictor variables in cross-sectional multilevel models: a new look at an old issue. , 2007, Psychological methods.

[17]  J. Wilcox,et al.  Reconsidering formative measurement. , 2007, Psychological methods.

[18]  James A. Bovaird.  Multilevel Structural Equation Models for Contextual Factors , 2007 .

[19]  James A. Bovaird,et al.  Modeling contextual effects in longitudinal studies , 2007 .

[20]  M. Croon,et al.  Predicting group-level outcome variables from variables measured at the individual level: a latent variable multilevel model. , 2007, Psychological methods.

[21]  Sik-Yum Lee,et al.  Structural equation modelling: A Bayesian approach. , 2007 .

[22]  H. Marsh,et al.  Applications of latent-variable models in educational psychology: The need for methodological-substantive synergies. , 2007 .

[23]  Bengt Muthén,et al.  Constructing Covariates in Multilevel Regression , 2007 .

[24]  S. Rabe-Hesketh,et al.  Multilevel modelling of complex survey data , 2006 .

[25]  H. Marsh,et al.  Reciprocal Effects of Self-Concept and Performance From a Multidimensional Perspective: Beyond Seductive Pleasure and Unidimensional Perspectives , 2006, Perspectives on psychological science : a journal of the Association for Psychological Science.

[26]  Ulrich Trautwein,et al.  Integration of multidimensional self-concept and core personality constructs: construct validation and relations to well-being and achievement. , 2006, Journal of personality.

[27]  Tapabrata Maiti,et al.  Principles and Practice of Structural Equation Modeling (2nd ed.) , 2006 .

[28]  Andrew J. Martin,et al.  A Multimethod Perspective on Self-Concept Research in Educational Psychology: A Construct Validity Approach. , 2006 .

[29]  Tihomir Asparouhov,et al.  Multilevel Modeling of Complex Survey Data , 2006 .

[30]  U. Trautwein,et al.  Big-Fish-Little-Pond effect , 2006, Vernon Wall Lecture.

[31]  U. Trautwein,et al.  Entwicklung eines Instruments zur Erfassung des Selbstkonzepts junger Erwachsener , 2005 .

[32]  Michael C Neale,et al.  People are variables too: multilevel structural equations modeling. , 2005, Psychological methods.

[33]  H. Marsh,et al.  Teacher frame of reference and the big-fish-little-pond effect , 2005 .

[34]  S. Perren,et al.  A Two-Level Confirmatory Factor Analysis of a Modified Rosenberg Self-Esteem Scale , 2005 .

[35]  Stan Lipovetsky,et al.  Generalized Latent Variable Modeling: Multilevel,Longitudinal, and Structural Equation Models , 2005, Technometrics.

[36]  Risto Lehtonen,et al.  Multilevel Statistical Models , 2005 .

[37]  J. Gill.  Hierarchical Linear Models , 2005 .

[38]  G. A. Marcoulides,et al.  Using the Delta Method for Approximate Interval Estimation of Parameter Functions in SEM , 2004 .

[39]  H. Marsh,et al.  Structural equation models of latent interactions: evaluation of alternative estimation strategies and indicator construction. , 2004, Psychological methods.

[40]  Peter Tymms,et al.  Effect sizes in multilevel models. , 2004 .

[41]  P. Tymms,et al.  The Effects of Student Composition on School Outcomes , 2004 .

[42]  S. Rabe-Hesketh,et al.  Generalized multilevel structural equation modeling , 2004 .

[43]  H. Marsh,et al.  The Big-Fish-Little-Pond Effect Stands Up to Scrutiny. , 2004 .

[44]  Karen Elliot,et al.  But what does it mean? The use of effect sizes in educational research , 2004 .

[45]  H. Marsh,et al.  Big-fish-little-pond effect on academic self-concept. A cross-cultural (26-country) test of the negative effects of academically selective schools. , 2003, The American psychologist.

[46]  Harvey Goldstein,et al.  Multilevel modelling of educational data , 2003 .

[47]  Karl J. Friston,et al.  Variance Components , 2003 .

[48]  D. A. Kenny,et al.  The statistical analysis of data from small groups. , 2002, Journal of personality and social psychology.

[49]  B. Thompson.  What Future Quantitative Social Science Research Could Look Like: Confidence Intervals for Effect Sizes , 2002 .

[50]  B. Muthén.  BEYOND SEM: GENERAL LATENT VARIABLE MODELING , 2002 .

[51]  G. A. Marcoulides,et al.  Multilevel Analysis Techniques and Applications , 2002 .

[52]  Petter Laake,et al.  Regression among factor scores , 2001 .

[53]  James Algina,et al.  A Note on Estimating the Jöreskog-Yang Model for Latent Variable Interaction Using LISREL 8.3 , 2001 .

[54]  Helfried Moosbrugger,et al.  Maximum likelihood estimation of latent interaction effects with the LMS method , 2000 .

[55]  D. Kaplan.  Structural Equation Modeling: Foundations and Extensions , 2000 .

[56]  R. Bagozzi,et al.  On the nature and direction of relationships between constructs and measures. , 2000, Psychological methods.

[57]  H. Marsh,et al.  Longitudinal multilevel models of the big-fish-little-pond effect on academic self-concept: counterbalancing contrast and reflected-glory effects in Hong Kong schools. , 2000, Journal of personality and social psychology.

[58]  P. Bliese.  Within-group agreement, non-independence, and reliability: Implications for data aggregation and analysis. , 2000 .

[59]  Wilfried Bos,et al.  TIMSS/III Dritte Internationale Mathematik- und Naturwissenschaftsstudie — Mathematische und naturwissenschaftliche Bildung am Ende der Schullaufbahn , 2000 .

[60]  S. Kozlowski,et al.  Multilevel Theory, Research, a n d M e t h o d s i n Organizations Foundations, Extensions, and New Directions , 2022 .

[61]  M. Csíkszentmihályi,et al.  Positive psychology. An introduction. , 2000, The American psychologist.

[62]  S. Lohr,et al.  Sampling: Design and Analysis , 1999 .

[63]  Roel Bosker,et al.  Multilevel analysis : an introduction to basic and advanced multilevel modeling , 1999 .

[64]  Alexander Seeshing Yeung,et al.  Longitudinal Structural Equation Models of Academic Self-Concept and Achievement: Gender Differences in the Development of Math and English Constructs , 1998 .

[65]  Rex B. Kline,et al.  Principles and Practice of Structural Equation Modeling , 1998 .

[66]  Wai-Yin Poon,et al.  ANALYSIS OF TWO-LEVEL STRUCTURAL EQUATION MODELS VIA EM TYPE ALGORITHMS , 1998 .

[67]  Jerome S. Bruner,et al.  A Narrative Model of Self‐Construction , 1997, Psyke & Logos.

[68]  H. Marsh,et al.  The Negative Effects of School-Average Ability on Academic Self-Concept: An Application of Multilevel Modelling , 1996 .

[69]  H. Marsh,et al.  The Effects of Gifted and Talented Programs on Academic Self-Concept: The Big Fish Strikes Again , 1995 .

[70]  I G Kreft,et al.  The Effect of Different Forms of Centering in Hierarchical Linear Models. , 1995, Multivariate behavioral research.

[71]  A. Satorra,et al.  Complex Sample Data in Structural Equation Modeling , 1995 .

[72]  Herbert W. Marsh,et al.  Using the National Longitudinal Study of 1988 to evaluate theoretical models of self-concept: The Self-Description Questionnaire. , 1994 .

[73]  Roderick P. McDonald,et al.  The Bilevel Reticular Action Model for Path Analysis with Latent Variables , 1994 .

[74]  B. Muthén,et al.  Multilevel Covariance Structure Analysis , 1994 .

[75]  K. Gavin.  The Effects of Gifted and Talented Programs on Students' Socialization Skills , 1994 .

[76]  Roderick P. McDonald,et al.  A general model for two-level data with responses missing at random , 1993 .

[77]  Bengt Muthén,et al.  Multilevel Factor Analysis of Class and Student Achievement Components , 1991 .

[78]  Herbert W. Marsh,et al.  Failure of High-Ability High Schools to Deliver Academic Benefits Commensurate With Their Students’ Ability Levels , 1991 .

[79]  R. Lennox,et al.  Conventional wisdom on measurement: A structural equation perspective. , 1991 .

[80]  Robert M. O'Brien,et al.  Estimating the Reliability of Aggregate-Level Variables Based on Individual-Level Characteristics , 1990 .

[81]  B. Muthén.  Latent variable modeling in heterogeneous populations , 1989 .

[82]  Harvey Goldstein,et al.  A general model for the analysis of multilevel data , 1988 .

[83]  Herbert W. Marsh,et al.  The big-fish-little-pond effect on academic self-concept , 1987 .

[84]  P. Abrami,et al.  Students' Evaluations of University Teaching: Research Findings, Methodological Issues, and Directions for Future Research , 1987 .

[85]  H. Marsh.  Self-concept: The Application of a Frame of Reference Model to Explain Paradoxical Results , 1984 .

[86]  Herbert W. Marsh,et al.  Determinants of student self-concept: Is it better to be a relatively large fish in a small pond even if you don't learn to swim as well? , 1984 .

[87]  Herbert W. Marsh,et al.  Self Description Questionnaire III: The construct validity of multidimensional self-concept ratings by late adolescents. , 1984 .

[88]  Herbert W. Marsh,et al.  The Influence of Student, Course, and Instructor Characteristics in Evaluations of University Teaching , 1980 .

[89]  Kjell Harnqvist,et al.  Primary Mental Abilities at Collective and Individual Levels. , 1978 .

[90]  M. Kane,et al.  STUDENT EVALUATIONS OF TEACHING: THE GENERALIZABILITY OF CLASS MEANS , 1976 .

[91]  Lee J. Cronbach,et al.  Research on Classrooms and Schools: Formulation of Questions, Design and Analysis. , 1976 .

[92]  Kenneth E. Harper,et al.  Contextual analysis , 1957, Mech. Transl. Comput. Linguistics.