Assumptions of cross-level measurement and structural invariance in the analysis of multilevel data: Problems and solutions

This article demonstrates assumptions of invariance that researchers often implicitly make when analyzing multilevel data. The first set of assumptions is measurementbased and corresponds to the fact that researchers often conduct single-level exploratory and confirmatory factor analyses, and reliability analyses, with multilevel data. The second assumption, that of structural invariance, is engineered into the common multilevel random coefficient model, in that such analyses impose structural invariance across multiple levels of analysis when lower-level relationships represent both between- and within-groups effects. The nature of these assumptions, and ways to address their tenability, are explored from a conceptual standpoint. Then an empirical example of these assumptions and ways to address them is provided.

[1]  J. M. Cortina,et al.  What Is Coefficient Alpha? An Examination of Theory and Applications , 1993 .

[2]  Stephen J. Gerras,et al.  Climate as a moderator of the relationship between leader-member exchange and content specific citizenship: safety climate as an exemplar. , 2003, The Journal of applied psychology.

[3]  Noreen M. Webb,et al.  Between-class and within-class effects in a reported aptitude * treatment interaction: Reanalysis of a study by G. L. Anderson. , 1975 .

[4]  B. Muthén.  Latent variable modeling in heterogeneous populations , 1989 .

[5]  John B. Nezlek,et al.  Using hierarchical linear modeling to analyze grouped data. , 1998 .

[6]  Bengt Muthén,et al.  Multilevel Factor Analysis of Class and Student Achievement Components , 1991 .

[7]  K. Sirotnik.  PSYCHOMETRIC IMPLICATIONS OF THE UNIT‐OF‐ANALYSIS PROBLEM (WITH EXAMPLES FROM THE MEASUREMENT OF ORGANIZATIONAL CLIMATE) , 1980 .

[8]  K. R. Milner,et al.  A multiple-goal, multilevel model of feedback effects on the regulation of individual and team performance. , 2004, The Journal of applied psychology.

[9]  Nicholas D. Myers,et al.  Reciprocal Relationships Between Collective Efficacy and Team Performance in Women's Ice Hockey. , 2004 .

[10]  L. Corrado.  Generalized Latent Variable Modeling: Multilevel, Longitudinal, and Structural Equation Models , 2005 .

[11]  Jay Magidson,et al.  Advances in factor analysis and structural equation models , 1979 .

[12]  Dwight D. Frink,et al.  The Effects of Organizational Restructure on Employee Satisfaction , 1996 .

[13]  B. Muthén,et al.  Multilevel Covariance Structure Analysis , 1994 .

[14]  B. N. Pollack,et al.  Hierarchical linear modeling and the "unit of analysis" problem: A solution for analyzing responses of intact group members. , 1998 .

[15]  D. Hofmann,et al.  The Structure and Function of Collective Constructs: Implications for Multilevel Research and Theory Development , 1999 .

[16]  Sandra E. Moritz,et al.  Levels of analysis issues in group psychology: Using efficacy as an example of a multilevel model. , 1998 .

[17]  Nicholas D. Myers,et al.  Collective Efficacy and Team Performance: A Longitudinal Study of Collegiate Football Teams , 2004 .

[18]  Rosalie J. Hall,et al.  Applying multilevel confirmatory factor analysis techniques to the study of leadership , 2005 .

[19]  John E. Mathieu,et al.  A FRAMEWORK FOR CONDUCTING MULTI-LEVEL CONSTRUCT VALIDATION , 2005 .

[20]  I G Kreft,et al.  The Effect of Different Forms of Centering in Hierarchical Linear Models. , 1995, Multivariate behavioral research.

[21]  A. Satorra,et al.  Complex Sample Data in Structural Equation Modeling , 1995 .

[22]  S. Rabe-Hesketh,et al.  Generalized multilevel structural equation modeling , 2004 .

[23]  Richard,et al.  Motivation through the Design of Work: Test of a Theory. , 1976 .

[24]  R. Vandenberg,et al.  A Review and Synthesis of the Measurement Invariance Literature: Suggestions, Practices, and Recommendations for Organizational Research , 2000 .

[25]  Bengt Muthen,et al.  10. Latent Variable Modeling of Longitudinal and Multilevel Data , 1997 .

[26]  Anthony S. Bryk,et al.  Hierarchical Linear Models: Applications and Data Analysis Methods , 1992 .

[27]  J. Nunnally.  Psychometric Theory (2nd ed), New York: McGraw-Hill. , 1978 .

[28]  D. A. Kenny,et al.  Separating individual and group effects , 1985 .

[29]  M. R. Novick,et al.  Statistical Theories of Mental Test Scores. , 1971 .

[30]  G. A. Marcoulides,et al.  Multilevel Analysis Techniques and Applications , 2002 .

[31]  A. Luong,et al.  Meetings and More Meetings: The Relationship Between Meeting Load and the Daily Well-Being of Employees. , 2005 .

[32]  J. Mathieu,et al.  Conceptual Framework and Statistical Procedures for Delineating and Testing Multilevel Theories of Homology , 2005 .

[33]  Timothy J. Robinson,et al.  Multilevel Analysis: Techniques and Applications , 2002 .

[34]  K. Klein,et al.  Levels Issues in Theory Development, Data Collection, and Analysis , 1994 .

[35]  Lee J. Cronbach,et al.  Research on Classrooms and Schools: Formulation of Questions, Design and Analysis. , 1976 .

[36]  P. Bliese.  Group Size, ICC Values, and Group-Level Correlations: A Simulation , 1998 .

[37]  Jill C. Bradley,et al.  The big five personality traits and individual job performance growth trajectories in maintenance and transitional job stages. , 2004, The Journal of applied psychology.