Data Aggregation in Multilevel Analysis: A Review of Conceptual and Statistical Issues

Understanding that the behavior of people takes place within a context, over the past 20 years research in education and the sport sciences has witnessed an increasing development of multilevel frameworks that are both conceptually and methodologically sound. Despite these advances, the use of multilevel models and research designs in education still could be improved. As noted by recent editorial commentaries and reviews, confusion often exists about appropriate levels of analysis both at the conceptual and statistical level. The purpose of this article is threefold. First, we explain how multilevel conceptualizations can aid our understanding of behavior, especially within a given context. Second, we review three aggregation methods for analyzing multilevel data: within group agreement, intraclass correlations, and within and between analysis. Third, we provide a conceptual and statistical example to demonstrate how these aggregation techniques could be utilized to answer questions in sport and physical activity.

[1]  Ann A O'Connell,et al.  Applications of Hierarchical Linear Models for Evaluations of Health Interventions , 2004, Evaluation & the health professions.

[2]  Michael K. Lindell,et al.  Recent Issues Regarding rWG, rWG, rWG(J), and rWG(J) , 2005 .

[3]  D. Rousseau.  Issues of level in organizational research: Multi-level and cross-level perspectives. , 1985 .

[4]  Stephanie L. Castro,et al.  Data analytic methods for the analysis of multilevel questions: A comparison of intraclass correlation coefficients, rwg(j), hierarchical linear modeling, within- and between-analysis, and random group resampling , 2002 .

[5]  Henry L. Tosi,et al.  Multilevel Theory Building: Benefits, Barriers, and New Developments , 1999 .

[6]  K. Danylchuk,et al.  Understanding Retention Factors in Coaching and Athletic Management Positions , 1996 .

[7]  K. Klein,et al.  Levels Issues in Theory Development, Data Collection, and Analysis , 1994 .

[8]  Weimo Zhu,et al.  A multilevel analysis of school factors associated with health-related fitness. , 1997, Research quarterly for exercise and sport.

[9]  Lawrence R. James,et al.  Personality, affect, and behavior in groups revisited: Comment on aggregation, levels of analysis, and a recent application of within and between analysis. , 1993 .

[10]  J. Bartko,et al.  On Various Intraclass Correlation Reliability Coefficients , 1976 .

[11]  Anthony S. Bryk,et al.  Hierarchical Linear Models: Applications and Data Analysis Methods , 1992 .

[12]  Measurement of Moderate to Vigorous Physical Activity of Middle School Girls, Using TriTrac Activity Monitors During Small-Sided, Game-Based Lessons , 2003 .

[13]  David A. Hofmann,et al.  Multilevel Analytical Techniques: Commonalties, Differences, and Continuing Questions. , 2000 .

[14]  Jianmin Guan,et al.  Children's Motivation in Elementary Physical Education: A Longitudinal Study , 2004, Research quarterly for exercise and sport.

[15]  Cheri Ostroff,et al.  Moving HR to a higher level: HR practices and organizational effectiveness. , 2000 .

[16]  David W. Eccles,et al.  Why an Expert Team is More Than a Team of Experts: A Social-Cognitive Conceptualization of Team Coordination and Communication in Sport. , 2004 .

[17]  William L. Hays,et al.  Theory Testing in Organizational Behavior: The Varient Approach. , 1985 .

[18]  David A. Hofmann,et al.  Centering Decisions in Hierarchical Linear Models: Implications for Research in Organizations , 1998 .

[19]  L. Gauvin,et al.  Public and private sector responses to essential drugs policies: a multilevel analysis of drug prescription and selling practices in Mali. , 2003, Social science & medicine.

[20]  Orlando Behling,et al.  Some Problems in the Philosophy of Science of Organizations , 1978 .

[21]  M. Solmon,et al.  The Unit of Analysis in Field Research: Issues and Approaches to Design and Data Analysis , 1998 .

[22]  A. Knoppers.  Explaining Male Dominance and Sex Segregation in Coaching: Three Approaches , 1992 .

[23]  W. Bennis,et al.  The Social Psychology of Organizations , 1966 .

[24]  L. Maes,et al.  Can the school make a difference? A multilevel analysis of adolescent risk and health behaviour. , 2003, Social science & medicine.

[25]  L. James,et al.  Estimating within-group interrater reliability with and without response bias. , 1984 .

[26]  Cheri Ostroff.  The relationship between satisfaction, attitudes, and performance: An organizational level analysis. , 1992 .

[27]  S. Kozlowski,et al.  Multilevel Theory, Research, a n d M e t h o d s i n Organizations Foundations, Extensions, and New Directions , 2022 .

[28]  Ted A. Baumgartner,et al.  Measurement for Evaluation in Physical Education and Exercise Science , 1987 .

[29]  R. Y. Calne,et al.  Exercise and sport , 1975, Nature.

[30]  Fuzhong Li,et al.  Neighborhood-level influences on physical activity among older adults: a multilevel analysis. , 2004, Journal of aging and physical activity.

[31]  P. Bliese.  Within-group agreement, non-independence, and reliability: Implications for data aggregation and analysis. , 2000 .

[32]  Terry E. Duncan,et al.  Neighborhood Physical Activity Opportunity: A Multilevel Contextual Model , 2002, Research quarterly for exercise and sport.

[33]  J. Kras.  Measurement for Evaluation in Physical Education and Exercise Science, 7th Edition , 2003 .

[34]  Terry E. Duncan,et al.  A multilevel analysis of sibling physical activity. , 2004 .

[35]  K. McGraw,et al.  Forming inferences about some intraclass correlation coefficients. , 1996 .

[36]  T. Eitle,et al.  Segregation and School Violence , 2003 .

[37]  James D. Thompson.  Organizations in Action , 1967 .

[38]  D. Hofmann.  An Overview of the Logic and Rationale of Hierarchical Linear Models , 1997 .

[39]  J. Fleiss,et al.  Intraclass correlations: uses in assessing rater reliability. , 1979, Psychological bulletin.

[40]  G. Northcraft,et al.  You have printed the following article : Why Differences Make a Difference : A Field Study of Diversity , Conflict , and Performance in Workgroups , 2007 .

[41]  L. J. Williams,et al.  The cross-level operator in regression, ancova, and contextual analysis , 2000 .

[42]  L. James.  Aggregation Bias in Estimates of Perceptual Agreement. , 1982 .

[43]  John P. Meyer,et al.  The measurement and antecedents of affective, continuance and normative commitment to the organization , 1990 .

[44]  A. H. Brayfield,et al.  AN INDEX OF JOB SATISFACTION , 1951 .

[45]  Stephen B. Fawcett,et al.  Measuring the Health Environment for Physical Activity and Nutrition among Youth: A Review of the Literature and Applications for Community Initiatives☆ , 2000 .

[46]  K. Lewin.  Field theory in social science , 1951 .

[47]  S. Kozlowski,et al.  A disagreement about within-group agreement: Disentangling issues of consistency versus consensus. , 1992 .

[48]  P. Bliese.  Group Size, ICC Values, and Group-Level Correlations: A Simulation , 1998 .

[49]  L. James,et al.  rwg: An assessment of within-group interrater agreement. , 1993 .

[50]  K. Jehn.  A Multimethod Examination of the Benefits and Detriments of Intragroup Conflict , 1995 .