The Accuracy of Significance Tests for Slope Variance Components in Multilevel Random Coefficient Models

This study examines the behavior of three tests for significant slope variance in multilevel random coefficient (MRC) models: the Hierarchical Linear Modeling chi-square test, the likelihood ratio test (LRT), and the corrected LRT. Monte Carlo simulations are conducted varying the numbers of groups, group size, and effect size. Results suggest that neither the number of groups nor group size influenced Type I errors. Group size has a stronger effect on power compared with the number of groups. The one-tailed LRT demonstrates the best balance between power and Type I errors. Recommendations for conducting MRC analyses are presented.

[1]  Stephanie C. Payne,et al.  Simultaneous Examination of the Antecedents and Consequences of Efficacy Beliefs at Multiple Levels of Analysis , 2002 .

[2]  Anthony S. Bryk,et al.  Hierarchical Linear Models: Applications and Data Analysis Methods , 1992 .

[3]  Jacob Cohen,et al.  A power primer. , 1992, Psychological bulletin.

[4]  Cora J. M. Maas,et al.  Robustness issues in multilevel regression analysis , 2004 .

[5]  David A. Hofmann,et al.  Centering Decisions in Hierarchical Linear Models: Implications for Research in Organizations , 1998 .

[6]  G. H. Lunney,et al.  USING ANALYSIS OF VARIANCE WITH A DICHOTOMOUS DEPENDENT VARIABLE: AN EMPIRICAL STUDY , 1970 .

[7]  Mark L. Davison,et al.  Using Hierarchical Linear Models to Examine Moderator Effects: Person-by-Organization Interactions , 2002 .

[8]  S. Jackson,et al.  Diversity in social context: a multi-attribute, multilevel analysis of team diversity and sales performance , 2004 .

[9]  Robert E. Ployhart,et al.  Longitudinal data analysis , 2002 .

[10]  H. Liao,et al.  A MULTILEVEL INVESTIGATION OF FACTORS INFLUENCING EMPLOYEE SERVICE PERFORMANCE AND CUSTOMER OUTCOMES , 2004 .

[11]  David M. Lahuis,et al.  Using Multilevel Random Coefficient Modeling to Investigate Rater Effects in Performance Ratings , 2007 .

[12]  Gillian B. Yeo,et al.  A multilevel analysis of effort, practice, and performance: effects; of ability, conscientiousness, and goal orientation. , 2004, The Journal of applied psychology.

[13]  Yuk Fai Cheong,et al.  HLM 6: Hierarchical Linear and Nonlinear Modeling , 2000 .

[14]  S. Raudenbush,et al.  Statistical power and optimal design for multisite randomized trials. , 2000, Psychological methods.

[15]  Paul D. Bliese,et al.  Multilevel random coefficient modeling in organizational research: Examples using SAS and S-PLUS. , 2002 .

[16]  Christopher L. Martin,et al.  A Multilevel Analysis of Procedural Justice Context , 1998 .

[17]  Nathan Bennett,et al.  Cohesiveness and Organizational Citizenship Behavior: A Multilevel Analysis Using Work Groups and Individuals , 1997 .

[18]  Neal Schmitt,et al.  An investigation of race and sex similarity effects in interviews: a multilevel approach to relational demography. , 2003, The Journal of applied psychology.

[19]  D. Stram,et al.  Variance components testing in the longitudinal mixed effects model. , 1994, Biometrics.

[20]  J. Stuart Bunderson,et al.  Recognizing and Utilizing Expertise in Work Groups: A Status Characteristics Perspective , 2003 .

[21]  Tania B. Huedo-Medina,et al.  Assessing heterogeneity in meta-analysis: Q statistic or I2 index? , 2006, Psychological methods.

[22]  G. A. Marcoulides,et al.  Multilevel Analysis Techniques and Applications , 2002 .

[23]  Jin Nam Choi.  Multilevel and Cross-Level Effects of Workplace Attitudes and Group Member Relations on Interpersonal Helping Behavior , 2006 .

[24]  D. Hofmann,et al.  The role of safety climate and communication in accident interpretation: Implications for learning from negative events , 1998 .

[25]  K. Liang,et al.  Asymptotic Properties of Maximum Likelihood Estimators and Likelihood Ratio Tests under Nonstandard Conditions , 1987 .

[26]  Conor Dolan,et al.  On the likelihood ratio test in structural equation modeling when parameters are subject to boundary constraints. , 2006, Psychological methods.

[27]  Thomas W. Britt,et al.  Social support, group consensus and stressor–strain relationships: Social context matters. , 2001 .

[28]  Roel Bosker,et al.  Multilevel analysis : an introduction to basic and advanced multilevel modeling , 1999 .

[29]  Nathan Bennett,et al.  A Case for Procedural Justice Climate: Development and Test of a Multilevel Model. , 2000 .

[30]  Tina T. Chen,et al.  Assessing the Burdens of Leadership: Effects of Formal Leadership Roles on Individual Performance Over Time , 2004 .

[31]  Nathan Bennett,et al.  Cohesiveness and Organizational Citizenship Behavior: A Multilevel Analysis Using Work Groups and Individuals , 1997 .

[32]  Jacob Cohen,et al.  Applied multiple regression/correlation analysis for the behavioral sciences , 1979 .

[33]  Michael R. Harwell.  An empirical study of the Hedges (1982) homogeneity test , 1997 .

[34]  Anders Skrondal,et al.  Likelihood Ratio Tests in Behavioral Genetics: Problems and Solutions , 2006, Behavior genetics.