Estimating Statistical Power and Required Sample Sizes for Organizational Research Using Multilevel Modeling

The use of multilevel modeling to investigate organizational phenomena is rapidly increasing. Unfortunately, little advice is readily available for organizational researchers attempting to determine statistical power when using multilevel models or when determining sample sizes for each level that will maximize statistical power. This article presents an introduction to statistical power in multilevel models. The unique factors influencing power in multilevel models and calculations for estimating power for simple fixed effects, variance components, and cross-level interactions are presented. The results of simulation studies and the existing general rules of thumb are discussed, and the available power analysis software is reviewed.

[1]  D. Afshartous.  Determination of Sample Size for Multilevel Model Design , 2011 .

[2]  Joop J. Hox,et al.  Multilevel modeling: When and why , 1998 .

[3]  Timothy J. Robinson,et al.  Multilevel Analysis: Techniques and Applications , 2002 .

[4]  André I. Khuri,et al.  Designs for Variance Components Estimation: Past and Present , 2000 .

[5]  Michael P. Cohen.  Determining sample size for surveys with data analyzed by hierarchical linear models , 1998 .

[6]  Anthony S. Bryk,et al.  Hierarchical Linear Models: Applications and Data Analysis Methods , 1992 .

[7]  Robert S. Barcikowski,et al.  Statistical Power with Group Mean as the Unit of Analysis , 1981 .

[8]  Roel Bosker,et al.  Standard Errors and Sample Sizes for Two-Level Research , 1993 .

[9]  C. Brown,et al.  Principles for Designing Randomized Preventive Trials in Mental Health: An Emerging Developmental Epidemiology Paradigm , 1999, American journal of community psychology.

[10]  Jacob Cohen,et al.  A power primer. , 1992, Psychological bulletin.

[11]  James T. Austin,et al.  Statistical Conclusion Validity for Organizational Science Researchers: A Review , 1998 .

[12]  William J. Browne,et al.  Implementation and performance issues in the Bayesian and likelihood fitting of multilevel models , 2000, Comput. Stat..

[13]  D. Hofmann,et al.  The application of hierarchical linear modeling to organizational research. , 2000 .

[14]  G. A. Marcoulides.  Multilevel Analysis Techniques and Applications , 2002 .

[15]  S. Raudenbush.  Statistical analysis and optimal design for cluster randomized trials , 1997 .

[16]  S. Raudenbush.  Educational Applications of Hierarchical Linear Models: A Review , 1988 .

[17]  Roel Bosker,et al.  Multilevel analysis : an introduction to basic and advanced multilevel modeling , 1999 .

[18]  T. Scandura,et al.  Research Methodology In Management: Current Practices, Trends, And Implications For Future Research , 2000 .

[19]  X. Liu,et al.  Statistical Power and Optimum Sample Allocation Ratio for Treatment and Control Having Unequal Costs per Unit of Randomization , 2003 .

[20]  I. Kreft.  Are multilevel techniques necessary?: An overview, including simulation studies , 2005 .

[21]  Cora J. M. Maas,et al.  Robustness issues in multilevel regression analysis , 2004 .

[22]  Robert S. Barcikowski,et al.  Optimum Sample Size and Number of Levels in a One-Way Random-Effects Analysis of Variance , 1973 .

[23]  P. Bliese,et al.  Group Size and Measures of Group-Level Properties: An Examination of Eta-Squared and ICC Values , 1998 .

[24]  Tom A B Snijders,et al.  Power and sample size in multilevel modeling , 2006 .

[25]  D. Hofmann.  An Overview of the Logic and Rationale of Hierarchical Linear Models , 1997 .

[26]  H. Engelhardt,et al.  Hierarchical Linear Models: Applications and Data Analysis Methods.Anthony S. Bryk , Stephen W. Raudenbush , 1994 .

[27]  P. Lachenbruch.  Statistical Power Analysis for the Behavioral Sciences (2nd ed.) , 1989 .

[28]  Russell V Lenth,et al.  Power Analysis for Experimental Research: A Practical Guide For The Biological, Medical and Social Sciences and Statistical Power Analysis: A Simple and General Model for Traditional and Modern Hypothesis Tests , 2004 .

[29]  S L Normand,et al.  On determination of sample size in hierarchical binomial models , 2001, Statistics in medicine.

[30]  Mark L. Davison,et al.  Using Hierarchical Linear Models to Examine Moderator Effects: Person-by-Organization Interactions , 2002 .

[31]  Tom A. B. Snijders.  Power and Sample Size in Multilevel Linear Models , 2005 .

[32]  Jan de Leeuw,et al.  Introducing Multilevel Modeling , 1998 .

[33]  S. Raudenbush,et al.  Statistical power and optimal design for multisite randomized trials. , 2000, Psychological methods.

[34]  Kosuke Imai,et al.  Survey Sampling , 1998, Nov/Dec 2017.

[35]  J. Hox,et al.  Sufficient Sample Sizes for Multilevel Modeling , 2005 .

[36]  L. James.  Aggregation Bias in Estimates of Perceptual Agreement. , 1982 .

[37]  Jacob Cohen.  Statistical Power Analysis for the Behavioral Sciences , 1969, The SAGE Encyclopedia of Research Design.

[38]  P. Bliese.  Within-group agreement, non-independence, and reliability: Implications for data aggregation and analysis. , 2000 .

[39]  Mark A. Mone,et al.  THE PERCEPTIONS AND USAGE OF STATISTICAL POWER IN APPLIED PSYCHOLOGY AND MANAGEMENT RESEARCH , 1996 .

[40]  S. Kozlowski,et al.  Multilevel Theory, Research, a n d M e t h o d s i n Organizations Foundations, Extensions, and New Directions , 2022 .

[41]  M H Boyle,et al.  Multilevel modelling of hierarchical data in developmental studies. , 2001, Journal of child psychology and psychiatry, and allied disciplines.