Confidence intervals and sample size calculations for the weighted eta-squared effect sizes in one-way heteroscedastic ANOVA

Effect size reporting and interpreting practices have been extensively recommended in academic journals when primary outcomes of all empirical studies have been analyzed. This article presents an alternative approach to constructing confidence intervals of the weighted eta-squared effect size within the context of one-way heteroscedastic ANOVA models. It is shown that the proposed interval procedure has advantages over an existing method in its theoretical justification, computational simplicity, and numerical performance. For design planning, the corresponding sample size procedures for precise interval estimation of the weighted eta-squared association measure are also delineated. Specifically, the developed formulas compute the necessary sample sizes with respect to the considerations of expected confidence interval width and tolerance probability of interval width within a designated value. Supplementary computer programs are provided to aid the implementation of the suggested techniques in practical applications of ANOVA designs when the assumption of homogeneous variances is not tenable.

[1]  Kate E Decleene,et al.  Publication Manual of the American Psychological Association , 2011 .

[2]  Scott E. Maxwell,et al.  Designing Experiments and Analyzing Data: A Model Comparison Perspective , 1990 .

[3]  Franklin A. Graybill,et al.  Introduction to The theory , 1974 .

[4]  H. Keselman,et al.  Consequences of Assumption Violations Revisited: A Quantitative Review of Alternatives to the One-Way Analysis of Variance F Test , 1996 .

[5]  Effect Size Reporting Practices in Published Articles , 2009 .

[6]  J. Wolfowitz,et al.  Introduction to the Theory of Statistics. , 1951 .

[7]  B. Thompson,et al.  How to Estimate and Interpret Various Effect Sizes , 2004 .

[8]  R. Grissom,et al.  Heterogeneity of variance in clinical data. , 2000, Journal of consulting and clinical psychology.

[9]  G. Keppel Design and analysis: A researcher's handbook, 3rd ed. , 1991 .

[10]  Michael R. Harwell,et al.  Summarizing Monte Carlo Results in Methodological Research: The One- and Two-Factor Fixed Effects ANOVA Cases , 1992 .

[11]  Wei Pan,et al.  A Comprehensive Review of Effect Size Reporting and Interpreting Practices in Academic Journals in Education and Psychology. , 2010 .

[12]  John T. E. Richardson Measures of effect size , 1996 .

[13]  R. Haase Classical and Partial Eta Square in Multifactor ANOVA Designs , 1983 .

[14]  J. Algina,et al.  Generalized eta and omega squared statistics: measures of effect size for some common research designs. , 2003, Psychological methods.

[15]  G. Casella,et al.  Statistical Inference , 2003, Encyclopedia of Social Network Analysis and Mining.

[16]  Scott E. Maxwell,et al.  Designing Experiments and Analyzing Data , 1992 .

[17]  E. Kulinskaya,et al.  Power Approximations in Testing for Unequal Means in a One-Way ANOVA Weighted for Unequal Variances , 2003 .

[18]  D. Sharpe Beyond Significance Testing: Reforming Data Analysis Methods in Behavioral Research. , 2004 .

[19]  C. Ferguson An effect size primer: A guide for clinicians and researchers. , 2009 .

[20]  J. H. Steiger,et al.  Beyond the F test: Effect size confidence intervals and tests of close fit in the analysis of variance and contrast analysis. , 2004, Psychological methods.

[21]  G. Keppel,et al.  Design and Analysis: A Researcher's Handbook , 1976 .

[22]  Michael Smithson,et al.  Correct Confidence Intervals for Various Regression Effect Sizes and Parameters: The Importance of Noncentral Distributions in Computing Intervals , 2001 .

[23]  R. Kirk Practical Significance: A Concept Whose Time Has Come , 1996 .

[24]  E. Odgaard,et al.  Confidence intervals for effect sizes: compliance and clinical significance in the Journal of Consulting and clinical Psychology. , 2010, Journal of consulting and clinical psychology.

[25]  H. Keselman,et al.  Approximate degrees of freedom tests: A unified perspective on testing for mean equality. , 1995 .

[26]  Herman Aguinis,et al.  Cautionary Note on Reporting Eta-Squared Values from Multifactor ANOVA Designs , 2004 .

[27]  William L. Hays,et al.  Statistics, 5th ed. , 1994 .

[28]  P. Lachenbruch Statistical Power Analysis for the Behavioral Sciences (2nd ed.) , 1989 .

[29]  D. Bonett Confidence intervals for standardized linear contrasts of means. , 2008, Psychological methods.

[30]  Ronald C. Serlin,et al.  QUANTITATIVE METHODS IN PSYCHOLOGY Comparison of ANOVA Alternatives Under Variance Heterogeneity and Specific Noncentrality Structures , 1986 .

[31]  K. J. Levy Some empirical power results associated with welch's robust analysis of variance technique , 1978 .

[32]  Ken Kelley,et al.  Methods for the Behavioral, Educational, and Social Sciences: An R package , 2007, Behavior research methods.

[33]  Kenneth A. Bollen,et al.  Monte Carlo Experiments: Design and Implementation , 2001 .

[34]  Lawrence L. Kupper,et al.  How Appropriate are Popular Sample Size Formulas , 1989 .

[35]  Allen I. Fleishman Confidence Intervals for Correlation Ratios , 1980 .

[36]  Scott E. Maxwell,et al.  Measures of strength of association: A comparative examination , 1981 .

[37]  B. L. Welch ON THE COMPARISON OF SEVERAL MEAN VALUES: AN ALTERNATIVE APPROACH , 1951 .

[38]  Joseph R. Rausch,et al.  Sample size planning for statistical power and accuracy in parameter estimation. , 2008, Annual review of psychology.

[39]  R. Grissom,et al.  Review of assumptions and problems in the appropriate conceptualization of effect size. , 2001, Psychological methods.

[40]  Jacob Cohen Measurement Educational and Psychological Educational and Psychological Measurement Eta-squared and Partial Eta-squared in Fixed Factor Anova Designs Educational and Psychological Measurement Additional Services and Information For , 2022 .

[41]  James Algina,et al.  A generally robust approach for testing hypotheses and setting confidence intervals for effect sizes. , 2008, Psychological methods.

[42]  T. Levine,et al.  Eta Squared, Partial Eta Squared, and Misreporting of Effect Size in Communication Research , 2002 .

[43]  Kent B. Monroe,et al.  Effect-Size Estimates: Issues and Problems in Interpretation , 1996 .

[44]  Carl J. Huberty,et al.  Statistical Practices of Educational Researchers: An Analysis of their ANOVA, MANOVA, and ANCOVA Analyses , 1998 .

[45]  K. Goulden,et al.  Effect Sizes for Research: A Broad Practical Approach , 2006 .

[46]  Jacob Cohen Statistical Power Analysis for the Behavioral Sciences , 1969, The SAGE Encyclopedia of Research Design.

[47]  R G Staudte,et al.  Interval estimates of weighted effect sizes in the one-way heteroscedastic ANOVA. , 2006, The British journal of mathematical and statistical psychology.

[48]  Gerry Leversha,et al.  Statistical inference (2nd edn), by Paul H. Garthwaite, Ian T. Jolliffe and Byron Jones. Pp.328. £40 (hbk). 2002. ISBN 0 19 857226 3 (Oxford University Press). , 2003, The Mathematical Gazette.

[49]  John T. E. Richardson,et al.  Eta Squared and Partial Eta Squared as Measures of Effect Size in Educational Research. , 2011 .

[50]  G. Glass,et al.  Measures of Association in Comparative Experiments: Their Development and Interpretation , 1969 .

[51]  James A. Breaugh,et al.  Effect Size Estimation: Factors to Consider and Mistakes to Avoid , 2003 .