Confidence Intervals for Standardized Effect Sizes: Theory, Application, and Implementation

The behavioral, educational, and social sciences are undergoing a paradigmatic shift in methodology, from disciplines that focus on the dichotomous outcome of null hypothesis significance tests to disciplines that report and interpret effect sizes and their corresponding confidence intervals. Due to the arbitrariness of many measurement instruments used in the behavioral, educational, and social sciences, some of the most widely reported effect sizes are standardized. Although forming confidence intervals for standardized effect sizes can be very beneficial, such confidence interval procedures are generally difficult to implement because they depend on noncentral t, F, and x2 distributions. At present, no main-stream statistical package provides exact confidence intervals for standardized effects without the use of specialized programming scripts. Methods for the Behavioral, Educational, and Social Sciences (MBESS) is an R package that has routines for calculating confidence intervals for noncentral t, F, and x2 distributions, which are then used in the calculation of exact confidence intervals for standardized effect sizes by using the confidence interval transformation and inversion principles. The present article discusses the way in which confidence intervals are formed for standardized effect sizes and illustrates how such confidence intervals can be easily formed using MBESS in R.

[1]  Xiao-Hua Zhou,et al.  Statistical Methods for Meta‐Analysis , 2008 .

[2]  Ken Kelley,et al.  Methods for the Behavioral, Educational, and Social Sciences: An R package , 2007, Behavior research methods.

[3]  F. Scholz.  Applications of the Noncentral t – Distribution , 2007 .

[4]  James Algina,et al.  Confidence Interval Coverage for Cohen's Effect Size Statistic , 2006 .

[5]  Joseph R. Rausch,et al.  Sample size planning for the standardized mean difference: accuracy in parameter estimation via narrow confidence intervals. , 2006, Psychological methods.

[6]  B. Schneider,et al.  Standards for Reporting on Empirical Social Science Research in AERA Publications American Educational Research Association , 2006 .

[7]  J. H. Steiger,et al.  Beyond the F test: Effect size confidence intervals and tests of close fit in the analysis of variance and contrast analysis. , 2004, Psychological methods.

[8]  Rex B. Kline,et al.  Beyond Significance Testing: Reforming Data Analysis Methods in Behavioral Research , 2004 .

[9]  B. Thompson.  What Future Quantitative Social Science Research Could Look Like: Confidence Intervals for Effect Sizes , 2002 .

[10]  J. H. Steiger,et al.  The Problem Is Epistemology, Not Statistics: Replace Significance Tests by Confidence Intervals and Quantify Accuracy of Risky Numerical Predictions , 2002 .

[11]  G. Cumming,et al.  A Primer on the Understanding, Use, and Calculation of Confidence Intervals that are Based on Central and Noncentral Distributions , 2001 .

[12]  Michael Smithson,et al.  Correct Confidence Intervals for Various Regression Effect Sizes and Parameters: The Importance of Noncentral Distributions in Computing Intervals , 2001 .

[13]  Richard K. Burdick.  Linear Models in Statistics , 2001, Technometrics.

[14]  James Algina,et al.  Determining Sample Size for Accurate Estimation of the Squared Multiple Correlation Coefficient , 2000, Multivariate behavioral research.

[15]  D. Krantz.  The Null Hypothesis Testing Controversy in Psychology , 1999 .

[16]  Stephen C. Arnold,et al.  Kendall's advanced theory of statistics. Vol.2A: Classical inference and the linear model , 1999 .

[17]  Cherng G. Ding.  On the computation of the distribution of the square of the sample multiple correlation coefficient , 1996 .

[18]  F. Schmidt.  Statistical Significance Testing and Cumulative Knowledge in Psychology: Implications for Training of Researchers , 1996 .

[19]  P. R. Nelson.  Continuous Univariate Distributions Volume 2 , 1996 .

[20]  N. L. Johnson,et al.  Continuous Univariate Distributions. , 1995 .

[21]  R. Berk,et al.  Continuous Univariate Distributions, Volume 2 , 1995 .

[22]  Jacob Cohen.  The earth is round (p < .05) , 1994 .

[23]  A. W. Kemp,et al.  Kendall's Advanced Theory of Statistics. , 1994 .

[24]  Barry B. Powell,et al.  Homer and the origin of the Greek alphabet: Homeric references in poets of the seventh century , 1991 .

[25]  Scott E. Maxwell,et al.  Designing Experiments and Analyzing Data: A Model Comparison Perspective , 1990 .

[26]  C Gatsonis,et al.  Multiple correlation: exact power and sample size calculations. , 1989, Psychological bulletin.

[27]  G. Newman,et al.  CONFIDENCE INTERVALS , 1987, The Lancet.

[28]  J. Patel,et al.  Handbook of the normal distribution , 1983 .

[29]  G. Glass,et al.  Meta-analysis in social research , 1981 .

[30]  Allen I. Fleishman.  Confidence Intervals for Correlation Ratios , 1980 .

[31]  Richard J. Harris.  A primer of multivariate statistics , 1975 .

[32]  A. Sampson.  A Tale of Two Regressions , 1974 .

[33]  Oscar Kempthorne,et al.  Probability, Statistics, and data analysis , 1973 .

[34]  N. L. Johnson,et al.  APPLICATIONS OF THE NON-CENTRAL t-DISTRIBUTION , 1940 .

[35]  J. Neyman.  Outline of a Theory of Statistical Estimation Based on the Classical Theory of Probability , 1937 .

[36]  J. Neyman,et al.  On the Problem of Confidence Intervals , 1935 .

[37]  Rory A. Fisher,et al.  The general sampling distribution of the multiple correlation coefficient , 1928 .