Confidence Intervals for Effect Sizes

Confidence intervals for reliability coefficients can be estimated in various ways. The present article illustrates a variety of these applications. This guidelines editorial also promulgates a request that EPM authors report confidence intervals for reliability estimates whenever they report score reliabilities and note what interval estimation methods they have used. This will reinforce reader understanding that all statistical estimates, including those for score reliability, are affected by sampling error variance. And these requirements may also facilitate understanding that tests are not impregnated with invariant reliability as a routine part of printing.

[1]  James H. Steiger,et al.  R2: A computer program for interval estimation, power Calculations, sample size estimation, and hypothesis testing in multiple regression , 1992 .

[2]  Alan E. Kazdin,et al.  Graduate Training in Statistics, Methodology, and Measurement in Psychology: A Survey of PhD Programs in North America , 1990 .

[3]  Brigitte N. Frederick.  Fixed-, Random-, and Mixed-Effects ANOVA Models: A User-Friendly Guide for Increasing the Generalizability of ANOVA Results. , 1999 .

[4]  G. Glass,et al.  Statistical methods in education and psychology, 3rd ed. , 1996 .

[5]  Leonard S. Feldt,et al.  A test of the hypothesis that Cronbach's alpha reliability coefficient is the same for two tests administered to the same sample , 1980 .

[6]  Johanna E. Nilsson,et al.  Practices Regarding Reporting of Reliability Coefficients: A Review of Three Journals , 1999 .

[7]  Noreen M. Webb,et al.  Using Generalizability Theory in Counseling and Development. , 1988 .

[8]  Bruce Thompson,et al.  Advances in Social Science Methodology , 1994 .

[9]  L. Harlow,et al.  What if there were no significance tests , 1997 .

[10]  B. Thompson,et al.  Factor Analytic Evidence for the Construct Validity of Scores: A Historical Overview and Some Guidelines , 1996 .

[11]  Richard J. Shavelson,et al.  Generalizability Theory: A Primer , 1991 .

[12]  Leonard S. Feldt.  Statistical Tests and Confidence Intervals for Cronbach's Coefficient Alpha. Iowa Testing Programs Occasional Papers Number 33. , 1986 .

[13]  Dale Whhtington,et al.  How Well Do Researchers Report their Measures? an Evaluation of Measurementin Published Educational Research , 1998 .

[14]  L. S. Feldt.  The Sampling Theory for the Intraclass Reliability Coefficient , 1990 .

[15]  Kathleen G. Purdy.  Confidence intervals for variance components , 1998 .

[16]  G. Cumming,et al.  A Primer on the Understanding, Use, and Calculation of Confidence Intervals that are Based on Central and Noncentral Distributions , 2001 .

[17]  Leland Wilkinson,et al.  Statistical Methods in Psychology Journals Guidelines and Explanations , 2005 .

[18]  Elazar J. Pedhazur,et al.  Measurement, Design, and Analysis: An Integrated Approach , 1994 .

[19]  B. Thompson.  Advances in educational research : substantive findings, methodological developments , 1991 .

[20]  T. Vacha-Haase,et al.  Reliability Generalization: Exploring Variance in Measurement Error Affecting Score Reliability Across Studies , 1998 .

[21]  Bruce Thompson,et al.  Computing Correct Confidence Intervals for Anova Fixed-and Random-Effects Effect Sizes , 2001 .

[22]  G. Glass,et al.  Statistical methods in education and psychology , 1970 .

[23]  David J. Woodruff,et al.  Statistical Inference for Coefficient Alpha , 1987 .

[24]  L. S. Feldt.  The approximate sampling distribution of Kuder-Richardson reliability coefficient twenty , 1965, Psychometrika.

[25]  B. Thompson,et al.  Sample Compositions and Variabilities in Published Studies versus Those in Test Manuals: Validity of Score Reliability Inductions , 2000 .

[26]  Leonard S. Feldt,et al.  Testing the Equality of Two Alpha Coefficients , 1996 .

[27]  W. Hays.  Statistics, 4th ed. , 1988 .

[28]  F. Graybill,et al.  Confidence Intervals on Variance Components. , 1993 .

[29]  L. Crocker,et al.  Introduction to Classical and Modern Test Theory , 1986 .

[30]  Bruce Thompson,et al.  Stepwise Regression and Stepwise Discriminant Analysis Need Not Apply here: A Guidelines Editorial , 1995 .

[31]  Walter Kristof.  The statistical theory of stepped-up reliability coefficients when a test has been divided into several equivalent parts , 1963 .

[32]  R. L. Winkler,et al.  Statistics : Probability, Inference and Decision , 1975 .

[33]  Jacob Cohen.  The earth is round (p < .05) , 1994 .

[34]  C. Hoyt.  Test reliability estimated by analysis of variance , 1941 .

[35]  Michael Smithson,et al.  Correct Confidence Intervals for Various Regression Effect Sizes and Parameters: The Importance of Noncentral Distributions in Computing Intervals , 2001 .

[36]  R. Kirk.  Practical Significance: A Concept Whose Time Has Come , 1996 .

[37]  R. Schiffer,et al.  INTRODUCTION , 1988, Neurology.

[38]  Sandra H. Eason,et al.  Why Generalizability Theory Yields Better Results than Classical Test Theory. , 1989 .

[39]  Bruce Thompson,et al.  Psychometrics is Datametrics: the Test is not Reliable , 2000 .