"Statistical," "practical", and "clinical": How many kinds of significance do counselors need to consider?

The present article reviews and distinguishes 3 related but different types of significance: “statistical,” “practical,” and “clinical.” A framework for conceptualizing the many “practical” effect size indices is described. Several effect size indices that counseling researchers can use, or that counselors reading the literature may encounter, are summarized. A way of estimating “corrected” intervention effects is proposed. It is suggested that readers should expect authors to report indices of “practical” or “clinical” significance, or both, within their research reports; and it is noted that indeed some journals now require such reports.

[1]  H. Friedman.  Magnitude of experimental effect and a table for its rapid estimation. , 1968 .

[2]  A. Kazdin.  The meanings and measurement of clinical significance. , 1999, Journal of consulting and clinical psychology.

[3]  Joel R. Levin,et al.  Book Reviews , 1998 .

[4]  George W. Snedecor,et al.  Calculation and interpretation of analysis of variance and covariance , 1935 .

[5]  M. Oakes.  Statistical Inference: A Commentary for the Social and Behavioural Sciences , 1986 .

[6]  Michael Smithson,et al.  Correct Confidence Intervals for Various Regression Effect Sizes and Parameters: The Importance of Noncentral Distributions in Computing Intervals , 2001 .

[7]  B. Thompson.  Research news and Comment: AERA Editorial Policies Regarding Statistical Significance Testing: Three Suggested Reforms , 1996 .

[8]  R. A. Fisher,et al.  Design of Experiments , 1936 .

[9]  Joseph Berkson,et al.  Some Difficulties of Interpretation Encountered in the Application of the Chi-Square Test , 1938 .

[10]  Bruce Thompson,et al.  Journal Editorial Policies Regarding Statistical Significance Tests: Heat Is to Fire as p Is to Importance , 1999 .

[11]  Interpreting Statistical Significance Test Results: A Proposed New "What If" Method. , 1999 .

[12]  Jacob Cohen.  Multiple regression as a general data-analytic system. , 1968 .

[13]  Ian Hacking.  Logic of Statistical Inference , 1965 .

[14]  Bruce Thompson,et al.  If Statistical Significance Tests are Broken/Misused, What Practices Should Supplement or Replace Them? , 1999 .

[15]  Olejnik,et al.  Measures of Effect Size for Comparative Studies: Applications, Interpretations, and Limitations. , 2000, Contemporary educational psychology.

[16]  R. Frick,et al.  The appropriate use of null hypothesis testing. , 1996 .

[17]  Jacob Cohen.  The earth is round (p < .05) , 1994 .

[18]  W. Hays,et al.  Statistics (3rd ed.). , 1982 .

[19]  R. P. Carver.  The Case Against Statistical Significance Testing , 1978 .

[20]  R. Kirk.  Practical Significance: A Concept Whose Time Has Come , 1996 .

[21]  Jacob Cohen.  Statistical Power Analysis for the Behavioral Sciences , 1969, The SAGE Encyclopedia of Research Design.

[22]  David R. Anderson,et al.  Null Hypothesis Testing: Problems, Prevalence, and an Alternative , 2000 .

[23]  N. Jacobson,et al.  Methods for defining and determining the clinical significance of treatment effects: description, application, and alternatives. , 1999, Journal of consulting and clinical psychology.

[24]  T. R. Knapp.  Canonical correlation analysis: A general parametric significance-testing system. , 1978 .

[25]  Nikolaos Gkantidis,et al.  Clinical Significance , 2022, The SAGE Encyclopedia of Research Design.

[26]  Bruce Thompson,et al.  A National Survey of AERA Members' Perceptions of Statistical Significance Tests and Other Statistical Issues. , 2000 .

[27]  Robert Rosenthal,et al.  The Interpretation of Levels of Significance by Psychological Researchers , 1963 .

[28]  G. Cumming,et al.  A Primer on the Understanding, Use, and Calculation of Confidence Intervals that are Based on Central and Noncentral Distributions , 2001 .

[29]  G. Glass.  Primary, Secondary, and Meta-Analysis of Research1 , 1976 .

[30]  S. Glantz.  Biostatistics: how to detect, correct and prevent errors in the medical literature. , 1980, Circulation.

[31]  B. Thompson.  Editorial Policies Regarding Statistical Significance Testing : Three Suggested Reforms , 2012 .

[32]  G. Gigerenzer,et al.  Do studies of statistical power have an effect on the power of studies , 1989 .

[33]  Robert Rosenthal,et al.  Interpretation of significance levels and effect sizes by psychological researchers. , 1986 .

[34]  W. Tryon.  THE INSCRUTABLE NULL HYPOTHESIS , 1998 .

[35]  Edwin G. Boring,et al.  Mathematical vs. scientific significance. , 1919 .

[36]  Carl J. Huberty,et al.  Historical Origins of Statistical Testing Practices: The Treatment of Fisher versus Neyman-Pearson Views in Textbooks. , 1993 .

[37]  Robert Rosenthal,et al.  Contemporary Issues in the Analysis of Data: A Survey of 551 Psychologists , 1993 .

[38]  F. Schmidt.  Statistical Significance Testing and Cumulative Knowledge in Psychology: Implications for Training of Researchers , 1996 .

[39]  B. Thompson,et al.  Reporting Practices and APA Editorial Policies Regarding Statistical Significance and Effect Size , 2000 .

[40]  The inscrutable null hypothesis. , 1998 .

[41]  R. Rosenthal,et al.  Statistical Procedures and the Justification of Knowledge in Psychological Science , 1989 .

[42]  Leland Wilkinson,et al.  Statistical Methods in Psychology Journals Guidelines and Explanations , 2005 .

[43]  R. Holloway,et al.  To test or not to test? , 1999, Neurology.

[44]  T. L. Kelley,et al.  An Unbiased Correlation Ratio Measure. , 1935, Proceedings of the National Academy of Sciences of the United States of America.

[45]  C. Fornell,et al.  Canonical Correlation Analysis As A Special Case Of A Structural Relations Model. , 1981, Multivariate behavioral research.

[46]  Chance and Nonsense: A Conversation about Interpreting Tests of Statistical Significance, Part 1. , 1985 .

[47]  Bruce Thompson,et al.  The Use of Statistical Significance Tests in Research: Bootstrap and Other Alternatives , 1993 .

[48]  P. Lachenbruch.  Statistical Power Analysis for the Behavioral Sciences (2nd ed.) , 1989 .

[49]  W. Dunlap,et al.  On the Logic and Purpose of Significance Testing , 1997 .

[50]  Gary R. Howard.  Book Reviews: Reflections on the “White Movement” in Multicultural Education , 2000 .

[51]  Patricia Snyder,et al.  Evaluating Results Using Corrected and Uncorrected Effect Size Estimates , 1993 .

[52]  Bruce Thompson,et al.  Book Reviews , 1998 .