WHY ENCOURAGING EFFECT SIZE REPORTING IS NOT WORKING : THE ETIOLOGY OF RESEARCHER RESISTANCE TO CHANGING PRACTICES

Abstract Given decades of lucid, blunt admonitions that statistical significance tests are often misused and that the tests are somewhat limited in their usefulness, what is needed is less repeated bashing of statistical tests and some honest reflection regarding the etiology of researchers' denial and psychological resistance (sometimes unconscious) to an improved practice. Three etiologies are briefly explored here: (a) atavism, (b) “is/ought” logic fallacies, and (c) confusion or desperation. Understanding the etiology of psychological resistance may ultimately lead to improved interventions to assist in overcoming researcher resistance to reporting effect sizes and using non-nil nulls and other analytic improvements.

[1]  Bruce Thompson,et al.  The Use of Statistical Significance Tests in Research: Bootstrap and Other Alternatives , 1993 .

[2]  Patricia Snyder,et al.  Evaluating Results Using Corrected and Uncorrected Effect Size Estimates , 1993 .

[3]  Comment on Significance Testing. , 1998 .

[4]  J. Hunter.  Needed: A Ban on the Significance Test , 1997 .

[5]  B. Thompson.  Editorial Policies Regarding Statistical Significance Testing : Three Suggested Reforms , 2012 .

[6]  F. Schmidt.  Statistical Significance Testing and Cumulative Knowledge in Psychology: Implications for Training of Researchers , 1996 .

[7]  Jacob Cohen.  The earth is round (p < .05) , 1994 .

[8]  M. Mahoney,et al.  Scientist as Subject: The Psychological Imperative , 1979 .

[9]  L. Harlow,et al.  What if there were no significance tests , 1997 .

[10]  Bruce Thompson,et al.  If Statistical Significance Tests are Broken/Misused, What Practices Should Supplement or Replace Them? , 1999 .

[11]  W. D. Hudson.  The Is-Ought Question , 1969 .

[12]  Robert Rosenthal,et al.  Effect sizes: Pearson's correlation, its display via the BESD, and alternative indices. , 1991 .

[13]  R. Frick,et al.  The appropriate use of null hypothesis testing. , 1996 .

[14]  Patricia Snyder,et al.  Use of Tests of Statistical Significance and Other Analytic Choices in a School Psychology Journal: Review of Practices and Suggested Alternatives. , 1998 .

[15]  Ramon E. Henkel.  Tests of Significance , 1976 .

[16]  R. Kirk.  Practical Significance: A Concept Whose Time Has Come , 1996 .

[17]  Bruce Thompson,et al.  IN PRAISE OF BRILLIANCE : WHERE THAT PRAISE REALLY BELONGS , 1998 .

[18]  Joel Kupfersmid,et al.  Improving what is published: A model in search of an editor. , 1988 .

[19]  K. Strike.  An Epistemology of Practical Research1 , 1979 .

[20]  R. Falk,et al.  Significance Tests Die Hard , 1995 .

[21]  John P. Campbell,et al.  Editorial: Some Remarks From the Outgoing Editor. , 1982 .

[22]  Chance and Nonsense: A Conversation about Interpreting Tests of Statistical Significance, Part 1. , 1985 .

[23]  Monica J. Harris.  Significance Tests are Not Enough , 1991 .

[24]  Patricia Snyder,et al.  Statistical Significance Testing Practices in The Journal of Experimental Education , 1997 .

[25]  Jacob Cohen,et al.  THINGS I HAVE LEARNED (SO FAR) , 1990 .

[26]  Bruce Thompson,et al.  Book Reviews , 1998 .

[27]  R. Rosenthal,et al.  Statistical Procedures and the Justification of Knowledge in Psychological Science , 1989 .

[28]  Jum C. Nunnally,et al.  The Place of Statistics in Psychology , 1960 .

[29]  W. W. Rozeboom.  The fallacy of the null-hypothesis significance test. , 1960, Psychological bulletin.

[30]  Patricia Snyder,et al.  Statistical Significance and Reliability Analyses in Recent Journal of Counseling & Development Research Articles , 1998 .

[31]  Bruce Thompson,et al.  Rejoinder: Editorial Policies Regarding Statistical Significance Tests: Further Comments , 1997 .

[32]  Johanna E. Nilsson,et al.  Statistical Significance Reporting: Current Trends and Uses in MECD. , 1998 .

[33]  R. L. Hagen.  In praise of the null hypothesis statistical test. , 1997 .

[34]  R. P. Carver.  The Case Against Statistical Significance Testing , 1978 .

[35]  B. Thompson.  Research news and Comment: AERA Editorial Policies Regarding Statistical Significance Testing: Three Suggested Reforms , 1996 .