My Ban on Null Hypothesis Significance Testing and Confidence Intervals

The journal, Basic and Applied Social Psychology, banned null hypothesis significance testing and confidence intervals. Was this justified, and if so, why? I address these questions with a focus on the different types of assumptions that compose the models on which p-values and confidence intervals are based. For the computation of p-values, in addition to problematic model assumptions, there also is the problem that p-values confound the implications of sample effect sizes and sample sizes. For the computation of confidence intervals, in contrast to the justification that they provide valuable information about the precision of the data, there is a triple confound involving three types of precision. These are measurement precision, precision of homogeneity, and sampling precision. Because it is possible to estimate all three separately, provided the researcher has tested the reliability of the dependent variable, there is no reason to confound them via the computation of a confidence interval. Thus, the ban is justified both with respect to null hypothesis significance testing and confidence intervals.

[1]  D. Trafimow.  Using the Coefficient of Confidence to Make the Philosophical Switch From A Posteriori to A Priori Inferential Statistics , 2017, Educational and psychological measurement.

[2]  Jerome R. Busemeyer,et al.  A Quantum Probability Model of Causal Reasoning , 2012, Front. Psychology.

[3]  David Trafimow,et al.  A taxonomy of model assumptions on which P is based and implications for added benefit in the sciences , 2019, International Journal of Social Research Methodology.

[4]  M. R. Novick,et al.  Statistical Theories of Mental Test Scores. , 1971 .

[5]  Jennifer Trueblood,et al.  A Quantum Probability Account of Order Effects in Inference , 2011, Cogn. Sci..

[6]  Arch G. Woodside,et al.  The good practices manifesto: Overcoming bad practices pervasive in current research in business , 2016 .

[7]  D. Gillies.  Philosophical Theories of Probability , 2000 .

[8]  N. Lazar,et al.  The ASA Statement on p-Values: Context, Process, and Purpose , 2016 .

[9]  D. Curran‐Everett,et al.  The fickle P value generates irreproducible results , 2015, Nature Methods.

[10]  Leif D. Nelson,et al.  False-Positive Psychology , 2011, Psychological science.

[11]  D. Trafimow,et al.  Replication, falsification, and the crisis of confidence in social psychology , 2015, Front. Psychol..

[12]  G. Loewenstein,et al.  Measuring the Prevalence of Questionable Research Practices With Incentives for Truth Telling , 2012, Psychological science.

[13]  D. Trafimow.  Implications of an initial empirical victory for the truth of the theory and additional empirical victories , 2017 .

[14]  Hung T. Nguyen,et al.  On Evidential Measures of Support for Reasoning with Integrated Uncertainty: A Lesson from the Ban of P-values in Statistical Inference , 2016, IUKM.

[15]  Ariel M. Aloe,et al.  Life After NHST: How to Describe Your Data Without “p-ing” Everywhere , 2015 .

[16]  Geoff Cumming,et al.  Introduction to the New Statistics: Estimation, Open Science, and Beyond , 2016 .

[17]  Hung T. Nguyen,et al.  Manipulating the Alpha Level Cannot Cure Significance Testing , 2017, Front. Psychol..

[18]  George E. P. Box,et al.  Empirical Model‐Building and Response Surfaces , 1988 .

[19]  Richard A. Berk,et al.  Statistical Assumptions as Empirical Commitments , 2001 .

[20]  H. Gulliksen.  Theory of mental tests , 1952 .

[21]  D. Mccloskey,et al.  The Cult of Statistical Significance: How the Standard Error Costs Us Jobs, Justice, and Lives , 2008 .

[22]  Imre Lakatos,et al.  The Methodology of Scientific Research Programmes , 1978 .

[23]  Michael C. Frank,et al.  Estimating the reproducibility of psychological science , 2015, Science.

[24]  D. Trafimow.  An a priori solution to the replication crisis , 2018, Philosophical Psychology.

[25]  J. Gallacher,et al.  Why representativeness should be avoided. , 2013, International journal of epidemiology.

[26]  J. Wicherts,et al.  The Rules of the Game Called Psychological Science , 2012, Perspectives on psychological science : a journal of the Association for Psychological Science.

[27]  Sander Greenland,et al.  Invited Commentary: The Need for Cognitive Science in Methodology , 2017, American journal of epidemiology.

[28]  J. Becker,et al.  The Aim and Structure of Physical Theory , 1955 .

[29]  D. Trafimow,et al.  From a Sampling Precision Perspective, Skewness Is a Friend and Not an Enemy! , 2019, Educational and psychological measurement.

[30]  Raymond Hubbard,et al.  Corrupt Research: The Case for Reconceptualizing Empirical Management and SocialScience , 2016 .

[31]  Means and standard deviations, or locations and scales? That is the question! , 2018, New Ideas in Psychology.

[32]  David Trafimow,et al.  Performing Inferential Statistics Prior to Data Collection , 2017, Educational and psychological measurement.