Performing Inferential Statistics Prior to Data Collection

Typically, in education and psychology research, the investigator collects data and subsequently performs descriptive and inferential statistics. For example, a researcher might compute group means and use the null hypothesis significance testing procedure to draw conclusions about the populations from which the groups were drawn. We propose an alternative inferential statistical procedure that is performed prior to data collection rather than afterwards. To use this procedure, the researcher specifies how close she or he desires the group means to be to their corresponding population means and how confident she or he wishes to be that this actually is so. We derive an equation that provides researchers with a way to determine the sample size needed to meet the specifications concerning closeness and confidence, regardless of the number of groups.

[1]  David Trafimow,et al.  A Test of the Null Hypothesis Significance Testing Procedure Correlation Argument , 2009, The Journal of general psychology.

[2]  D. Bakan,et al.  The test of significance in psychological research. , 1966, Psychological bulletin.

[3]  G. Loftus,et al.  Why Figures with Error Bars Should Replace p Values Some Conceptual Arguments and Empirical Demonstrations , 2015 .

[4]  G. Loftus.  Psychology Will Be a Much Better Science When We Change the Way We Analyze Data , 1996 .

[5]  Marek McGann,et al.  How Mean is the Mean? , 2013, Front. Psychol..

[6]  W. Hays.  Statistical theory. , 1968, Annual review of psychology.

[7]  P. Suppes.  Qualitative Theory of Subjective Probability , 1994 .

[8]  Dana Quade,et al.  The Minimally Important Difference Significant Criterion for Sample Size , 1992 .

[9]  Joseph Berkson,et al.  Some Difficulties of Interpretation Encountered in the Application of the Chi-Square Test , 1938 .

[10]  D. Trafimow.  Using the Coefficient of Confidence to Make the Philosophical Switch From A Posteriori to A Priori Inferential Statistics , 2017, Educational and psychological measurement.

[11]  Jacob Cohen.  The earth is round (p < .05) , 1994 .

[12]  W. W. Rozeboom.  The fallacy of the null-hypothesis significance test. , 1960, Psychological bulletin.

[13]  P. Meehl.  Theoretical risks and tabular asterisks: Sir Karl, Sir Ronald, and the slow progress of soft psychology. , 1978 .

[14]  S. Wellek.  Testing Statistical Hypotheses of Equivalence and Noninferiority , 2010 .

[15]  Gerd Gigerenzer,et al.  The superego, the ego, and the id in statistical reasoning , 1993 .

[16]  A. Nowacki,et al.  Understanding Equivalence and Noninferiority Testing , 2011, Journal of General Internal Medicine.

[17]  V. Barnett,et al.  Applied Linear Statistical Models , 1975 .

[18]  D. Trafimow.  Hypothesis testing and theory evaluation at the boundaries: surprising insights from Bayes's theorem. , 2003, Psychological review.

[19]  Jeffrey N. Rouder,et al.  Robust misinterpretation of confidence intervals , 2013, Psychonomic bulletin & review.

[20]  F. Schmidt.  Statistical Significance Testing and Cumulative Knowledge in Psychology: Implications for Training of Researchers , 1996 .

[21]  B. Thompson.  Two and One‐Half Decades of Leadership in Measurement and Evaluation , 1992 .

[22]  Carol Saunders,et al.  PLS: A Silver Bullet? , 2006 .

[23]  George A. Marcoulides,et al.  Assessing Structural Equation Models by Equivalence Testing With Adjusted Fit Indexes , 2016 .

[24]  Wei Liu,et al.  Testing Statistical Hypotheses of Equivalence and Noninferiority, 2nd edn by Stefan Wellek , 2011 .

[25]  P. Meehl.  Theory-Testing in Psychology and Physics: A Methodological Paradox , 1967, Philosophy of Science.

[26]  Jeffrey N. Rouder,et al.  The fallacy of placing confidence in confidence intervals , 2015, Psychonomic bulletin & review.

[27]  Michael H. Kutner.  Applied Linear Statistical Models , 1974 .

[28]  D. Trafimow.  Using Epistemic Ratios to Evaluate Hypotheses: An Imprecision Penalty for Imprecise Hypotheses , 2006, Genetic, social, and general psychology monographs.

[29]  D. Lykken.  Statistical significance in psychological research. , 1968, Psychological bulletin.

[30]  Karl Raimund Sir Popper,et al.  Realism and the aim of science , 1983 .

[31]  Heike Freud,et al.  Error And The Growth Of Experimental Knowledge , 2016 .

[32]  William L. Hays,et al.  Statistics, 5th ed. , 1994 .

[33]  C. Saunders,et al.  Editor's comments: PLS: a silver bullet? , 2006 .