Bayesian t tests for accepting and rejecting the null hypothesis

Progress in science often comes from discovering invariances in relationships among variables; these invariances often correspond to null hypotheses. As is commonly known, it is not possible to state evidence for the null hypothesis in conventional significance testing. Here we highlight a Bayes factor alternative to the conventional t test that will allow researchers to express preference for either the null hypothesis or the alternative. The Bayes factor has a natural and straightforward interpretation, is based on reasonable assumptions, and has better properties than other methods of inference that have been advocated in the psychological literature. To facilitate use of the Bayes factor, we provide an easy-to-use, Web-based program that performs the necessary calculations.

[1]  H. Jeffreys,et al.  Theory of probability , 1896 .

[2]  R. Shepard.  Stimulus and response generalization: A stochastic model relating generalization to distance in psychological space , 1957 .

[3]  D. Lindley.  A STATISTICAL PARADOX , 1957 .

[4]  Frank R. Clarke,et al.  Constant‐Ratio Rule for Confusion Matrices in Speech Communication , 1957 .

[5]  R. Shepard.  Stimulus and response generalization: tests of a model relating generalization to distance in psychological space. , 1958, Journal of experimental psychology.

[6]  R. Luce,et al.  Individual Choice Behavior: A Theoretical Analysis. , 1960 .

[7]  Gustav Theodor Fechner,et al.  Elements of psychophysics , 1966 .

[8]  D. M. Green,et al.  Signal detection theory and psychophysics , 1966 .

[9]  H. Akaike.  A new look at the statistical model identification , 1974 .

[10]  Stephen E. Fienberg,et al.  Discrete Multivariate Analysis: Theory and Practice , 1976 .

[11]  On the Psychophysical Law: An Information Theory Interpretation , 1975, Perceptual and motor skills.

[12]  James P. Egan,et al.  Signal detection theory and ROC analysis , 1975 .

[13]  John W. Tukey,et al.  Exploratory Data Analysis. , 1979 .

[14]  G. Schwarz.  Estimating the Dimension of a Model , 1978 .

[15]  R. Duncan Luce,et al.  Individual Choice Behavior: A Theoretical Analysis , 1979 .

[16]  A. Zellner,et al.  Posterior odds ratios for selected regression hypotheses , 1980 .

[17]  G. Logan.  Toward an instance theory of automatization. , 1988 .

[18]  E. Reingold,et al.  Using direct and indirect measures to study perception without awareness , 1988, Perception & psychophysics.

[19]  L. Jacoby.  A process dissociation framework: Separating automatic from intentional uses of memory , 1991 .

[20]  G. Logan.  Shapes of reaction-time distributions and shapes of learning curves: a test of the instance theory of automaticity. , 1992, Journal of experimental psychology. Learning, memory, and cognition.

[21]  Pei Wang.  Comparing Categorization Models | A psychological experiment , 1993 .

[22]  E. Lehmann.  The Fisher, Neyman-Pearson Theories of Testing Hypotheses: One Theory or Two? , 1993 .

[23]  Jacob Cohen.  The earth is round (p < .05) , 1994 .

[24]  L L Jacoby,et al.  Unconscious perception: attention, awareness, and control. , 1994, Journal of experimental psychology. Learning, memory, and cognition.

[25]  William L. Hays,et al.  Statistics, 5th ed. , 1994 .

[26]  A. Raftery.  Bayesian Model Selection in Social Research , 1995 .

[27]  L. Wasserman,et al.  A Reference Bayesian Test for Nested Hypotheses and its Relationship to the Schwarz Criterion , 1995 .

[28]  David B. Dunson,et al.  Bayesian Data Analysis , 2010 .

[29]  John A. Swets,et al.  Signal Detection Theory and ROC Analysis in Psychology and Diagnostics: Collected Papers , 1996 .

[30]  L. Wasserman,et al.  Computing Bayes Factors by Combining Simulation and Asymptotic Approximations , 1997 .

[31]  I. J. Myung,et al.  Applying Occam’s razor in modeling cognition: A Bayesian approach , 1997 .

[32]  Pierre-Simon Laplace, 1749-1827 : a life in exact science , 1997 .

[33]  S. Dehaene,et al.  Imaging unconscious semantic priming , 1998, Nature.

[34]  S. Goodman.  Toward Evidence-Based Medical Statistics. 1: The P Value Fallacy , 1999, Annals of Internal Medicine.

[35]  M. J. Bayarri,et al.  Calibration of ρ Values for Testing Precise Null Hypotheses , 2001 .

[36]  G. Cumming,et al.  A Primer on the Understanding, Use, and Calculation of Confidence Intervals that are Based on Central and Noncentral Distributions , 2001 .

[37]  Stephen W. Hawking,et al.  On the shoulders of giants : the great works of physics and astronomy , 2002 .

[38]  M. Masson.  Using confidence intervals for graphically based data interpretation. , 2003, Canadian journal of experimental psychology = Revue canadienne de psychologie experimentale.

[39]  P. Meehl.  Theoretical Risks and Tabular Asterisks: Sir Karl, Sir Ronald, and the Slow Progress of Soft Psychology , 2004 .

[40]  Rex B. Kline,et al.  Beyond Significance Testing: Reforming Data Analysis Methods in Behavioral Research , 2004 .

[41]  Roger Ratcliff,et al.  Comparing categorization models. , 2004, Journal of experimental psychology. General.

[42]  A. Brix.  Bayesian Data Analysis, 2nd edn , 2005 .

[43]  Jun Lu,et al.  An introduction to Bayesian hierarchical models with an application in the theory of signal detection , 2005, Psychonomic bulletin & review.

[44]  P. Killeen,et al.  An Alternative to Null-Hypothesis Significance Tests , 2005, Psychological science.

[45]  M. Lee,et al.  Bayesian statistical inference in psychology: comment on Trafimow (2003). , 2005, Psychological review.

[46]  Jeffrey N Rouder,et al.  Relational and Arelational Confidence Intervals , 2005, Psychological science.

[47]  J. Hyde,et al.  The Gender Similarities Hypothesis , 2005 .

[48]  W. Johnson,et al.  The Bayesian Two-Sample t Test , 2005 .

[49]  E. Plant,et al.  The Consequences of Race for Police Officers' Responses to Criminal Suspects , 2005, Psychological science.

[50]  E. Wagenmakers,et al.  A Bayesian Perspective on Hypothesis Testing , 2006, Psychological science.

[51]  Peter R. Kileen.  Beyond statistical inference: A decision theory for science , 2006 .

[52]  E. Wagenmakers.  A practical solution to the pervasive problems ofp values , 2007, Psychonomic bulletin & review.

[53]  Dongchu Sun,et al.  Objective priors for hypothesis testing in one‐way random effects models , 2007 .

[54]  Jeffrey N Rouder,et al.  Detecting chance: A solution to the null sensitivity problem in subliminal priming , 2007, Psychonomic bulletin & review.

[55]  Janet Shibley Hyde,et al.  New Directions in the Study of Gender Similarities and Differences , 2007 .

[56]  K. Malmberg,et al.  Discriminating between changes in bias and changes in accuracy for recognition memory of emotional stimuli , 2008, Memory & cognition.

[57]  F. Gregory Ashby,et al.  Complex decision rules in categorization : contrasting novice and experienced performance , 1992 .

[58]  M. Clyde,et al.  Mixtures of g Priors for Bayesian Variable Selection , 2008 .

[59]  Jie W Weiss,et al.  Bayesian Statistical Inference for Psychological Research , 2008 .

[60]  Thomas Augustin.  Stevens' power law and the problem of meaningfulness. , 2008, Acta psychologica.

[61]  M. Coory,et al.  Bayesian versus frequentist statistical inference for investigating a one-off cancer cluster reported to a health department , 2009, BMC medical research methodology.