How to quantify support for and against the null hypothesis: A flexible WinBUGS implementation of a default Bayesian t test

We propose a sampling-based Bayesian t test that allows researchers to quantify the statistical evidence in favor of the null hypothesis. This Savage—Dickey (SD) t test is inspired by the Jeffreys—Zellner—Siow (JZS) t test recently proposed by Rouder, Speckman, Sun, Morey, and Iverson (2009). The SD test retains the key concepts of the JZS test but is applicable to a wider range of statistical problems. The SD test allows researchers to test order restrictions and applies to two-sample situations in which the different groups do not share the same variance.

[1]  H. Jeffreys,et al.  Theory of probability , 1896 .

[2]  J. Dickey,et al.  The Weighted Likelihood Ratio, Sharp Hypotheses about Chances, the Order of a Markov Chain , 1970 .

[3]  G. C. Tiao,et al.  Bayesian inference in statistical analysis , 1973 .

[4]  A. Zellner,et al.  Posterior odds ratios for selected regression hypotheses , 1980 .

[5]  L. Hedges.  Distribution Theory for Glass's Estimator of Effect size and Related Estimators , 1981 .

[6]  R. Nosofsky.  Attention, similarity, and the identification-categorization relationship. , 1986, Journal of experimental psychology. General.

[7]  M. Kendall,et al.  Kendall's advanced theory of statistics , 1995 .

[8]  J. Berger,et al.  The application of robust Bayesian analysis to hypothesis testing and Occam's Razor , 1992 .

[9]  J. Kruschke,et al.  ALCOVE: an exemplar-based connectionist model of category learning. , 1992, Psychological review.

[10]  J. Q. Smith,et al.  1. Bayesian Statistics 4 , 1993 .

[11]  Walter R. Gilks,et al.  A Language and Program for Complex Bayesian Modelling , 1994 .

[12]  A. Raftery.  Bayesian Model Selection in Social Research , 1995 .

[13]  L. Wasserman,et al.  Computing Bayes Factors Using a Generalization of the Savage-Dickey Density Ratio , 1995 .

[14]  J. Berger,et al.  The Intrinsic Bayes Factor for Model Selection and Prediction , 1996 .

[15]  Anthony O'Hagan,et al.  Kendall's Advanced Theory of Statistics: Vol. 2B, Bayesian Inference. , 1996 .

[16]  L. Wasserman,et al.  Computing Bayes Factors by Combining Simulation and Asymptotic Approximations , 1997 .

[17]  Young K. Truong,et al.  Polynomial splines and their tensor products in extended linear modeling: 1994 Wald memorial lecture , 1997 .

[18]  I. J. Myung,et al.  Applying Occam’s razor in modeling cognition: A Bayesian approach , 1997 .

[19]  T. Louis,et al.  Bayes and Empirical Bayes Methods for Data Analysis. , 1997 .

[20]  Dani Gamerman,et al.  Markov Chain Monte Carlo: Stochastic Simulation for Bayesian Inference , 1997 .

[21]  Bradley P. Carlin,et al.  BAYES AND EMPIRICAL BAYES METHODS FOR DATA ANALYSIS , 1996, Stat. Comput..

[22]  Allan S. Cohen,et al.  On the Behrens-Fisher Problem: A Review , 1998 .

[23]  E. Moreno,et al.  Default Bayesian analysis of the Behrens-Fisher problem , 1999 .

[24]  David M. Riefer,et al.  Theoretical and empirical review of multinomial process tree modeling , 1999, Psychonomic bulletin & review.

[25]  Leland Wilkinson,et al.  Statistical Methods in Psychology Journals Guidelines and Explanations , 2005 .

[26]  Wasserman,et al.  Bayesian Model Selection and Model Averaging. , 2000, Journal of mathematical psychology.

[27]  Andrew Thomas,et al.  WinBUGS - A Bayesian modelling framework: Concepts, structure, and extensibility , 2000, Stat. Comput..

[28]  J. Busemeyer,et al.  A contribution of cognitive decision models to clinical assessment: decomposing performance on the Bechara gambling task. , 2002, Psychological assessment.

[29]  David J. Spiegelhalter,et al.  Bayesian graphical modelling: a case‐study in monitoring health outcomes , 2002 .

[30]  C. F. Bond,et al.  One Hundred Years of Social Psychology Quantitatively Described , 2003 .

[31]  Jun Lu,et al.  An introduction to Bayesian hierarchical models with an application in the theory of signal detection , 2005, Psychonomic bulletin & review.

[32]  Herbert Hoijtink,et al.  Inequality constrained analysis of variance: a Bayesian approach. , 2005, Psychological methods.

[33]  Andrew Gelman,et al.  Data Analysis Using Regression and Multilevel/Hierarchical Models , 2006 .

[34]  E. Wagenmakers.  A practical solution to the pervasive problems ofp values , 2007, Psychonomic bulletin & review.

[35]  Jun Lu,et al.  Signal Detection Models with Random Participant and Item Effects , 2007 .

[36]  Rhiannon Weaver,et al.  Parameters, Predictions, and Evidence in Computational Modeling: A Statistical View Informed by ACT-R , 2008, Cogn. Sci..

[37]  Michael D. Lee,et al.  A Bayesian approach to diffusion process models of decision-making , 2008 .

[38]  Tom Lodewyckx,et al.  Bayesian Versus Frequentist Inference , 2008 .

[39]  H. Hoijtink,et al.  Bayesian Evaluation of Informative Hypotheses. , 2008 .

[40]  K. McRae,et al.  Proceedings of the 30th Annual Conference of the Cognitive Science Society. , 2008 .

[41]  Jeffrey N. Rouder,et al.  A hierarchical process-dissociation model. , 2008, Journal of experimental psychology. General.

[42]  M. Clyde,et al.  Mixtures of g Priors for Bayesian Variable Selection , 2008 .

[43]  Jie W Weiss,et al.  Bayesian Statistical Inference for Psychological Research , 2008 .

[44]  Michael D. Lee,et al.  A Survey of Model Evaluation Approaches With a Tutorial on Hierarchical Bayesian Methods , 2008, Cogn. Sci..

[45]  M. Lee.  Three case studies in the Bayesian analysis of cognitive models , 2008, Psychonomic bulletin & review.

[46]  J. Osborne.  Best Practices in Quantitative Methods , 2009 .

[47]  Jeffrey N. Rouder,et al.  Bayesian t tests for accepting and rejecting the null hypothesis , 2009, Psychonomic bulletin & review.

[48]  Eric-Jan Wagenmakers,et al.  Methodological and empirical developments for the Ratcliff diffusion model of response times and accuracy , 2009 .

[49]  E. Wagenmakers,et al.  Bayesian parameter estimation in the Expectancy Valence model of the Iowa gambling task , 2010 .