Bayesian Assessment of Null Values Via Parameter Estimation and Model Comparison.

Psychologists have been trained to do data analysis by asking whether null values can be rejected. Is the difference between groups nonzero? Is choice accuracy not at chance level? These questions have been traditionally addressed by null hypothesis significance testing (NHST). NHST has deep problems that are solved by Bayesian data analysis. As psychologists transition to Bayesian data analysis, it is natural to ask how Bayesian analysis assesses null values. The article explains and evaluates two different Bayesian approaches. One method involves Bayesian model comparison (and uses Bayes factors). The second method involves Bayesian parameter estimation and assesses whether the null value falls among the most credible values. Which method to use depends on the specific question that the analyst wants to answer, but typically the estimation approach (not using Bayes factors) provides richer information than the model comparison approach.

[1]  W. Johnson,et al.  Must psychologists change the way they analyze their data? , 2011, Journal of personality and social psychology.

[2]  Jeffrey N. Rouder,et al.  A Bayes factor meta-analysis of Bem’s ESP claim , 2011, Psychonomic bulletin & review.

[3]  Z. Dienes Bayesian Versus Orthodox Statistics: Which Side Are You On? , 2011, Perspectives on psychological science : a journal of the Association for Psychological Science.

[4]  S. Donovan The Sign of Four , 2011 .

[5]  E. Wagenmakers,et al.  Why psychologists must change the way they analyze their data: the case of psi: comment on Bem (2011). , 2011, Journal of personality and social psychology.

[6]  D. Bem Feeling the future: experimental evidence for anomalous retroactive influences on cognition and affect. , 2011, Journal of personality and social psychology.

[7]  W. Vanpaemel,et al.  Prior sensitivity in theory testing: An apologia for the Bayes factor , 2010 .

[8]  J. Kruschke Doing Bayesian Data Analysis: A Tutorial with R and BUGS , 2010 .

[9]  John K Kruschke,et al.  Bayesian data analysis. , 2010, Wiley interdisciplinary reviews. Cognitive science.

[10]  Bradley P. Carlin,et al.  Bayesian Adaptive Methods for Clinical Trials , 2010 .

[11]  Ronald Christensen,et al.  Bayesian Ideas and Data Analysis: An Introduction for Scientists and Statisticians , 2010 .

[12]  J. Kruschke What to believe: Bayesian methods for data analysis , 2010, Trends in Cognitive Sciences.

[13]  Robert E Weiss,et al.  Bayesian methods for data analysis. , 2010, American journal of ophthalmology.

[14]  Simon Jackman,et al.  Bayesian Analysis for the Social Sciences , 2009 .

[15]  J. Raaijmakers,et al.  How to quantify support for and against the null hypothesis: A flexible WinBUGS implementation of a default Bayesian t test , 2009, Psychonomic bulletin & review.

[16]  Andrew Gelman,et al.  Why We (Usually) Don't Have to Worry About Multiple Comparisons , 2009, 0907.2478.

[17]  Jeffrey N. Rouder,et al.  Bayesian t tests for accepting and rejecting the null hypothesis , 2009, Psychonomic bulletin & review.

[18]  C. Robert,et al.  Bayesian Modeling Using WinBUGS , 2009 .

[19]  M. Aitkin,et al.  Bayes factors: Prior sensitivity and model generalizability , 2008 .

[20]  Z. Diénès Understanding Psychology as a Science: An Introduction to Scientific and Statistical Inference , 2008 .

[21]  E. Wagenmakers A practical solution to the pervasive problems ofp values , 2007, Psychonomic bulletin & review.

[22]  K. Koch Introduction to Bayesian Statistics , 2007 .

[23]  R. A. Bailey Designing Experiments and Analyzing Data: a Model Comparison Perspective, 2nd edn , 2005 .

[24]  M. Lee,et al.  Bayesian statistical inference in psychology: comment on Trafimow (2003). , 2005, Psychological review.

[25]  Jeffrey N. Rouder,et al.  A hierarchical model for estimating response time distributions , 2005, Psychonomic bulletin & review.

[26]  G. Loftus Psychology Will Be a Much Better Science When We Change the Way We Analyze Data , 1996 .

[27]  L. Wasserman,et al.  The Selection of Prior Distributions by Formal Rules , 1996 .

[28]  Scott E. Maxwell,et al.  Designing Experiments and Analyzing Data: A Model Comparison Perspective , 1990 .

[29]  James O. Berger,et al.  Statistical Analysis and the Illusion of Objectivity , 1988 .

[30]  J. Cornfield A BAYESIAN TEST OF SOME CLASSICAL HYPOTHESES- WITH APPLICATIONS TO SEQUENTIAL CLINICAL TRIALS , 1966 .

[31]  F. J. Anscombe,et al.  Fixed-Sample-Size Analysis of Sequential Observations , 1954 .

[32]  Mu Zhu The Counter-intuitive Non-informative Prior for the Bernoulli Family , 2004 .

[33]  David Lindley,et al.  The future of statistics-A Bayesian 21st Century , 1974 .

[34]  L. M. M.-T. Theory of Probability , 1929, Nature.