An Introduction to Bayesian Hypothesis Testing for Management Research

In management research, empirical data are often analyzed using p-value null hypothesis significance testing (pNHST). Here we outline the conceptual and practical advantages of an alternative analysis method: Bayesian hypothesis testing and model selection using the Bayes factor. In contrast to pNHST, Bayes factors allow researchers to quantify evidence in favor of the null hypothesis. Also, Bayes factors do not require adjustment for the intention with which the data were collected. The use of Bayes factors is demonstrated through an extended example for hierarchical regression based on the design of an experiment recently published in the Journal of Management. This example also highlights the fact that p values overestimate the evidence against the null hypothesis, misleading researchers into believing that their findings are more reliable than is warranted by the data.

[1]  Adrian E. Raftery,et al.  Bayesian Model Averaging: A Tutorial , 2016 .

[2]  Michèle B. Nuijten,et al.  A default Bayesian hypothesis test for mediation , 2015, Behavior research methods.

[3]  S. Lilienfeld,et al.  The encyclopedia of clinical psychology , 2015 .

[4]  A. D. de Groot,et al.  The meaning of “significance” for different types of research [translated and annotated by Eric-Jan Wagenmakers, Denny Borsboom, Josine Verhagen, Rogier Kievit, Marjan Bakker, Angelique Cramer, Dora Matzke, Don Mellenbergh, and Han L. J. van der Maas] , 2014 .

[5]  J. Rouder Optional stopping: No problem for Bayesians , 2014, Psychonomic bulletin & review.

[6]  Jeffrey N. Rouder,et al.  Why Hypothesis Tests Are Essential for Psychological Science , 2014, Psychological science.

[7]  Regina Nuzzo,et al.  Scientific method: Statistical errors , 2014, Nature.

[8]  Jeffrey N. Rouder,et al.  Robust misinterpretation of confidence intervals , 2013, Psychonomic bulletin & review.

[9]  R. Kievit,et al.  The meaning of "significance" for different types of research [translated and annotated by Eric-Jan Wagenmakers, Denny Borsboom, Josine Verhagen, Rogier Kievit, Marjan Bakker, Angelique Cramer, Dora Matzke, Don Mellenbergh, and Han L. J. van der Maas]. 1969. , 2014, Acta psychologica.

[10]  V. Johnson Revised standards for statistical evidence , 2013, Proceedings of the National Academy of Sciences.

[11]  M. Zyphur BAYESIAN ESTIMATION AND INFERENCE: A USER’S GUIDE , 2013 .

[12]  G. Cumming The New Statistics: Why and How , 2013 .

[13]  J. Rouder,et al.  Default Bayes Factors for Model Selection in Regression , 2012, Multivariate behavioral research.

[14]  H. Beek F1000Prime recommendation of False-positive psychology: undisclosed flexibility in data collection and analysis allows presenting anything as significant. , 2012 .

[15]  David J. Lunn,et al.  The BUGS Book: A Practical Introduction to Bayesian Analysis , 2013 .

[16]  Jeffrey N. Rouder,et al.  Default Bayes factors for ANOVA designs , 2012 .

[17]  Sik-Yum Lee,et al.  A tutorial on the Bayesian approach for analyzing structural equation models , 2012 .

[18]  M. J. Bayarri,et al.  Criteria for Bayesian model choice with application to variable selection , 2012, 1209.5240.

[19]  E. Wagenmakers,et al.  A Default Bayesian Hypothesis Test for ANOVA Designs , 2012 .

[20]  B. Efron A 250-year argument: Belief, behavior, and the bootstrap , 2012 .

[21]  Daniel G. Bachrach,et al.  Role Expectations as Antecedents of Citizenship and the Moderating Effects of Work Context , 2012 .

[22]  Herbert Hoijtink,et al.  BIEMS: A Fortran 90 Program for Calculating Bayes Factors for Inequality and Equality Constrained Models , 2012 .

[23]  Mark P. Sharfman,et al.  Legitimacy, Visibility, and the Antecedents of Corporate Social Performance: An Investigation of the Instrumental Perspective , 2011 .

[24]  H. Hoijtink Informative Hypotheses: Theory and Practice for Behavioral and Social Scientists , 2011 .

[25]  Jeffrey N. Rouder,et al.  Bayes factor approaches for testing interval null hypotheses. , 2011, Psychological methods.

[26]  Michael E J Masson,et al.  A tutorial on a practical Bayesian alternative to null-hypothesis significance testing , 2011, Behavior research methods.

[27]  W. Vanpaemel,et al.  Prior sensitivity in theory testing: An apologia for the Bayes factor , 2010 .

[28]  Jonathan J. Forster,et al.  Default Bayesian model determination methods for generalised linear mixed models , 2010, Comput. Stat. Data Anal..

[29]  E. Wagenmakers,et al.  Bayesian hypothesis testing for psychologists: A tutorial on the Savage–Dickey method , 2010, Cognitive Psychology.

[30]  Andrew Gelman,et al.  Bayesian Statistics Then and Now , 2010, 1012.1423.

[31]  C. D. De Dreu,et al.  Self-concept clarity and the management of social conflict. , 2010, Journal of personality.

[32]  Christopher Rao,et al.  Graphs in Statistical Analysis , 2010 .

[33]  Joseph Hilbe,et al.  Data Analysis Using Regression and Multilevel/Hierarchical Models , 2009 .

[34]  Jeffrey N. Rouder,et al.  Bayesian t tests for accepting and rejecting the null hypothesis , 2009, Psychonomic bulletin & review.

[35]  C. Gallistel,et al.  The Importance of Proving the Null , 2022 .

[36]  M. Aitkin,et al.  Bayes factors: Prior sensitivity and model generalizability , 2008 .

[37]  Jie W Weiss,et al.  Bayesian Statistical Inference for Psychological Research , 2008 .

[38]  M. Clyde,et al.  Mixtures of g Priors for Bayesian Variable Selection , 2008 .

[39]  Z. Diénès Understanding Psychology as a Science: An Introduction to Scientific and Statistical Inference , 2008 .

[40]  S. E. Ahmed,et al.  Markov Chain Monte Carlo: Stochastic Simulation for Bayesian Inference , 2008, Technometrics.

[41]  E. Wagenmakers A practical solution to the pervasive problems ofp values , 2007, Psychonomic bulletin & review.

[42]  Sik-Yum Lee,et al.  Structural equation modelling: A Bayesian approach. , 2007 .

[43]  Dale J. Poirier,et al.  The Growth of Bayesian Methods in Statistics and Economics Since 1970 , 2006 .

[44]  Tony O’Hagan Bayes factors , 2006 .

[45]  M. Hernán,et al.  Intake of wine, beer, and spirits and the risk of clinical common cold. , 2002, American journal of epidemiology.

[46]  Julian J. Faraway,et al.  Practical Regression and Anova using R , 2002 .

[47]  B. Beersma,et al.  A theory-based measure of conflict management strategies in the workplace , 2001 .

[48]  M. J. Bayarri,et al.  Calibration of ρ Values for Testing Precise Null Hypotheses , 2001 .

[49]  I. J. Myung,et al.  GUEST EDITORS' INTRODUCTION: Special Issue on Model Selection , 2000 .

[50]  Adrian E. Raftery,et al.  Bayesian model averaging: a tutorial (with comments by M. Clyde, David Draper and E. I. George, and a rejoinder by the authors , 1999 .

[51]  D. Pauler The Schwarz criterion and related methods for normal linear models , 1998 .

[52]  L. Wasserman,et al.  Computing Bayes Factors by Combining Simulation and Asymptotic Approximations , 1997 .

[53]  A. Raftery,et al.  Estimating Bayes Factors via Posterior Simulation with the Laplace—Metropolis Estimator , 1997 .

[54]  I. J. Myung,et al.  Applying Occam’s razor in modeling cognition: A Bayesian approach , 1997 .

[55]  M. Artés Statistical errors. , 1977, Medicina clinica.

[56]  Brian Dennis,et al.  Discussion: Should Ecologists Become Bayesians? , 1996 .

[57]  M. Schervish P Values: What They are and What They are Not , 1996 .

[58]  A. Raftery Bayesian Model Selection in Social Research , 1995 .

[59]  Donald B. Rubin,et al.  Avoiding Model Selection in Bayesian Social Research , 1995 .

[60]  Jacob Cohen The earth is round (p < .05) , 1994 .

[61]  Xiao-Li Meng,et al.  Posterior Predictive $p$-Values , 1994 .

[62]  StatisticsHarvard,et al.  Avoiding model selection in Bayesian social research , 1994 .

[63]  D. Lindley,et al.  The Analysis of Experimental Data: The Appreciation of Tea and Wine , 1993 .

[64]  Adrian E. Raftery,et al.  Bayesian Model Selection in Structural Equation Models , 1992 .

[65]  M. Aitkin Posterior Bayes Factors , 1991 .

[66]  J. Berger,et al.  Testing Precise Hypotheses , 1987 .

[67]  D. Johnstone,et al.  Tests of Significance in Theory and Practice , 1986 .

[68]  D. V. Lindley,et al.  [Why Isn't Everyone a Bayesian?]: Comment , 1986 .

[69]  B. Efron Why Isn't Everyone a Bayesian? , 1986 .

[70]  A. Zellner,et al.  Posterior odds ratios for selected regression hypotheses , 1980 .

[71]  P. Meehl Theoretical risks and tabular asterisks: Sir Karl, Sir Ronald, and the slow progress of soft psychology. , 1978 .

[72]  G. Schwarz Estimating the Dimension of a Model , 1978 .

[73]  D. J. Bartholomew,et al.  Scientific Inference, 3rd Ed. , 1976 .

[74]  Jacob Cohen,et al.  Applied multiple regression/correlation analysis for the behavioral sciences , 1979 .

[75]  J. Dickey,et al.  Bayes factors for independence in contingency tables , 1974 .

[76]  J. Dickey,et al.  The Weighted Likelihood Ratio, Sharp Hypotheses about Chances, the Order of a Markov Chain , 1970 .

[77]  David Lindley,et al.  Introduction to Probability and Statistics from a Bayesian Viewpoint , 1966 .

[78]  D. Cox Some problems connected with statistical inference , 1958 .