Power Analysis, p Values, and Bayesian Techniques: How Bayesian Techniques Can Be Used in HRD Literature

The Problem Researchers have described challenges and problems in reporting research that uses only p values and power to make decisions to reject the null hypothesis. Confusion about how to interpret null hypothesis statistical tests has resulted from mixed information presented in statistics articles and textbooks. The Solution Combining evidence from data with initial beliefs, Bayesian inference techniques help to provide uncontroversial support of a null hypothesis or alternative hypothesis. An overview of the limitations associated with only using p values and power to make decisions to reject or retain the null hypothesis are presented. Analyses across multiple studies with common parameters can be pooled using Bayesian techniques as a means for conducting meta-analysis. Examples using Bayesian techniques are given. The Stakeholders When designing a research study, researchers often use external elements and likelihood to make powerful inferences using Bayesian techniques. Those performing research in populations requiring sampling.

[1]  Jim Albert,et al.  Bayesian Computation with R , 2008 .

[2]  K. Bartlett,et al.  Training and organizational commitment among nurses following industry and organizational change in New Zealand and the United States , 2004 .

[3]  Steffen L. Lauritzen,et al.  Graphical models in R , 1996 .

[4]  J. Kruschke Bayesian estimation supersedes the t test. , 2013, Journal of experimental psychology. General.

[5]  Gorka Navarrete,et al.  Bayesian Hypothesis Testing: An Alternative to Null Hypothesis Significance Testing (NHST) in Psychology and Social Sciences , 2017 .

[6]  G. Woodworth Bayesian Full Rank MANOVA/MANCOVA: An Intermediate Exposition with Interactive Computer Examples , 1979 .

[7]  R. Kirk Practical Significance: A Concept Whose Time Has Come , 1996 .

[8]  D. J. Bartholomew,et al.  Scientific Inference, 3rd Ed. , 1976 .

[9]  J. Rouder,et al.  Default Bayes Factors for Model Selection in Regression , 2012, Multivariate behavioral research.

[10]  Andrew Gelman,et al.  Why We (Usually) Don't Have to Worry About Multiple Comparisons , 2009, 0907.2478.

[11]  J. Kiefer Conditional Confidence Statements and Confidence Estimators , 1977 .

[12]  Douglas G Altman,et al.  Bayesian random effects meta‐analysis of trials with binary outcomes: methods for the absolute risk difference and relative risk scales by D. E. Warn, S. G. Thompson and D. J. Spiegelhalter, Statistics in Medicine 2002; 21: 1601–1623 , 2005, Statistics in medicine.

[13]  E. S. Pearson,et al.  On the Problem of the Most Efficient Tests of Statistical Hypotheses , 1933 .

[14]  Baiyin Yang,et al.  Meta-Analysis Research and Theory Building , 2002 .

[15]  S. E. Ahmed,et al.  Markov Chain Monte Carlo: Stochastic Simulation for Bayesian Inference , 2008, Technometrics.

[16]  Torrin M. Liddell,et al.  The Bayesian New Statistics: Hypothesis testing, estimation, meta-analysis, and power analysis from a Bayesian perspective , 2016, Psychonomic bulletin & review.

[17]  J. Carlin,et al.  Beyond Power Calculations , 2014, Perspectives on psychological science : a journal of the Association for Psychological Science.

[18]  Ronald Aylmer Sir Fisher,et al.  020: The Goodness of Fit of Regression Formulae and the Distribution of Regression Coefficients. , 1922 .

[19]  D. Madigan,et al.  Model Selection and Accounting for Model Uncertainty in Graphical Models Using Occam's Window , 1994 .

[20]  S. James Press,et al.  Bayesian inference in the multivariate mixed model MANOVA , 1997 .

[21]  Xiao-Hua Zhou,et al.  Statistical Methods for Meta‐Analysis , 2008 .

[22]  W. Hager The statistical theories of Fisher and of Neyman and Pearson: A methodological perspective , 2013 .

[23]  Eric-Jan Wagenmakers,et al.  Calibrated Bayes Factors Should Not Be Used: A Reply to Hoijtink, van Kooten, and Hulsker , 2016, Multivariate behavioral research.

[24]  Sw. Banerjee,et al.  Hierarchical Modeling and Analysis for Spatial Data , 2003 .

[25]  Mike G. Tsionas,et al.  Bayes factors vs. P-values , 2018 .

[26]  Andrew GelmanyJanuary,et al.  Prior distribution , 2000 .

[27]  Jeffrey N. Rouder,et al.  Bayesian inference for psychology. Part II: Example applications with JASP , 2017, Psychonomic Bulletin & Review.

[28]  The art of statistics , 1965 .

[29]  N. Lazar,et al.  The ASA Statement on p-Values: Context, Process, and Purpose , 2016 .

[30]  D. J. Johnstone,et al.  Tests of Significance Following R. A. Fisher1 , 1987, The British Journal for the Philosophy of Science.

[31]  M. J. Bayarri,et al.  Calibration of ρ Values for Testing Precise Null Hypotheses , 2001 .

[32]  Joachim Vandekerckhove,et al.  Editorial: Bayesian methods for advancing psychological science , 2018, Psychonomic bulletin & review.

[33]  Yongtao Guan,et al.  On the Null Distribution of Bayes Factors in Linear Regression , 2018, Journal of the American Statistical Association.

[34]  Bruno D. Zumbo,et al.  A note on misconceptions concerning prospective and retrospective power , 1998 .

[35]  A. Raftery Bayesian Model Selection in Social Research , 1995 .

[36]  Elwood F. Holton,et al.  The effectiveness of managerial leadership development programs: a meta-analysis of studies from 1982-2001 , 2004 .

[37]  E. S. Pearson,et al.  On the Problem of the Most Efficient Tests of Statistical Hypotheses , 1933 .

[38]  Steven G. Deeks,et al.  Breaking Free of Sample Size Dogma to Perform Innovative Translational Research , 2011, Science Translational Medicine.

[39]  Regina Nuzzo,et al.  Scientific method: Statistical errors , 2014, Nature.

[40]  Peter R. Nelson,et al.  Multiple Comparisons: Theory and Methods , 1997 .

[41]  Annamaria Guolo,et al.  Random-effects meta-analysis: the number of studies matters , 2017, Statistical methods in medical research.

[42]  M. J. Bayarri,et al.  Confusion Over Measures of Evidence (p's) Versus Errors (α's) in Classical Statistical Testing , 2003 .

[43]  Jeffrey N. Rouder,et al.  Default Bayes factors for ANOVA designs , 2012 .

[44]  S. Goodman,et al.  A comment on replication, p-values and evidence. , 1992, Statistics in medicine.

[45]  Darlene F. Russ-Eft,et al.  Organizational responsiveness of Russian and American growth-oriented small and medium enterprises (SMEs) , 2010 .

[46]  R. Baker,et al.  The Relationship between Administrative Intensity and Student Retention and Success: A Three-Year Study , 2018, Education Sciences.

[47]  M. Clyde,et al.  Mixtures of g Priors for Bayesian Variable Selection , 2008 .

[48]  C. Rover Bayesian Random-Effects Meta-Analysis Using the bayesmeta R Package , 2017, Journal of Statistical Software.

[49]  G. Glass Primary, Secondary, and Meta-Analysis of Research1 , 1976 .

[50]  Kazuo Shigemasu,et al.  Bayesian manova and manocova under exchangeability , 1985 .

[51]  J. Durlak How to select, calculate, and interpret effect sizes. , 2009, Journal of pediatric psychology.

[52]  Jamie L. Callahan,et al.  Making Subjective Judgments in Quantitative Studies: The Importance of Using Effect Sizes and Confidence Intervals. , 2006 .

[53]  A. Alexandrova The British Journal for the Philosophy of Science , 1965, Nature.

[54]  Raymond Hubbard,et al.  P Values are not Error Probabilities , 2003 .

[55]  Z. Dienes,et al.  Four reasons to prefer Bayesian analyses over significance testing , 2017, Psychonomic bulletin & review.

[56]  Jeffrey N. Rouder,et al.  The fallacy of placing confidence in confidence intervals , 2015, Psychonomic bulletin & review.

[57]  Jianwen Liao,et al.  Organizational Absorptive Capacity and Responsiveness: An Empirical Investigation of Growth–Oriented SMEs , 2003 .

[58]  Ingram Olkin,et al.  Estimation of a Single Effect Size: Parametric and Nonparametric Methods , 1985 .

[59]  Pamela A. Moss,et al.  Standards for Reporting on Empirical Social Science Research in AERA Publications American Educational Research Association , 2006 .

[60]  Eiki Satake,et al.  Beyond P values and Hypothesis Testing: Using the Minimum Bayes Factor to Teach Statistical Inference in Undergraduate Introductory Statistics Courses , 2017 .

[61]  Glenn Shafer,et al.  A Mathematical Theory of Evidence , 2020, A Mathematical Theory of Evidence.

[62]  Annetine C. Gelijns,et al.  An Introduction to a Bayesian Method for Meta-Analysis: The Confidence Profile Method , 1990 .

[63]  Kim F. Nimon HRDQ Submissions of Quantitative Research Reports: Three Common Comments in Decision Letters and a Checklist , 2017 .

[64]  R. Hubbard,et al.  Why P Values Are Not a Useful Measure of Evidence in Statistical Significance Testing , 2008 .

[65]  Rodney X. Sturdivant,et al.  Applied Logistic Regression: Hosmer/Applied Logistic Regression , 2005 .

[66]  D. Spiegelhalter,et al.  Bayesian random effects meta‐analysis of trials with binary outcomes: methods for the absolute risk difference and relative risk scales , 2002, Statistics in medicine.

[67]  J. Bijak Forecasting international migration in Europe : a Bayesian view , 2011 .

[68]  K R Abrams,et al.  Bayesian methods in meta-analysis and evidence synthesis. , 2001, Statistical methods in medical research.

[69]  H. Jeffreys A Treatise on Probability , 1922, Nature.

[70]  B. Efron Why Isn't Everyone a Bayesian? , 1986 .

[71]  Jeffrey N. Rouder,et al.  Bayes factor approaches for testing interval null hypotheses. , 2011, Psychological methods.

[72]  J. E. Walsh,et al.  Contributions to the Theory of Rank Order Statistics--The Two Sample Case , 1958 .

[73]  Robert West,et al.  Using Bayes factors for testing hypotheses about intervention effectiveness in addictions research , 2016, Addiction.

[74]  Jörg Rieskamp,et al.  An Introduction to Bayesian Hypothesis Testing for Management Research , 2015 .

[75]  Arthur P. Dempster,et al.  A Generalization of Bayesian Inference , 1968, Classic Works of the Dempster-Shafer Theory of Belief Functions.

[76]  Winston Bennett,et al.  Conducting Meta-Analysis Using SAS , 2001 .

[77]  Jeffrey N. Rouder,et al.  Bayesian t tests for accepting and rejecting the null hypothesis , 2009, Psychonomic bulletin & review.

[78]  David W. Hosmer,et al.  Applied Logistic Regression , 1991 .