How Do I Know What My Theory Predicts?

To get evidence for or against a theory relative to the null hypothesis, one needs to know what the theory predicts. The amount of evidence can then be quantified by a Bayes factor. Specifying the sizes of the effect one’s theory predicts may not come naturally, but I show some ways of thinking about the problem, some simple heuristics that are often useful when one has little relevant prior information. These heuristics include the room-to-move heuristic (for comparing mean differences), the ratio-of-scales heuristic (for regression slopes), the ratio-of-means heuristic (for regression slopes), the basic-effect heuristic (for analysis of variance effects), and the total-effect heuristic (for mediation analysis).

[1]  Z. Dienes,et al.  Application of the ex-Gaussian function to the effect of the word blindness suggestion on Stroop task performance suggests no word blindness , 2013, Front. Psychol..

[2]  Z. Diénès Understanding Psychology as a Science: An Introduction to Scientific and Statistical Inference , 2008 .

[3]  J. Rouder Optional stopping: No problem for Bayesians , 2014, Psychonomic bulletin & review.

[4]  Ron S. Kenett,et al.  Many perspectives on Deborah Mayo's "Statistical Inference as Severe Testing: How to Get Beyond the Statistics Wars" , 2019, 1905.08876.

[5]  K. Popper,et al.  Conjectures and refutations;: The growth of scientific knowledge , 1972 .

[6]  Kate Cavanagh,et al.  A randomised controlled trial of a brief online mindfulness-based intervention. , 2013, Behaviour research and therapy.

[7]  Alexander Etz,et al.  Bayesian Inference and Testing Any Hypothesis You Can Specify , 2018, Advances in Methods and Practices in Psychological Science.

[8]  Cédric Batailler,et al.  New recommendations for testing indirect effects in mediational models: The need to report and test component paths. , 2018, Journal of personality and social psychology.

[9]  C. Gallistel,et al.  The Importance of Proving the Null , 2022 .

[10]  Z. Dienes,et al.  Can unconscious intentions be more effective than conscious intentions? Test of the role of metacognition in hypnotic response , 2020, Cortex.

[11]  Z. Dienes,et al.  Facial beauty affects implicit and explicit learning of men and women differently , 2015, Front. Psychol..

[12]  Z. Dienes,et al.  Four reasons to prefer Bayesian analyses over significance testing , 2017, Psychonomic bulletin & review.

[13]  Alexander Etz,et al.  Introduction to Bayesian Inference for Psychology , 2018, Psychonomic bulletin & review.

[14]  Christopher D. Chambers,et al.  Redefine statistical significance , 2017, Nature Human Behaviour.

[15]  A. Muise,et al.  Eroticism Versus Nurturance , 2019, Social Psychology.

[16]  Z. Dienes How Bayes factors change scientific practice , 2016 .

[17]  Conor Mayo-Wilson,et al.  Statistical Inference as Severe Testing: How to Get beyond the Statistics , 2021, The Philosophical Review.

[18]  B. Levy,et al.  To Love is to Suffer: Older Adults’ Daily Emotional Contagion to Perceived Spousal Suffering , 2015, The journals of gerontology. Series B, Psychological sciences and social sciences.

[19]  J. Pearl,et al.  Causal Inference in Statistics: A Primer , 2016 .

[20]  Michela Schröder-Abé,et al.  Benefits of power posing: effects on dominance and social sensitivity , 2017 .

[21]  Jeffrey N. Rouder,et al.  Bayesian t tests for accepting and rejecting the null hypothesis , 2009, Psychonomic bulletin & review.

[22]  Eric-Jan Wagenmakers,et al.  Informed Bayesian t-Tests , 2017, The American Statistician.

[23]  Sander Greenland,et al.  Invited Commentary: The Need for Cognitive Science in Methodology , 2017, American journal of epidemiology.

[24]  S. Dehaene,et al.  Unconscious memory suppression , 2018, Cognition.

[25]  M. Lee,et al.  Determining informative priors for cognitive models , 2018, Psychonomic bulletin & review.

[26]  Axel Cleeremans,et al.  Intentional binding as Bayesian cue combination: Testing predictions with trait individual differences. , 2018, Journal of experimental psychology. Human perception and performance.

[27]  E. Wagenmakers,et al.  Bayesian Inference for Correlations in the Presence of Measurement Error and Estimation Uncertainty , 2017 .

[28]  Jin Fan,et al.  Hypnotic suggestion and the modulation of Stroop interference. , 2002, Archives of general psychiatry.

[29]  Michèle B. Nuijten,et al.  A default Bayesian hypothesis test for mediation , 2015, Behavior research methods.

[30]  Richard D. Morey,et al.  Is There a Free Lunch in Inference? , 2016, Top. Cogn. Sci..

[31]  P. Meehl Theory-Testing in Psychology and Physics: A Methodological Paradox , 1967, Philosophy of Science.

[32]  E. Woody An SEM Perspective on Evaluating Mediation: What Every Clinical Researcher Needs to Know , 2011 .

[33]  Eric-Jan Wagenmakers,et al.  The JASP guidelines for conducting and reporting a Bayesian analysis , 2019, Psychonomic Bulletin & Review.

[34]  Z. Dienes,et al.  Bayes to the rescue: Does the type of hypnotic induction matter? , 2019 .

[35]  Zoltan Dienes,et al.  How Bayesian statistics are needed to determine whether mental states are unconscious , 2015 .

[36]  Wolf Vanpaemel,et al.  Strong theory testing using the prior predictive and the data prior. , 2020, Psychological review.

[37]  Deborah G. Mayo,et al.  Statistical Inference as Severe Testing , 2018 .

[38]  Francis Tuerlinckx,et al.  Increasing Transparency Through a Multiverse Analysis , 2016, Perspectives on psychological science : a journal of the Association for Psychological Science.

[39]  T. Baguley Standardized or simple effect size: what should be reported? , 2009, British journal of psychology.

[40]  James M. Dickey,et al.  Scientific Reporting and Personal Probabilities: Student's Hypothesis , 1973 .

[41]  Zoltan Dienes,et al.  Using Bayes to get the most out of non-significant results , 2014, Front. Psychol..

[42]  Axel Cleeremans,et al.  The Sense of Agency as Tracking Control , 2016, PloS one.

[43]  Jeffrey N. Rouder,et al.  The philosophy of Bayes’ factors and the quantification of statistical evidence , 2016 .

[44]  L. M. M.-T. Theory of Probability , 1929, Nature.

[45]  Jeffrey Rouder,et al.  Optional Stopping and the Interpretation of The Bayes Factor , 2019 .

[46]  Felix D. Schönbrodt,et al.  Sequential Hypothesis Testing With Bayes Factors: Efficiently Testing Mean Differences , 2017, Psychological methods.