An intuitive framework for Bayesian posterior simulation methods

Abstract Purpose Bayesian inference has become popular. It offers several pragmatic approaches to account for uncertainty in inference decision-making. Various estimation methods have been introduced to implement Bayesian methods. Although these algorithms are powerful, they are not always easy to grasp for non-statisticians. This paper aims to provide an intuitive framework of four essential Bayesian computational methods for epidemiologists and other health researchers. We do not cover an extensive mathematical discussion of these approaches, but instead offer a non-quantitative description of these algorithms and provide some illuminating examples. Materials and methods Bayesian computational methods, namely importance sampling, rejection sampling, Markov chain Monte Carlo, and data augmentation are presented. Results and conclusions The substantial amount of research published on Bayesian inference has highlighted its popularity among researchers, while the basic concepts are not always straightforward for interested learners. We show that alternative approaches such as a weighted prior approach, which are intuitively appealing and easy-to-understand, work well in the case of low-dimensional problems and appropriate prior information. Otherwise, MCMC is a trouble-free tool in those cases.

[1]  Joachim Vandekerckhove,et al.  Bayesian inference for psychology, part III: Parameter estimation in nonstandard models , 2017, Psychonomic Bulletin & Review.

[2]  S Greenland,et al.  Data augmentation priors for Bayesian and semi‐Bayes analyses of conditional‐logistic and proportional‐hazards regression , 2001, Statistics in medicine.

[3]  Sander Greenland,et al.  Separation in Logistic Regression: Causes, Consequences, and Control. , 2018, American journal of epidemiology.

[4]  George E. P. Box,et al.  Sampling and Bayes' inference in scientific modelling and robustness , 1980 .

[5]  Christian P. Robert,et al.  Introducing Monte Carlo Methods with R , 2009 .

[6]  Sander Greenland,et al.  Penalization, bias reduction, and default priors in logistic and related categorical and survival regressions , 2015, Statistics in medicine.

[7]  Milica Miočević A Tutorial in Bayesian Mediation Analysis With Latent Variables , 2019, Methodology.

[8]  J. Hammersley,et al.  Poor Man's Monte Carlo , 1954 .

[9]  Nando de Freitas,et al.  Sequential Monte Carlo Methods in Practice , 2001, Statistics for Engineering and Information Science.

[10]  Edward J. Bedrick,et al.  Bayesian Binomial Regression: Predicting Survival at a Trauma Center , 1997 .

[11]  Sander Greenland,et al.  Bayesian perspectives for epidemiological research: I. Foundations and basic methods. , 2006, International journal of epidemiology.

[12]  A. W. Rosenbluth,et al.  MONTE CARLO CALCULATION OF THE AVERAGE EXTENSION OF MOLECULAR CHAINS , 1955 .

[13]  David Morgan,et al.  Bayesian applications in pharmaceutical statistics , 2018, Pharmaceutical statistics.

[14]  Jeffrey N. Rouder,et al.  Bayesian inference for psychology. Part I: Theoretical advantages and practical ramifications , 2017, Psychonomic Bulletin & Review.

[15]  A. Owen,et al.  Safe and Effective Importance Sampling , 2000 .

[16]  Sander Greenland,et al.  Sparse data bias: a problem hiding in plain sight , 2016, British Medical Journal.

[17]  Jeffrey N. Rouder,et al.  Bayesian inference for psychology. Part II: Example applications with JASP , 2017, Psychonomic Bulletin & Review.

[18]  R. Christensen,et al.  A New Perspective on Priors for Generalized Linear Models , 1996 .

[19]  Norman T. J. Bailey The Elements of Stochastic Processes with Applications to the Natural Sciences , 1964 .

[20]  Radford M. Neal Annealed importance sampling , 1998, Stat. Comput..

[21]  Marc Pfister,et al.  Concepts and Challenges in Quantitative Pharmacology and Model-Based Drug Development , 2008, The AAPS Journal.

[22]  Marc R Gastonguay,et al.  Pharmacometrics: A Multidisciplinary Field to Facilitate Critical Thinking in Drug Development and Translational Research Settings , 2008, Journal of clinical pharmacology.

[23]  A. F. Smith,et al.  Bayesian Methods in Practice: Experiences in the Pharmaceutical Industry , 1986 .

[24]  Alexander Etz,et al.  Introduction to Bayesian Inference for Psychology , 2018, Psychonomic bulletin & review.

[25]  M. Evans,et al.  Methods for Approximating Integrals in Statistics with Special Emphasis on Bayesian Integration Problems , 1995 .

[26]  Jim Albert,et al.  Teaching Bayes' Rule: A Data-Oriented Approach , 1997 .

[27]  Prathiba Natesan,et al.  Fitting Bayesian Models for Single-Case Experimental Designs , 2019, Methodology.

[28]  Andrew P Grieve,et al.  25 years of Bayesian methods in the pharmaceutical industry: a personal, statistical bummel , 2007, Pharmaceutical statistics.

[29]  T. Hesterberg,et al.  Weighted Average Importance Sampling and Defensive Mixture Distributions , 1995 .

[30]  Brandon M. Turner,et al.  Journal of Mathematical Psychology a Tutorial on Approximate Bayesian Computation , 2022 .