A Spreadsheet for Bayesian Posterior Compatibility Intervals and Magnitude-Based Decisions

The usual compatibility (confidence) interval for an effect in a sample can be modified to a Bayesian posterior compatibility (credibility) interval by combining the value of the effect and its interval with a prior belief in the effect expressed as its own value and interval. The spreadsheet accompanying this article provides such analyses for four kinds of effect: differences in means and other t-distributed estimates; percent or factor effects for such means derived from analyses of log-transformed dependent variables; ratios of risks, odds, hazards, and counts derived from generalized linear models; and Pearson correlation coefficients. Inclusion of a smallest important value for the effect allows the spreadsheet to provide a probabilistic magnitude-based decision about implementation of a clinically or practically relevant effect and about adequate precision for a non-clinical effect. The spreadsheet shows that realistic weakly informative priors applied to compatibility intervals from typically small samples produce posterior intervals that are practically the same as the original intervals. The minimally informative prior implicit in the magnitude-based decision method therefore provides acceptable Bayesian probabilistic estimates of the true magnitude of effects. Weakly informative priors should nevertheless be used to shrink unrealistically large compatibility limits arising from very small sample sizes and to reduce bias in effect magnitudes from generalized linear models with sparse data. Use of more-informative priors is problematic, owing to the difficulty of quantifying a belief and to bias in the belief.