Bregman divergence to generalize Bayesian influence measures for data analysis

Abstract For existing Bayesian cross-validated measure of influence of each observation on the posterior distribution, this paper considers a generalization using the Bregman Divergence (BD). We investigate various practically useful and desirable properties of these BD based measures to demonstrate the superiority of these measures compared to existing Bayesian measures of influence and Bayesian residual based diagnostics. We provide a practical and easily comprehensible method for calibrating these BD based measures. Also, we show how to compute our BD based measure via Markov chain Monte Carlo (MCMC) samples from a single posterior based on the full data. Using a Bayesian meta-analysis of clinical trials, we illustrate how our new measures of influence of observations have more useful practical roles for data analysis than popular Bayesian residual analysis tools.

[1]  John W. Tukey,et al.  Exploratory Data Analysis. , 1979 .

[2]  R. Ehlers,et al.  Bayesian influence diagnostics using normalized functional Bregman divergence , 2019, Communications in Statistics - Theory and Methods.

[3]  R. Hogg.  Statistical Robustness: One View of its use in Applications Today , 1979 .

[4]  D. Dey,et al.  Bayesian analysis of outlier problems using divergence measures , 1995 .

[5]  R. McCulloch.  Local Model Influence , 1989 .

[6]  Bradley P. Carlin,et al.  Markov Chain Monte Carlo conver-gence diagnostics: a comparative review , 1996 .

[7]  Seymour Geisser,et al.  Estimative influence measures for the multivariate general linear model , 1985 .

[8]  Inderjit S. Dhillon,et al.  Clustering with Bregman Divergences , 2005, J. Mach. Learn. Res..

[9]  S. Chatterjee.  Sensitivity analysis in linear regression , 1988 .

[10]  D. Dey,et al.  Bayesian Model Assessment and Selection Using Bregman Divergence , 2020 .

[11]  Robert E. Weiss,et al.  Bayesian Marginal Influence Assessment , 1998 .

[12]  R. Welsch,et al.  The Hat Matrix in Regression and ANOVA , 1978 .

[13]  Hong Chang,et al.  Bayesian Approach for Nonlinear Random Effects Models , 1997 .

[14]  Dipak K. Dey,et al.  Bayesian model diagnostics using functional Bregman divergence , 2014, J. Multivar. Anal..

[15]  Eric T. Bradlow,et al.  Case Influence Analysis in Bayesian Inference , 1997 .

[16]  R. Weiss.  An approach to Bayesian sensitivity analysis , 1996 .

[17]  P. J. Huber.  Robust methods of estimation of regression coefficients 1 , 1977 .

[18]  L. I. Pettit,et al.  Diagnostics in Bayesian Model Choice , 1986 .

[19]  R Peto,et al.  Relevance of breast cancer hormone receptors and other factors to the efficacy of adjuvant tamoxifen: patient-level meta-analysis of randomised trials , 2011, The Lancet.

[20]  R. Dennis Cook,et al.  Detection of Influential Observation in Linear Regression , 2000, Technometrics.

[21]  Shun-ichi Amari,et al.  $\alpha$ -Divergence Is Unique, Belonging to Both $f$-Divergence and Bregman Divergence Classes , 2009, IEEE Transactions on Information Theory.