Bayesian model diagnostics using functional Bregman divergence

It is crucial to check validation of any statistical model after fitting it for a given set of data. In Bayesian statistics, a researcher can check the fit of the model using a variety of strategies. In this paper we consider two major aspects, first checking that the posterior inferences are reasonable, given the substantive context of the model; and then examining the sensitivity of inferences to reasonable changes in the prior distribution and the likelihood. Here we consider functional Bregman divergence between posterior distributions for model diagnostics, which produce methods for outlier detection as well as for prior sensitivity analysis. The methodology is exemplified through a logistic regression and a circular data model.

[1]  Shun-ichi Amari,et al.  $\alpha$ -Divergence Is Unique, Belonging to Both $f$-Divergence and Bregman Divergence Classes , 2009, IEEE Transactions on Information Theory.

[2]  Ming-Hui Chen.  Importance-Weighted Marginal Bayesian Posterior Density Estimation , 1994 .

[3]  I. Csiszár.  Generalized projections for non-negative functions , 1995 .

[4]  S. Geisser,et al.  A Predictive View of the Detection and Characterization of Influential Observations in Regression Analysis , 1983 .

[5]  P. Guttorp,et al.  Finding the Location of a Signal: A Bayesian Analysis , 1988 .

[6]  S. Chib,et al.  Understanding the Metropolis-Hastings Algorithm , 1995 .

[7]  Inderjit S. Dhillon,et al.  Clustering with Bregman Divergences , 2005, J. Mach. Learn. Res..

[8]  A. Dawid,et al.  Game theory, maximum entropy, minimum discrepancy and robust Bayesian decision theory , 2004, math/0410076.

[9]  Ben Taskar,et al.  Structured Prediction, Dual Extragradient and Bregman Projections , 2006, J. Mach. Learn. Res..

[10]  D. Dey,et al.  Robust Bayesian analysis using divergence measures , 1994 .

[11]  L. Bregman.  The relaxation method of finding the common point of convex sets and its application to the solution of problems in convex programming , 1967 .

[12]  D. Dey,et al.  Bayesian analysis of outlier problems using divergence measures , 1995 .

[13]  M. Dufwenberg.  Game theory. , 2011, Wiley interdisciplinary reviews. Cognitive science.

[14]  Maya R. Gupta,et al.  Functional Bregman Divergence and Bayesian Estimation of Distributions , 2006, IEEE Transactions on Information Theory.

[15]  Roland Badeau,et al.  Beta-Divergence as a Subclass of Bregman Divergence , 2011, IEEE Signal Processing Letters.

[16]  R. McCulloch.  Local Model Influence , 1989 .

[17]  J. Ghosh,et al.  On convergence of posterior distributions , 1995 .

[18]  D. K. Dey,et al.  ON BAYESIAN ROBUSTNESS OF CONTAMINATED CLASSES OF PRIORS , 1991 .

[19]  John A Silander,et al.  Multivariate forecasts of potential distributions of invasive plant species. , 2009, Ecological applications : a publication of the Ecological Society of America.

[20]  S. Chib,et al.  Bayesian analysis of binary and polychotomous response data , 1993 .

[21]  M. A. Stephens.  TECHNIQUES FOR DIRECTIONAL DATA , 1969 .

[22]  Dipak K. Dey,et al.  Generalized extreme value regression for binary response data: An application to B2B electronic payments system adoption , 2011, 1101.1373.