Bayesian Restricted Likelihood Methods

Bayesian methods have proven themselves to be successful across a wide range of scientific problems and have many well-documented advantages over competing methods. However, these methods run into difficulties for two major and prevalent classes of problems: handling data sets with outliers and dealing with model misspecification. This dissertation introduces the restricted likelihood as a solution to these problems. When working with the restricted likelihood, we summarize the data through a set of (insufficient) statistics, targeting inferential quantities of interest, and update the prior distribution with the summary statistics rather than the complete data. By choice of conditioning statistics, we retain the main benefits of Bayesian methods while reducing the sensitivity of the analysis to features of the data not picked up by the conditioning statistics. The method is general, but this dissertation concentrates on applying the restricted likelihood to the common Bayesian linear model when outliers are of concern. For conditioning statistics we mostly consider classical robust M-estimators. In this sense, the method can be viewed as a blend of classical robust estimation techniques with the Bayesian paradigm. Of considerable interest is the comparison of the new method with more traditional approaches to dealing with outliers. In the face of model inadequacy caused by outliers, the traditional view is that one should build a better model which attempts to explain the outlier generating process. Two classical parametric approaches to deal with the problem are to replace the standard density by a thick-tailed or mixture density. Using several ii data analyses, the benefits of the new approach over the traditional approaches is made apparent. These benefits often manifest themselves through more realistic assumptions and more precise predictive performance. A major contribution of this work is the development of implementation strategies to fit these models. Since restricted likelihoods are rarely tractable, implementation is nontrivial. For low dimensional problems, computational methods relying on density estimation and numerical integration can often be efficiently employed. These methods break down in higher dimensions and we develop a novel Markov Chain Monte Carlo (MCMC) algorithm to handle these situations. In particular, the MCMC algorithm for the traditional posterior is augmented with a step that simulates new data conditional on the parameters and observed summary statistics. Derivations of the adjustments needed for this data augmented algorithm are given for the linear regression setting. Model choice within the restricted likelihood paradigm is also introduced.