On the Local Sensitivity of M-Estimation: Bayesian and Frequentist Applications

Author(s): Giordano, Ryan James | Advisor(s): Jordan, Michael I; McAuliffe, Jon | Abstract: This thesis uses the local sensitivity of M-estimators to address a number ofextant problems in Bayesian and frequentist statistics. First, by exploiting aduality from the Bayesian robustness literature between sensitivity andcovariances, I provide significantly improved covariance estimates for meanfield variational Bayes (MFVB) procedures at little extra computational cost.Prior to this work, applications of MFVB have arguably been limited toprediction problems rather than inference problems for lack of reliableuncertainty measures. Second, I provide practical finite-sample accuracy boundsfor the ``infinitesimal jackknife'' (IJ), a classical measure of localsensitivity to an empirical process. In doing so, I bridge a gap betweenclassical IJ theory and recent machine learning practice, showing that stringentclassical conditions for the consistency of the IJ can be relaxed for restrictedbut useful classes of weight vectors, such as those of leave-K-out crossvalidation. Finally, I provide techniques to quantify the sensitivity of theinferred number of clusters in Bayesian nonparametric (BNP) unsupervisedclustering problems to the form of the Dirichlet process prior. By consideringlocal sensitivity to be an approximation to global sensitivity rather than ameasure of robustness per se, I provide tools with considerablyimproved ability to extrapolate to different priors. Because each of thesediverse applications are based on the same formal technique---the Taylor seriesexpansion of an M-estimator---this work captures in a unified way thecomputational difficulties associated with each, and I provide open-source toolsin Python and R to assist in their computation.