Bias-Aware Confidence Intervals for Empirical Bayes Analysis

In an empirical Bayes analysis, we use data from repeated sampling to imitate inferences made by an oracle Bayesian with extensive knowledge of the data-generating distribution. Existing results provide a comprehensive characterization of when and why empirical Bayes point estimates accurately recover oracle Bayes behavior. In this paper, we develop confidence intervals that provide asymptotic frequentist coverage of empirical Bayes estimands. Our intervals include an honest assessment of bias even in situations where empirical Bayes point estimates may converge very slowly. Applied to multiple testing situations, our approach provides flexible and practical confidence statements about the local false sign rate. As an intermediate step in our approach, we develop methods for inference about linear functionals of the effect size distribution in the Bayes deconvolution problem that may be of independent interest.

[1]  Ananda Theertha Suresh,et al.  Sample complexity of population recovery , 2017, COLT.

[2]  I. Ibragimov,et al.  On Nonparametric Estimation of the Value of a Linear Functional in Gaussian White Noise , 1985 .

[3]  Cun-Hui Zhang.  Fourier Methods for Estimating Mixing Densities and Distributions , 1990 .

[4]  Sanford Weisberg,et al.  Computing science and statistics : proceedings of the 30th Symposium on the Interface, Minneapolis, Minnesota, May 13-16, 1998 : dimension reduction, computational complexity and information , 1998 .

[5]  Dimitris N. Politis,et al.  On a family of smoothing kernels of in nite order , 2022 .

[6]  Peter D. Hoff,et al.  Adaptive sign error control , 2017, Journal of Statistical Planning and Inference.

[7]  M. Reiß,et al.  Wasserstein and total variation distance between marginals of L\'evy processes , 2017, 1710.02715.

[8]  Bernard W. Silverman,et al.  Discretization effects in statistical inverse problems , 1991, J. Complex..

[9]  J. Zubizarreta.  Stable Weights that Balance Covariates for Estimation With Incomplete Outcome Data , 2015 .

[10]  T. Cai,et al.  Minimax estimation of linear functionals over nonconvex parameter spaces , 2004, math/0406427.

[11]  Adel Javanmard,et al.  Confidence intervals and hypothesis testing for high-dimensional regression , 2013, J. Mach. Learn. Res..

[12]  C. Morris.  Parametric Empirical Bayes Confidence Intervals , 1983 .

[13]  William Fithian,et al.  Statistical methods for replicability assessment , 2019, The Annals of Applied Statistics.

[14]  Iain Dunning,et al.  JuMP: A Modeling Language for Mathematical Optimization , 2015, SIAM Rev..

[15]  Alexander Peysakhovich,et al.  Improving pairwise comparison models using Empirical Bayes shrinkage , 2018, ArXiv.

[16]  T. Shakespeare,et al.  Observational Studies , 2003 .

[17]  Matthew Stephens,et al.  False discovery rates: a new deal , 2016, bioRxiv.

[18]  Joseph G Ibrahim,et al.  Heavy-tailed prior distributions for sequence count data: removing the noise and preserving large differences , 2018, bioRxiv.

[19]  W. Huber,et al.  Moderated estimation of fold change and dispersion for RNA-seq data with DESeq2 , 2014, Genome Biology.

[20]  David L. Donoho,et al.  Hardest One-Dimensional Subproblems , 2008 .

[21]  Timothy B. Armstrong,et al.  Optimal Inference in a Class of Regression Models , 2015, 1511.06028.

[22]  L. Devroye.  A Note on the Usefulness of Superkernels in Density Estimation , 1992 .

[23]  Roger Koenker,et al.  Unobserved Heterogeneity in Income Dynamics: An Empirical Bayes Perspective , 2014 .

[24]  Clifford B. Cordy,et al.  Deconvolution of a Distribution Function , 1997 .

[25]  Gordon K Smyth,et al.  Linear Models and Empirical Bayes Methods for Assessing Differential Expression in Microarray Experiments , 2004, Statistical applications in genetics and molecular biology.

[26]  H. Robbins.  An Empirical Bayes Approach to Statistics , 1956 .

[27]  Timothy B. Armstrong,et al.  Sensitivity Analysis using Approximate Moment Condition Models , 2018, Quantitative Economics.

[28]  I. Johnstone,et al.  Needles and straw in haystacks: Empirical Bayes estimates of possibly sparse sequences , 2004, math/0410088.

[29]  E. Lander,et al.  Gene expression correlates of clinical prostate cancer behavior. , 2002, Cancer cell.

[30]  I. Dattner,et al.  ON DECONVOLUTION OF DISTRIBUTION FUNCTIONS , 2010, 1006.3918.

[31]  Stefan Wager,et al.  Covariate-Powered Empirical Bayes Estimation , 2019, NeurIPS.

[32]  R. Koenker,et al.  CONVEX OPTIMIZATION, SHAPE CONSTRAINTS, COMPOUND DECISIONS, AND EMPIRICAL BAYES RULES , 2013 .

[33]  A. Juditsky,et al.  Nonparametric estimation by convex programming , 2009, 0908.3108.

[34]  Dahua Lin,et al.  Distributions.jl: Definition and Modeling of Probability Distributions in the JuliaStats Ecosystem , 2019, J. Stat. Softw..

[35]  Paul Deheuvels,et al.  Uniform Limit Laws for Kernel Density Estimators on Possibly Unbounded Intervals , 2000 .

[36]  Bradley Efron,et al.  Two modeling strategies for empirical Bayes estimation. , 2014, Statistical science : a review journal of the Institute of Mathematical Statistics.

[37]  Lawrence D. Brown,et al.  The Poisson Compound Decision Problem Revisited , 2010, 1006.4582.

[38]  Arlene K. H. Kim.  Minimax bounds for estimation of normal mixtures , 2011, 1112.4565.

[39]  Jianqing Fan.  On the Optimal Rates of Convergence for Nonparametric Deconvolution Problems , 1991 .

[40]  Francis Tuerlinckx,et al.  Type S error rates for classical and Bayesian single and multiple comparison procedures , 2000, Comput. Stat..

[41]  Stephen P. Boyd,et al.  ECOS: An SOCP solver for embedded systems , 2013, 2013 European Control Conference (ECC).

[42]  E. Candès,et al.  A knockoff filter for high-dimensional selective inference , 2016, The Annals of Statistics.

[43]  D. Donoho,et al.  Geometrizing Rates of Convergence, III , 1991 .

[44]  Mark G. Low.  Bias-Variance Tradeoffs in Functional Estimation Problems , 1995 .

[45]  Bradley Efron,et al.  deconvolveR: A G-Modeling Program for Deconvolution and Empirical Bayes Estimation , 2020, Journal of Statistical Software.

[46]  Alberto Abadie,et al.  Choosing among Regularized Estimators in Empirical Economics: The Risk of Machine Learning , 2019, Review of Economics and Statistics.

[47]  C. Stein,et al.  Estimation with Quadratic Loss , 1992 .

[48]  Alexandre B. Tsybakov,et al.  Introduction to Nonparametric Estimation , 2008, Springer series in statistics.

[49]  Wenhua Jiang,et al.  General maximum likelihood empirical Bayes estimation of normal means , 2009, 0908.1709.

[50]  Bradley Efron,et al.  Empirical Bayes deconvolution estimates , 2016 .

[51]  Yuya Sasaki,et al.  Inference based on Kotlarski's Identity , 2018, 1808.09375.

[52]  S. Zeger,et al.  A Smooth Nonparametric Estimate of a Mixing Distribution Using Mixtures of Gaussians , 1996 .

[53]  M. Nussbaum.  Asymptotic Equivalence of Density Estimation and Gaussian White Noise , 1996 .

[54]  J. Robins,et al.  Adaptive nonparametric confidence sets , 2006, math/0605473.

[55]  B. Efron,et al.  Stein's Estimation Rule and Its Competitors- An Empirical Bayes Approach , 1973 .

[56]  D. Donoho.  Statistical Estimation and Optimal Recovery , 1994 .

[57]  V. Genon-Catalot,et al.  Laguerre and Hermite bases for inverse problems , 2018, Journal of the Korean Statistical Society.

[58]  Stefan Wager,et al.  Debiased Inference of Average Partial Effects in Single-Index Models , 2018, 1811.02547.

[59]  C. Matias,et al.  MINIMAX ESTIMATION OF LINEAR FUNCTIONALS IN THE CONVOLUTION MODEL , 2004 .

[60]  Yoav Zemel,et al.  Statistical Aspects of Wasserstein Distances , 2018, Annual Review of Statistics and Its Application.

[61]  Y. Benjamini,et al.  False Discovery Rate–Adjusted Multiple Confidence Intervals for Selected Parameters , 2005 .

[62]  Marianna Pensky,et al.  Minimax theory of estimation of linear functionals of the deconvolution density with or without sparsity , 2014, 1411.1660.

[63]  F. Comte,et al.  Adaptive estimation of linear functionals in the convolution model and applications , 2009, 0902.1443.

[64]  Stefan Wager,et al.  Augmented minimax linear estimation , 2017, The Annals of Statistics.

[65]  B. Efron.  Tweedie’s Formula and Selection Bias , 2011, Journal of the American Statistical Association.

[66]  Stefan Wager,et al.  Optimized Regression Discontinuity Designs , 2017, Review of Economics and Statistics.

[67]  F. Götze,et al.  The Berry-Esseen bound for student's statistic , 1996 .

[68]  Nathan Kallus,et al.  Generalized Optimal Matching Methods for Causal Inference , 2016, J. Mach. Learn. Res..

[69]  E. Giné,et al.  Rates of strong uniform consistency for multivariate kernel density estimators , 2002 .

[70]  D. Yekutieli,et al.  Selective Sign-Determining Multiple Confidence Intervals with FCR Control , 2014, Statistica Sinica.

[71]  Art B. Owen.  Confidence intervals with control of the sign error in low power settings , 2016 .

[72]  Christian P. Robert,et al.  Large-scale inference , 2010 .

[73]  D. Donoho.  One-sided inference about functionals of a density , 1988 .

[74]  O. Sacko,et al.  Hermite density deconvolution , 2020, Latin American Journal of Probability and Mathematical Statistics.

[75]  David A. Hirshberg,et al.  Balancing Out Regression Error: Efficient Treatment Effect Estimation without Smooth Propensities , 2017 .

[76]  F. Comte,et al.  Sobolev-Hermite versus Sobolev nonparametric density estimation on R\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\ , 2017, Annals of the Institute of Statistical Mathematics.

[77]  T. Tony Cai,et al.  A note on nonparametric estimation of linear functionals , 2003 .

[78]  G. Imbens,et al.  Approximate residual balancing: debiased inference of average treatment effects in high dimensions , 2016, 1604.07125.

[79]  Zhuang Ma,et al.  Group-Linear Empirical Bayes Estimates for a Heteroscedastic Normal Mean , 2015, 1503.08503.

[80]  Raj Chetty,et al.  The Impacts of Neighborhoods on Intergenerational Mobility Ii: County-Level Estimates , 2016 .

[81]  Paul Deheuvels,et al.  Asymptotic Certainty Bands for Kernel Density Estimators Based upon a Bootstrap Resampling Scheme , 2008 .

[82]  Lawrence D. Brown,et al.  NONPARAMETRIC EMPIRICAL BAYES AND COMPOUND DECISION APPROACHES TO ESTIMATION OF A HIGH-DIMENSIONAL VECTOR OF NORMAL MEANS , 2009, 0908.1712.

[83]  Alan Edelman,et al.  Julia: A Fresh Approach to Numerical Computing , 2014, SIAM Rev..

[84]  B. Efron,et al.  Data Analysis Using Stein's Estimator and its Generalizations , 1975 .

[85]  Charles F. Manski,et al.  Confidence Intervals for Partially Identified Parameters , 2003 .

[86]  John D. Storey,et al.  Empirical Bayes Analysis of a Microarray Experiment , 2001 .

[87]  J. Johannes.  DECONVOLUTION WITH UNKNOWN ERROR DISTRIBUTION , 2007, 0705.3482.

[88]  Bradley Efron,et al.  Bayes, Oracle Bayes and Empirical Bayes , 2019, Statistical Science.

[89]  T. Louis,et al.  Empirical Bayes Confidence Intervals Based on Bootstrap Samples , 1987 .

[90]  J. Torrea,et al.  Sobolev spaces associated to the harmonic oscillator , 2006, math/0608684.