Optimal Rates and Tradeoffs in Multiple Testing

Multiple hypothesis testing is a central topic in statistics, but despite abundant work on the false discovery rate (FDR) and the corresponding Type-II error concept known as the false non-discovery rate (FNR), a fine-grained understanding of the fundamental limits of multiple testing has not been developed. Our main contribution is to derive a precise non-asymptotic tradeoff between FNR and FDR for a variant of the generalized Gaussian sequence model. Our analysis is flexible enough to permit analyses of settings where the problem parameters vary with the number of hypotheses $n$, including various sparse and dense regimes (with $o(n)$ and $\mathcal{O}(n)$ signals). Moreover, we prove that the Benjamini-Hochberg algorithm as well as the Barber-Candes algorithm are both rate-optimal up to constants across these regimes.

[1]  Y. Benjamini,et al.  Controlling the false discovery rate: a practical and powerful approach to multiple testing , 1995 .

[2]  D. Donoho,et al.  Asymptotic Minimaxity Of False Discovery Rate Thresholding For Sparse Exponential Data , 2006, math/0602311.

[3]  Y. Benjamini,et al.  Multiple Hypotheses Testing with Weights , 1997 .

[4]  Y. Benjamini,et al.  THE CONTROL OF THE FALSE DISCOVERY RATE IN MULTIPLE TESTING UNDER DEPENDENCY , 2001 .

[5]  John D. Storey A direct approach to false discovery rates , 2002 .

[6]  L. Wasserman,et al.  Operating characteristics and extensions of the false discovery rate procedure , 2002 .

[7]  John D. Storey,et al.  Strong control, conservative point estimation and simultaneous conservative consistency of false discovery rates: a unified approach , 2004 .

[8]  D. Donoho,et al.  Higher criticism for detecting sparse heterogeneous mixtures , 2004, math/0410072.

[9]  I. Johnstone,et al.  Adapting to unknown sparsity by controlling the false discovery rate , 2005, math/0505374.

[10]  L. Wasserman,et al.  False discovery control with p-value weighting , 2006 .

[11]  Zhiyi Chi On the performance of FDR control: Constraints and a partial solution , 2007, 0710.3287.

[12]  G. Blanchard,et al.  Two simple sufficient conditions for FDR control , 2008, 0802.1406.

[13]  P. Hall,et al.  PROPERTIES OF HIGHER CRITICISM UNDER STRONG DEPENDENCE , 2008, 0803.2095.

[14]  Dean P. Foster,et al.  α‐investing: a procedure for sequential control of expected false discoveries , 2008 .

[15]  Harrison H. Zhou,et al.  False Discovery Rate Control With Groups , 2010, Journal of the American Statistical Association.

[16]  P. Hall,et al.  Innovated Higher Criticism for Detecting Sparse Signals in Correlated Noise , 2009, 0902.3837.

[17]  J. Ghosh,et al.  Asymptotic Bayes-optimality under sparsity of some multiple testing procedures , 2010, 1002.3501.

[18]  Jiashun Jin,et al.  UPS delivers optimal phase diagram in high-dimensional variable selection , 2010, 1010.5028.

[19]  Étienne Roquain,et al.  On false discovery rate thresholding for classification under sparsity , 2011, 1106.6147.

[20]  Jiashun Jin,et al.  Higher Criticism for Large-Scale Inference: especially for Rare and Weak effects , 2014, 1410.4743.

[21]  Jiashun Jin,et al.  Rare and Weak effects in Large-Scale Inference: methods and phase diagrams , 2014, 1410.4578.

[22]  Zhigen Zhao,et al.  Rate optimal multiple testing procedure in high-dimensional regression , 2014, 1404.2961.

[23]  Aaditya Ramdas,et al.  The p-filter: multi-layer FDR control for grouped hypotheses , 2015, 1512.03397.

[24]  E. Candès,et al.  Controlling the false discovery rate via knockoffs , 2014, 1404.5609.

[25]  Adel Javanmard,et al.  On Online Control of False Discovery Rate , 2015, ArXiv.

[26]  E. Arias-Castro,et al.  Distribution-free Multiple Testing , 2016, 1604.07520.

[27]  Ang Li,et al.  Multiple testing with the structure‐adaptive Benjamini–Hochberg algorithm , 2016, Journal of the Royal Statistical Society: Series B (Statistical Methodology).

[28]  Michael I. Jordan,et al.  A unified treatment of multiple testing with prior knowledge using the p-filter , 2017, The Annals of Statistics.