Capturing the severity of type II errors in high-dimensional multiple testing

The severity of type II errors is frequently ignored when deriving a multiple testing procedure, even though utilizing it properly can greatly help in making correct decisions. This paper puts forward a theory behind developing a multiple testing procedure that can incorporate the type II error severity and is optimal in the sense of minimizing a measure of false non-discoveries among all procedures controlling a measure of false discoveries. The theory is developed under a general model allowing arbitrary dependence by taking a compound decision theoretic approach to multiple testing with a loss function incorporating the type II error severity. We present this optimal procedure in its oracle form and offer numerical evidence of its superior performance over relevant competitors.

[1]  R. A. Waller,et al.  A Bayes Rule for the Symmetric Multiple Comparisons Problem , 1969 .

[2]  Wenguang Sun,et al.  Large‐scale multiple testing under dependence , 2009 .

[3]  James G. Scott,et al.  An exploration of aspects of Bayesian multiple testing , 2006 .

[4]  Y. Benjamini,et al.  Multiple Hypotheses Testing with Weights , 1997 .

[5]  L. Wasserman,et al.  Operating characteristics and extensions of the false discovery rate procedure , 2002 .

[6]  Edsel A. Peña,et al.  POWER-ENHANCED MULTIPLE DECISION FUNCTIONS CONTROLLING FAMILY-WISE ERROR AND FALSE DISCOVERY RATES. , 2009, Annals of statistics.

[7]  Wenguang Sun,et al.  Oracle and Adaptive Compound Decision Rules for False Discovery Rate Control , 2007 .

[8]  L. Wasserman,et al.  False discovery control with p-value weighting , 2006 .

[9]  Isaac Dialsingh,et al.  Large-scale inference: empirical Bayes methods for estimation, testing, and prediction , 2012 .

[10]  Y. Benjamini,et al.  Controlling the false discovery rate: a practical and powerful approach to multiple testing , 1995 .

[11]  Debashis Ghosh,et al.  A GENERAL DECISION THEORETIC FORMULATION OF PROCEDURES CONTROLLING FDR AND FNR FROM A BAYESIAN PERSPECTIVE , 2008 .

[12]  D. B. Duncan,et al.  A Bayesian Approach to Multiple Comparisons , 1965 .

[13]  S. Karlin,et al.  Classes of orderings of measures and related correlation inequalities. I. Multivariate totally positive distributions , 1980 .

[14]  M. Newton.  Large-Scale Simultaneous Hypothesis Testing: The Choice of a Null Hypothesis , 2008 .