Adaptive sign error control

In multiple testing scenarios, typically the sign of a parameter is inferred when its estimate exceeds some significance threshold in absolute value. Typically, the significance threshold is chosen to control the experimentwise type I error rate, family-wise type I error rate or the false discovery rate. However, controlling these error rates does not explicitly control the sign error rate. In this paper, we propose two procedures for adaptively selecting an experimentwise significance threshold in order to control the sign error rate. The first controls the sign error rate conservatively, without any distributional assumptions on the parameters of interest. The second is an empirical Bayes procedure, and achieves optimal performance asymptotically when a model for the distribution of the parameters is correctly specified. We also discuss an adaptive procedure to minimize the sign error rate when the experimentwise type I error rate is held fixed.

[1]  Art B. Owen.  Confidence intervals with control of the sign error in low power settings , 2016 .

[2]  J. Tukey.  The Future of Data Analysis , 1962 .

[3]  Y. Benjamini,et al.  Selection Adjusted Confidence Intervals With More Power to Determine the Sign , 2013 .

[4]  L. Wasserman,et al.  Weighted Hypothesis Testing , 2006, math/0604172.

[5]  Y. Benjamini,et al.  False Discovery Rate–Adjusted Multiple Confidence Intervals for Selected Parameters , 2005 .

[6]  Francis Tuerlinckx,et al.  Type S error rates for classical and Bayesian single and multiple comparison procedures , 2000, Comput. Stat..

[7]  J. Tukey.  The Philosophy of Multiple Comparisons , 1991 .

[8]  Y. Benjamini.  Discovering the false discovery rate , 2010 .

[9]  Keming Yu,et al.  A Three-Parameter Asymmetric Laplace Distribution and Its Extension , 2005 .

[10]  L. Wasserman,et al.  A stochastic process approach to false discovery control , 2004, math/0406519.

[11]  Bradley Efron,et al.  Large-scale inference , 2010 .

[12]  Wenge Guo,et al.  Controlling False Discoveries in Multidimensional Directional Decisions, with Applications to Gene Expression Data on Ordered Categories , 2010, Biometrics.

[13]  Isaac Dialsingh,et al.  Large-scale inference: empirical Bayes methods for estimation, testing, and prediction , 2012 .

[14]  R. Viertl.  On the Future of Data Analysis , 2002 .

[15]  E. Lander,et al.  Gene expression correlates of clinical prostate cancer behavior. , 2002, Cancer cell.

[16]  John D. Storey.  The optimal discovery procedure: a new approach to simultaneous significance testing , 2007 .

[17]  Edgar Dobriban,et al.  Optimal multiple testing under a Gaussian prior on the effect sizes , 2015, Biometrika.

[18]  Peter D. Hoff,et al.  Adaptive multigroup confidence intervals with constant coverage , 2016, 1612.08287.

[19]  D. Yekutieli,et al.  Selective Sign-Determining Multiple Confidence Intervals with FCR Control , 2014, Statistica Sinica.

[20]  Y. Benjamini,et al.  Controlling the false discovery rate: a practical and powerful approach to multiple testing , 1995 .

[21]  Shyamal D Peddada,et al.  Mixed directional false discovery rate control in multiple pairwise comparisons using weighted p‐values , 2015, Biometrical journal. Biometrische Zeitschrift.

[22]  John D. Storey.  A direct approach to false discovery rates , 2002 .

[23]  J. Carlin,et al.  Beyond Power Calculations , 2014, Perspectives on psychological science : a journal of the Association for Psychological Science.

[24]  Matthew Stephens,et al.  False discovery rates: a new deal , 2016, bioRxiv.