Asymptotically optimal, sequential, multiple testing procedures with prior information on the number of signals

Assuming that data are collected sequentially from independent streams, we consider the simultaneous testing of multiple binary hypotheses under two general setups; when the number of signals (correct alternatives) is known in advance, and when we only have a lower and an upper bound for it. In each of these setups, we propose feasible procedures that control, without any distributional assumptions, the familywise error probabilities of both type I and type II below given, user-specified levels. Then, in the case of i.i.d. observations in each stream, we show that the proposed procedures achieve the optimal expected sample size, under every possible signal configuration, asymptotically as the two error probabilities vanish at arbitrary rates. A simulation study is presented in a completely symmetric case and supports insights obtained from our asymptotic results, such as the fact that knowledge of the exact number of signals roughly halves the expected number of observations compared to the case of no prior information.

[1]  A. Wald,et al.  A Sequential Decision Procedure for Choosing One of Three Hypotheses Concerning the Unknown Mean of a Normal Distribution , 1949 .

[2]  R. Khan,et al.  Sequential Tests of Statistical Hypotheses. , 1972 .

[3]  Jay Bartroff,et al.  Multistage Tests of Multiple Hypotheses , 2010, 1107.1919.

[4]  K. Gabriel,et al.  On closed testing procedures with special reference to ordered analysis of variance , 1976 .

[5]  Joseph P. Romano,et al.  Generalizations of the familywise error rate , 2005, math/0507420.

[6]  S. Holm A Simple Sequentially Rejective Multiple Test Procedure , 1979 .

[7]  Michael Baron,et al.  Sequential Bonferroni Methods for Multiple Hypothesis Testing with Strong Control of Family-Wise Error Rates I and II , 2012 .

[8]  Alexander G. Tartakovsky,et al.  Asymptotic Optimality of Certain Multihypothesis Sequential Tests: Non‐i.i.d. Case , 1998 .

[9]  P. Armitage Sequential Analysis with More than Two Alternative Hypotheses, and its Relation to Discriminant Function Analysis , 1950 .

[10]  M. Basseville,et al.  Sequential Analysis: Hypothesis Testing and Changepoint Detection , 2014 .

[11]  Georgios Fellouris,et al.  Logarithmically efficient simulation for misclassification probabilities in sequential multiple testing , 2016, 2016 Winter Simulation Conference (WSC).

[12]  Yajun Mei,et al.  Asymptotic Optimality Theory for Decentralized Sequential Multihypothesis Testing Problems , 2011, IEEE Transactions on Information Theory.

[13]  Yajun Mei Asymptotic Optimality Theory for Decentralized Sequential Hypothesis Testing in Sensor Networks , 2008, IEEE Transactions on Information Theory.

[14]  G. Hommel A stagewise rejective multiple test procedure based on a modified Bonferroni test , 1988 .

[15]  J. Bartroff,et al.  Sequential tests of multiple hypotheses controlling false discovery and nondiscovery rates , 2013, Sequential analysis.

[16]  Y. Benjamini,et al.  Controlling the false discovery rate: a practical and powerful approach to multiple testing , 1995 .

[17]  Shyamal K. De,et al.  Step-up and step-down methods for testing multiple hypotheses in sequential experiments , 2012 .

[18]  G. Lorden Nearly-optimal sequential tests for finitely many parameter values , 1977 .

[19]  Venugopal V. Veeravalli,et al.  Multihypothesis sequential probability ratio tests - Part II: Accurate asymptotic expansions for the expected sample size , 2000, IEEE Trans. Inf. Theory.

[20]  Jay Bartroff,et al.  Generalized Likelihood Ratio Statistics and Uncertainty Adjustments in Efficient Adaptive Design of Clinical Trials , 2008, 1105.4667.

[21]  Sirin Nitinawarat,et al.  Universal sequential outlier hypothesis testing , 2014, 2014 IEEE International Symposium on Information Theory.

[22]  J. Bartroff,et al.  Sequential Tests of Multiple Hypotheses Controlling Type I and II Familywise Error Rates. , 2013, Journal of statistical planning and inference.

[23]  Venugopal V. Veeravalli,et al.  Multihypothesis sequential probability ratio tests - Part I: Asymptotic optimality , 1999, IEEE Trans. Inf. Theory.