Logarithmically efficient simulation for misclassification probabilities in sequential multiple testing

We consider the problem of estimating via Monte Carlo simulation the misclassification probabilities of two sequential multiple testing procedures. The first one stops when all local test statistics exceed simultaneously either a positive or a negative threshold. The second assumes knowledge of the true number of signals, say m, and stops when the gap between the top m test statistics and the remaining ones exceeds a threshold. For each multiple testing procedure, we propose an importance sampling algorithm for the estimation of its misclassification probability. These algorithms are shown to be logarithmically efficient when the data for the various statistical hypotheses are independent, and each testing problem satisfies an asymptotic stability condition and a symmetry condition. Our theoretical results are illustrated by a simulation study in the special case of testing the drifts of Gaussian random walks.

[1]  Georgios Fellouris,et al.  Asymptotically optimal, sequential, multiple testing procedures with prior information on the number of signals , 2016, 1603.02791.

[2]  M. Basseville,et al.  Sequential Analysis: Hypothesis Testing and Changepoint Detection , 2014 .

[3]  Paul Erdös,et al.  On a Theorem of Hsu and Robbins , 1949 .

[4]  T. Lai,et al.  Efficient importance sampling for Monte Carlo evaluation of exceedance probabilities , 2007, math/0703910.

[5]  Tze Leung Lai,et al.  Importance Sampling for Generalized Likelihood Ratio Procedures in Sequential Analysis , 2005 .

[6]  J. Bartroff,et al.  Sequential Tests of Multiple Hypotheses Controlling Type I and II Familywise Error Rates. , 2013, Journal of statistical planning and inference.

[7]  D. Siegmund Importance Sampling in the Monte Carlo Study of Sequential Tests , 1976 .

[8]  R. Khan,et al.  Sequential Tests of Statistical Hypotheses. , 1972 .

[9]  Paul Glasserman,et al.  Uniformly Efficient Importance Sampling for the Tail Distribution of Sums of Random Variables , 2008, Math. Oper. Res..

[10]  Peter W. Glynn,et al.  Stochastic Simulation: Algorithms and Analysis , 2007 .

[11]  Shyamal K. De,et al.  Step-up and step-down methods for testing multiple hypotheses in sequential experiments , 2012 .

[12]  Jeffrey F. Collamore Importance Sampling Techniques for the Multidimensional Ruin Problem for General Markov Additive Sequences of Random Vectors , 2002 .

[13]  Jose Blanchet,et al.  Efficient importance sampling in ruin problems for multidimensional regularly varying random walks , 2010, Journal of Applied Probability.

[14]  Michael Baron,et al.  Sequential Bonferroni Methods for Multiple Hypothesis Testing with Strong Control of Family-Wise Error Rates I and II , 2012 .

[15]  P. Glasserman,et al.  Counterexamples in importance sampling for large deviations probabilities , 1997 .

[16]  H Robbins,et al.  Complete Convergence and the Law of Large Numbers. , 1947, Proceedings of the National Academy of Sciences of the United States of America.

[17]  Alexander G. Tartakovsky,et al.  Multichannel Sequential Detection—Part I: Non-i.i.d. Data , 2016, IEEE Transactions on Information Theory.