The Fallback Procedure for Evaluating a Single Family of Hypotheses

ABSTRACT In testing multiple hypotheses, control of the familywise error rate is often considered. We develop a procedure called the “fallback procedure” to control the familywise error rate when multiple primary hypotheses are tested. With the fallback procedure, the Type I error rate (α) is partitioned among the various hypotheses of interest. Unlike the standard Bonferroni adjustment, however, testing hypotheses proceeds in an order determined a priori. As long as hypotheses are rejected, the Type I error rate can be accumulated, making tests of later hypotheses more powerful than under the Bonferroni procedure. Unlike the fixed sequence test, the fallback test allows consideration of all hypotheses even if one or more hypotheses are not rejected early in the process, thereby avoiding a common concern about the fixed sequence procedure. We develop properties of the fallback procedure, including control of the familywise error rate for an arbitrary number of hypotheses via illustrating the procedure as a closed testing procedure, as well as making the test more powerful via alpha exhaustion. We compare it to other procedures for controlling familywise error rates, finding that the fallback procedure is a viable alternative to the fixed sequence procedure when there is some doubt about the power for the first hypothesis. These results expand on the previously developed properties of the fallback procedure (Wiens, 2003). Several examples are discussed to illustrate the relative advantages of the fallback procedure.