A better stopping rule for conventional statistical tests

The goal of some research studies is to demonstrate the existence of an effect. Statistical testing, withp less than .05, is one criterion for establishing the existence of this effect. In this situation, the fixedsample stopping rule, in which the number of subjects is determined in advance, is impractical and inefficient. This article presents a sequential stopping rule that is practical and about 30% more efficient: Once a minimum number of subjects is tested, stop withp less than .01 or greater than .36; otherwise, keep testing. This procedure keeps alpha at .05 and can be adjusted to fit researchers’ needs and inclinations.

[1]  J. Andel.  Sequential Analysis , 2022, The SAGE Encyclopedia of Research Design.

[2]  David L. DeMets,et al.  An overview of sequential methods and their application in clinical trials , 1984 .

[3]  A Pollock,et al.  When to stop a clinical trial. , 1992, BMJ.

[4]  K. K. Lan,et al.  Stochastically curtailed tests in long–term clinical trials , 1982 .

[5]  Max Halperin,et al.  More flexible sequential and non-sequential designs in long-term clinical trial , 1984 .

[6]  Kenneth Mullen,et al.  First Course in Probability and Statistics , 1973 .

[7]  J. Haybittle,et al.  Repeated assessment of results in clinical trials of cancer treatment. , 1971, The British journal of radiology.

[8]  L. Weiss.  Testing one Simple Hypothesis Against Another , 1953 .

[9]  C. A. Boneau,et al.  The effects of violations of assumptions underlying the test. , 1960, Psychological bulletin.

[10]  L. Billard,et al.  A partial sequential t-test , 1991 .

[11]  S. Pocock.  Group sequential methods in the design and analysis of clinical trials , 1977 .

[12]  D. Siegmund.  Sequential Analysis: Tests and Confidence Intervals , 1985 .

[13]  Ben N. Theall.  SEQUENTIAL RELIABILITY TESTS APPLIED TO CHECK OUT EQUIPMENT , 1963 .

[14]  Peter Armitage,et al.  RESTRICTED SEQUENTIAL PROCEDURES , 1957 .

[15]  P. O'Brien,et al.  A multiple testing procedure for clinical trials. , 1979, Biometrics.

[16]  A. Wald,et al.  A Sequential Decision Procedure for Choosing One of Three Hypotheses Concerning the Unknown Mean of a Normal Distribution , 1949 .

[17]  R. Frick,et al.  The appropriate use of null hypothesis testing. , 1996 .

[18]  J. Whitehead,et al.  The double triangular test: a sequential test for the two-sided alternative with early stopping under the null hypothesis , 1990 .

[19]  Theodor D. Sterling,et al.  Publication decisions revisited: the effect of the outcome of statistical tests on the decision to p , 1995 .

[20]  C. Jennison,et al.  Group Sequential tests and Repeated Confidence Intervals , 1988 .

[21]  Changsoon Park.  An approximation method for the characteristics of the sequential probability ratio test , 1992 .

[22]  A modification of a truncated partial sequential procedure , 1982 .

[23]  R. Linn,et al.  Sequential Testing for Dichotomous Decisions , 1970 .

[24]  Donald A. Rock,et al.  The Development and Evaluation of Several Programmed Testing Methods , 1968 .

[25]  R. Frick.  Accepting the null hypothesis , 1995, Memory & cognition.

[26]  R. Doll.  Clinical trials: retrospect and prospect. , 1982, Statistics in medicine.

[27]  Michael A. Proschan,et al.  Effects of assumption violations on type I error rate in group sequential monitoring , 1992 .

[28]  M. K. Vagholkar,et al.  A Sequential Procedure for Testing a Null Hypothesis Against a Two‐Sided Alternative Hypothesis , 1969 .

[29]  S. Goodman,et al.  Evidence and scientific research. , 1988, American journal of public health.

[30]  P. Armitage,et al.  Design and analysis of randomized clinical trials requiring prolonged observation of each patient. I. Introduction and design. , 1976, British Journal of Cancer.

[31]  M. Pike,et al.  Design and analysis of randomized clinical trials requiring prolonged observation of each patient. II. analysis and examples. , 1977, British Journal of Cancer.

[32]  Eric V. Slud,et al.  Two-Sample Repeated Significance Tests Based on the Modified Wilcoxon Statistic , 1982 .

[33]  L. V. Jones,et al.  Sequential analysis in psychological research. , 1954, Psychological bulletin.