Connections between permutation and t‐tests: relevance to adaptive methods

A permutation test assigns a p-value by conditioning on the data and treating the different possible treatment assignments as random. The fact that the conditional type I error rate given the data is controlled at level α ensures validity of the test even if certain adaptations are made. We show the connection between permutation and t-tests, and use this connection to explain why certain adaptations are valid in a t-test setting as well. We illustrate this with an example of blinded sample size recalculation.

[1]  J. S. Hunter,et al.  Statistics for Experimenters: An Introduction to Design, Data Analysis, and Model Building. , 1979 .

[2]  G. E. Noether,et al.  On a Theorem by Wald and Wolfowitz , 1949 .

[3]  A. V. D. Vaart,et al.  Asymptotic Statistics: Frontmatter , 1998 .

[4]  Arjun K. Gupta.  The Theory of Linear Models and Multivariate Analysis , 1981 .

[5]  Meinhard Kieser,et al.  Simple procedures for blinded sample size adjustment that do not affect the type I error rate , 2003, Statistics in medicine.

[6]  A. Gould,et al.  Sample size re-estimation without unblinding for normally distributed outcomes with unknown variance , 1992 .

[7]  W. Hoeffding.  A Combinatorial Central Limit Theorem , 1951 .

[8]  D Edwards,et al.  On model prespecification in confirmatory randomized studies. , 1999, Statistics in medicine.

[9]  E. Glimm,et al.  Some Notes on Blinded Sample Size Re-Estimation , 2013, 1301.4167.

[10]  W. Lehmacher,et al.  Adaptive Sample Size Calculations in Group Sequential Trials , 1999, Biometrics.

[11]  Meinhard Kieser,et al.  Blinded sample size reassessment in non‐inferiority and equivalence trials , 2003, Statistics in medicine.

[12]  P. Spreij.  Probability and Measure , 1996 .

[13]  M. Proschan,et al.  Internal pilot studies I: type I error rate of the naive t-test. , 1999, Statistics in medicine.

[14]  M. Proschan,et al.  Unplanned adaptations before breaking the blind , 2012, Statistics in medicine.

[15]  J. Wolfowitz,et al.  Statistical Tests Based on Permutations of the Observations , 1944 .

[16]  P. Bauer,et al.  Evaluation of experiments with adaptive interim analyses. , 1994, Biometrics.

[17]  Øyvind Langsrud,et al.  Rotation tests , 2005, Stat. Comput..

[18]  D. Zucker,et al.  Internal pilot studies II: comparison of various procedures. , 1999, Statistics in medicine.

[19]  D. Basu,et al.  On Statistics Independent of a Complete Sufficient Statistic , 2011 .

[20]  T. Friede,et al.  Blinded sample size re‐estimation in crossover bioequivalence trials , 2014, Pharmaceutical statistics.

[21]  Thomas R. Fleming,et al.  A paired prentice-Wilcoxon test for censored paired data , 1987 .

[22]  J. Matthews,et al.  Randomization in Clinical Trials: Theory and Practice; , 2003 .

[23]  K. Fang,et al.  Generalized Multivariate Analysis , 1990 .