Evaluation of the Power of Rerandomization Tests, with Application to Weather Modification Experiments

Abstract Computations of power of rerandomization tests by exact methods are known to be computationally exorbitant. We introduce a much cheaper naive method of evaluating power and show—both by simulation and analytically for a special case—that it very mildly overestimates true power. We further derive a normal approximation to re-randomization distributions of linear statistics and illustrate its closeness to the true distributions. This yields analytical formulas for calculating power approximately at negligible computational cost. The proposed methods therefore eliminate the previously prohibitive cost of calculating power for rerandomization tests and make it practical to evaluate the sample sizes needed for a randomized experiment to be analyzed by rerandomization (see also Gabriel and Hall 1983). Its application to weather modification experiments is illustrated.

[1]  Donald Fraser,et al.  Randomization Tests for a Multivariate Two-Sample Problem , 1958 .

[2]  The management of weather resources. , 1978, Science.

[3]  Perspectives from a Weather Modification Experiment , 1980 .

[4]  O. Kempthorne,et al.  RANDOMIZATION TESTS FOR COMPARING SURVIVAL CURVES , 1963 .

[5]  A. W. Davis.  On Certain Ratio Statistics in Weather Modification Experiments , 1979 .

[6]  Chin-Fei Hsu,et al.  Statistical Techniques and Key Issues for the Evaluation of Operational Weather Modification , 1981 .

[7]  First and Second Moments of the Randomization Test in Two-Associate PBIB Designs , 1969 .

[8]  W. R. van Zwet,et al.  Asymptotic Expansions for the Power of Distributionfree Tests in the Two-Sample Problem , 1976 .

[9]  W. Hoeffding.  The Large-Sample Power of Tests Based on Permutations of Observations , 1952 .

[10]  E. Pitman.  SIGNIFICANCE TESTS WHICH MAY BE APPLIED TO SAMPLES FROM ANY POPULATIONS III. THE ANALYSIS OF VARIANCE TEST , 1938 .

[11]  M. Dwass.  Modified Randomization Tests for Nonparametric Hypotheses , 1957 .

[12]  E. E. Adderley.  Non-Parametric Methods of Analysis Applied to Large-Scale Cloud-Seeding Experiments. , 1961 .

[13]  W. J. Hall,et al.  Rerandomization Inference on Regression and Shift Effects: Computationally Feasible Methods , 1983 .

[14]  D. E. Shaw,et al.  A Cloud-Seeding Experiment in Tasmania , 1979 .

[15]  A. Dempster.  Elements of Continuous Multivariate Analysis , 1969 .

[16]  D. E. Cain,et al.  Evaluation by Monte Carlo Tests of Effects of Cloud Seeding on Growing Season Rainfall in North Dakota , 1975 .

[17]  T. E. Doerfler,et al.  The behaviour of some significance tests under experimental randomization , 1969 .

[18]  Paul W. Mielke,et al.  Asymptotic Behavior of Two-Sample Tests Based on Powers of Ranks for Detecting Scale and Location Alternatives , 1972 .

[19]  On The Distribution of Statistics Suitable for Evaluating Rainfall Stimulation Experiments , 1969 .

[20]  K. Ruben Gabriel,et al.  Some statistical issues in weather experimentation , 1979 .