The impact of distributional shape on the power of randomization tests for two independent groups: a simulation study using small balanced samples

SUMMARY The importance of randomization tests is very well known in experimental research, particularly in biometry. The aim of the present re search is to evaluate the impact of distributional shape on the power of the randomizat ion test for difference between the means of two independent groups (with n1=n2=16). To manipulate shape in terms of asymmetry and kurtosis, we used g-and-h distributions. We evaluated the power of the randomization test, and also the power of the Stude nt-t test, as a comparison standard, with data simulated from 12 g -and-h distributions for seven values of effect size. For each condition, we generated 20 000 samples, and fo r each one the power of randomization tests was estimated using 1000 permutations. We set the value of Type I error probability at 0.05. The results show gains i n power for both tests with increasing skewness and/or kurtosis, with a slight advantage f or the randomization tests over the Student-t test.

[1]  David C. Hoaglin,et al.  Summarizing Shape Numerically: The g‐and‐h Distributions , 2011 .

[2]  Eugene S. Edgington,et al.  Randomization Tests , 2011, International Encyclopedia of Statistical Science.

[3]  Patrick Dattalo,et al.  Statistical Power Analysis , 2008 .

[4]  Todd C. Headrick,et al.  Parametric Probability Densities and Distribution Functions for Tukey g -and- h Transformations and their Use for Fitting Data , 2008 .

[5]  David C. Hoaglin,et al.  g -and-h-Distributions† , 2006 .

[6]  Marc G. Genton,et al.  The Multivariate g-and-h Distribution , 2006, Technometrics.

[7]  Kabir K. Dutta,et al.  Financial Institutions Center On Measuring Skewness and Kurtosis in Short Rate Distributions : The Case of the US Dollar London Inter Bank Offer Rates , 2002 .

[8]  E. Edgington,et al.  Randomization Tests (3rd ed.) , 1998 .

[9]  T. Mills.  Modelling Skewness and Kurtosis in the London Stock Exchange Ft‐Se Index Return Distributions , 1995 .

[10]  S. S. Young,et al.  Resampling-Based Multiple Testing: Examples and Methods for p-Value Adjustment , 1993 .

[11]  Sangit Chatterjee,et al.  A Data-Analytic Look at Skewness and Elongation in Common-Stock-Return Distributions , 1991 .

[12]  P. Lachenbruch.  Statistical Power Analysis for the Behavioral Sciences (2nd ed.) , 1989 .

[13]  L. A. Marascuilo,et al.  Statistical Methods for the Social and Behavioral Sciences. , 1989 .

[14]  W. Hays.  Statistics, 4th ed. , 1988 .

[15]  Sangit Chatterjee,et al.  On Measuring Skewness and Elongation in Common Stock Return Distributions: The Case of the Market Index , 1988 .

[16]  Martinez Jorge,et al.  Some properties of the tukey g and h family of distributions , 1984 .

[17]  Franklin A. Graybill,et al.  Introduction to the Theory of Statistics, 3rd ed. , 1974 .

[18]  E. Edgington.  Approximate Randomization Tests , 1969 .

[19]  Eugene S. Edgington,et al.  Statistical inference : the distribution-free approach , 1969 .

[20]  E. Edgington,et al.  Statistical inference and nonrandom samples. , 1966, Psychological bulletin.

[21]  Jacob Cohen,et al.  The statistical power of abnormal-social psychological research: a review. , 1962, Journal of abnormal and social psychology.

[22]  Donald Fraser,et al.  Randomization Tests for a Multivariate Two-Sample Problem , 1958 .

[23]  M. Dwass.  Modified Randomization Tests for Nonparametric Hypotheses , 1957 .

[24]  Oscar Kempthorne,et al.  THE RANDOMIZATION THEORY OF' EXPERIMENTAL INFERENCE* , 1955 .

[25]  E. Pitman.  SIGNIFICANCE TESTS WHICH MAY BE APPLIED TO SAMPLES FROM ANY POPULATIONS III. THE ANALYSIS OF VARIANCE TEST , 1938 .

[26]  E. J. G. Pitman,et al.  Significance Tests Which May be Applied to Samples from Any Populations. II. The Correlation Coefficient Test , 1937 .

[27]  J. I.  The Design of Experiments , 1936, Nature.

[28]  E. S. Pearson,et al.  On the Problem of the Most Efficient Tests of Statistical Hypotheses , 1933 .

[29]  E. S. Pearson,et al.  ON THE USE AND INTERPRETATION OF CERTAIN TEST CRITERIA FOR PURPOSES OF STATISTICAL INFERENCE PART I , 1928 .