Quantitative Methods Inquires 1 COMPARING DISTRIBUTIONS : THE TWO-SAMPLE ANDERSON-DARLING TEST AS AN ALTERNATIVE TO THE KOLMOGOROV-SMIRNOFF TEST

This paper introduces the two-sample Anderson-Darling (AD) test of goodness of fit as a tool for comparing distributions, response time distributions in particular. We discuss the problematic use of pooling response times across participants, and alternative tests of distributions, the most common being the Kolmogorov-Smirnoff (KS) test. We compare the KS test and the AD test, presenting conclusive evidence that the AD test is more powerful: when comparing two distributions that vary (1) in shift only, (2) in scale only, (3) in symmetry only, or (4) that have the same mean and standard deviation but differ on the tail ends only, the AD test proves to detect differences better than the KS test. In addition, the AD test has a type I error rate corresponding to alpha whereas the KS test is overly conservative. Finally, the AD test requires less data than the KS test to reach sufficient statistical power.

[1]  S. B. Vincent.  The function of the vibrissae in the behavior of the white rat , 1912 .

[2]  O. William.  Journal Of The American Statistical Association V-28 , 1932 .

[3]  AN Kolmogorov-Smirnov,et al.  Sulla determinazione empírica di uma legge di distribuzione , 1933 .

[4]  A. Kolmogoroff.  Confidence Limits for an Unknown Distribution Function , 1941 .

[5]  F. Massey.  The Kolmogorov-Smirnov Test for Goodness of Fit , 1951 .

[6]  T. W. Anderson,et al.  Asymptotic Theory of Certain "Goodness of Fit" Criteria Based on Stochastic Processes , 1952 .

[7]  H. Chernoff,et al.  The Use of Maximum Likelihood Estimates in {\chi^2} Tests for Goodness of Fit , 1954 .

[8]  D. Darling,et al.  A Test of Goodness of Fit , 1954 .

[9]  D. Darling.  The Kolmogorov-Smirnov, Cramer-von Mises Tests , 1957 .

[10]  E. Gumbel,et al.  Statistics of extremes , 1960 .

[11]  H. Lilliefors.  On the Kolmogorov-Smirnov Test for Normality with Mean and Variance Unknown , 1967 .

[12]  H. Lilliefors.  On the Kolmogorov-Smirnov Test for the Exponential Distribution with Mean Unknown , 1969 .

[13]  M. Stephens.  EDF Statistics for Goodness of Fit and Some Comparisons , 1974 .

[14]  A. Pettitt.  A two-sample Anderson-Darling rank statistic , 1976 .

[15]  J. D. T. Oliveira,et al.  The Asymptotic Theory of Extreme Order Statistics , 1979 .

[16]  Brian H. Ross,et al.  On appropriate procedures for combining probability distributions within the same family , 1980 .

[17]  Jeff Miller,et al.  Divided attention: Evidence for coactivation with redundant signals , 1982, Cognitive Psychology.

[18]  M. Stephens,et al.  K-Sample Anderson–Darling Tests , 1987 .

[19]  S. Yantis,et al.  An interactive race model of divided attention. , 1991, Journal of experimental psychology. Human perception and performance.

[20]  Jacob Cohen,et al.  A power primer. , 1992, Psychological bulletin.

[21]  G. Logan.  Shapes of reaction-time distributions and shapes of learning curves: a test of the instance theory of automaticity. , 1992, Journal of experimental psychology. Learning, memory, and cognition.

[22]  S Yantis,et al.  Dividing attention between color and shape: Evidence of coactivation , 1993, Perception & psychophysics.

[23]  Catherine C. Eckel,et al.  Are Women Less Selfish Than Men?: Evidence From Dictator Experiments , 1998 .

[24]  H. Vidrio,et al.  Evaluating Knowledge Retention of Third‐year Medical Students Taught with an Innovative Pharmacology Program , 2002, Academic medicine : journal of the Association of American Medical Colleges.

[25]  Denis Cousineau,et al.  QMPE: Estimating Lognormal, Wald, and Weibull RT distributions with a parameter-dependent lower bound , 2004, Behavior research methods, instruments, & computers : a journal of the Psychonomic Society, Inc.

[26]  Jeffrey N. Rouder,et al.  An evaluation of the Vincentizing method of forming group-level response time distributions , 2004, Psychonomic bulletin & review.

[27]  Denis Cousineau,et al.  Fitting distributions using maximum likelihood: Methods and packages , 2004, Behavior research methods, instruments, & computers : a journal of the Psychonomic Society, Inc.

[28]  A. Pandit.  Circadian Rhythm Variation in Attempted Suicide by Deliberate Self-Poisoning, and in Completed Suicide, in Central Nepal , 2004 .

[29]  J. D. den Boer,et al.  Platelet serotonin levels in pervasive developmental disorders and mental retardation: diagnostic group differences, within-group distribution, and behavioral correlates. , 2004, Journal of the American Academy of Child and Adolescent Psychiatry.

[30]  Jeffrey N. Rouder,et al.  A hierarchical model for estimating response time distributions , 2005, Psychonomic bulletin & review.

[31]  Robert V. Kenyon,et al.  Influences of the Perception of Self-Motion on Postural Parameters , 2006, Cyberpsychology Behav. Soc. Netw..

[32]  D. Cousineau.  Implementing and evaluating the nested maximum likelihood estimation technique , 2007 .

[33]  Denis Cousineau,et al.  How to use MATLAB to fit the ex-Gaussian and other probability functions to a distribution of response times , 2008 .