Methodological Commentary The Precision of Reliability and Validity Estimates Re-Visited: Distinguishing Between Clinical and Statistical Significance of Sample Size Requirements

In a previous JCEN Methodological Commentary (Cicchetti, 1999), I proposed very speci®c and cogent arguments to question the clinical meaningfulness of Charter's (1999) recommendations to the wider community of clinical and experimental neuropsychologists of a minimum of 400 subjects for determining precise split-half, coef®cient alpha, test±retest, alternate forms, and inter-examiner reliability assessment procedures, and validity coef®cients. To refresh the reader's memory, and using Charter's own example, I concluded unabashedly that to increase sample size N from 50 to 300 (a factor of 600%) was simply not worth the considerable added cost and time, to `increase' a lower-bound precision reliability estimate from .82 to .87, while simultaneously `decreasing' the upper bound reliability estimate from .94 to .92. I stand ®rmly behind that statement, and would add to it that Charter's further conclusion of an N, in fact of `400 or more' in both his earlier and current Methodological Commentary strains credulity even more. Unfortunately, the content of Charter's reply, or rebuttal to my comments uses precisely the same arguments as previously, and the interested reader is referred, once again to my earlier critique of his work (Cicchetti, 1999). This critique will stress the inappropriateness of considering precision solely in the context of increasing N, or using sample sizes of 400 and more, as appears to be Charter's main objective or desideratum. This will be discussed in the broader context of both the necessity to consider the practical or clinical meaningfulness of precision estimates, and the underlying rationale for calculating con®dence intervals (CIs) around these estimates, in the ®rst place. Other less critical issues will also be raised, as required.