Misuse of statistical methods in arthritis and rheumatism

Articles published in Arthritis and Rheumatism in 1982 were compared with those from 1967–68 to evaluate trends in statistical methods and in the quantity and character of statistical misuse. Results show that among articles in 1982 using statistics, 66% contained methodologic errors. The percentage was similar in 1967–68, although fewer articles used statistics. In 1967–68 the most common error was failure to identify the statistical method used, whereas in 1982 multiple testing errors predominated, specifically, use of the t-test to compare 3 or more groups and comparison of 2 groups on multiple variables. The availability of calculators and computers which readily perform complex data analysis may underly the emergence of multiple testing errors. To compare multiple groups, we suggest using analysis of variance instead of the t-test. To avoid multiple testing errors, we recommend limiting the number of tests performed, lowering the alpha (significance) level, or using multivariate techniques.