Measures of clinical significance.

Behavioral scientists are interested in answering three basic questions when examining the relationships between variables (Kirk, 2001). First, is an observed result real or should it be attributed to chance (i.e., statistical significance)? Second, if the result is real, how large is it (i.e., effect size)? Third, is the result large enough to be meaningful and useful (i.e., clinical or practical significance)? In this last column in the series, we treat clinical significance as equivalent to practical significance. Judgments by the researcher and the consumers (e.g., clinicians and patients) regarding clinical significance consider factors such as clinical benefit, cost, and side effects. Although there is no formal statistical test of clinical significance, researchers suggest using one of three types of effect size measures to assist in interpreting clinical significance. These include the strength of association between variables (r family effect size measures), the magnitude of the difference between treatment and comparison groups (d family effect size measures), and measures of risk potency. In this paper, we review the d and r effect size measures and five measures of risk potency: odds ratio, risk ratio, relative risk reduction, risk difference, and number needed to treat. Finally, we review a relatively new effect size, AUC (which for historical reasons irrelevant to the current discussion stands for area under the receiver operating characteristic [ROC] curve), that integrates many of the others and is directly related to clinical significance. Each of these measures, however, has limitations that require the clinician to be cautious about interpretation. Guidelines are offered to facilitate the interpretation and understanding of clinical significance. Problems With Statistical Significance