Evaluating clinical significance through equivalence testing: Extending the normative comparisons approach

Abstract The field of psychology, as with many other disciplines, has been increasingly interested in being able to measure the effectiveness of behavioral interventions. This trend has led to a number of different approaches for measuring clinical significance, each addressing a slightly different aspect of the clinical outcome. Recently, clinical psychologists (and clients) have supported the contention that one of the most important therapeutic questions is whether clients are functioning equivalently to normal controls following an intervention. To address this question, Kendall, Marrs-Garcia, Nath, and Sheldrick (1999) presented an approach to measuring clinical significance that utilizes tests of equivalence. The present study clarifies the nature of the hypotheses being conducted in measuring clinical significance with tests of equivalence and extends the approach by incorporating recent advances in equivalence testing. A revised approach for evaluating clinical significance via equivalence testing is proposed, and an empirical example demonstrating this approach is provided.

[1]  Robert A. Cribbie,et al.  The Effects of Heteroscedasticity on Tests of Equivalence , 2007 .

[2]  P. Kendall,et al.  The Clinical Significance of Treatments: A Comparison of Three Treatments for Conduct Disordered Children , 2006 .

[3]  T. Achenbach What Are Norms and Why Do We Need Valid Ones , 2006 .

[4]  R. Frost,et al.  The dimensions of perfectionism , 1990, Cognitive Therapy and Research.

[5]  Donald J. Schuirmann A comparison of the Two One-Sided Tests Procedure and the Power Approach for assessing the equivalence of average bioavailability , 1987, Journal of Pharmacokinetics and Biopharmaceutics.

[6]  S. Bauer,et al.  Clinical Significance Methods: A Comparison of Statistical Techniques , 2004, Journal of personality assessment.

[7]  E. Wise Methods for Analyzing Psychotherapy Outcomes: A Review of Clinical Significance, Reliable Change, and Recommendations for Future Directions , 2004, Journal of personality assessment.

[8]  Robert A. Cribbie,et al.  Recommendations for applying tests of equivalence. , 2004, Journal of clinical psychology.

[9]  David L Streiner,et al.  Unicorns Do Exist: A Tutorial on “Proving” the Null Hypothesis , 2003, Canadian journal of psychiatry. Revue canadienne de psychiatrie.

[10]  B. Ogles,et al.  Clinical significance: history, application, and current practice. , 2001, Clinical psychology review.

[11]  Ulrich Rendtel,et al.  Editorial , 2014, Journal of basic microbiology.

[12]  P. Kendall Clinical significance. , 2020, Journal of consulting and clinical psychology.

[13]  P. Kendall,et al.  Normative comparisons for the evaluation of clinical significance. , 1999, Journal of consulting and clinical psychology.

[14]  P. Hewitt,et al.  Psychological distress and the frequency of perfectionistic thinking. , 1998, Journal of personality and social psychology.

[15]  T. C. Oshima,et al.  Type I Error Rates for Welch’s Test and James’s Second-Order Test Under Nonnormality and Inequality of Variance When There Are Two Groups , 1994 .

[16]  J. L. Rogers,et al.  Using significance tests to evaluate equivalence between two experimental groups. , 1993, Psychological bulletin.

[17]  P. Hewitt,et al.  Perfectionism in the self and social contexts: conceptualization, assessment, and association with psychopathology. , 1991, Journal of personality and social psychology.

[18]  N. Jacobson,et al.  Clinical significance: a statistical approach to defining meaningful change in psychotherapy research. , 1991, Journal of consulting and clinical psychology.

[19]  Alan E. Kazdin,et al.  Behavior modification in applied settings, 6th ed. , 1989 .

[20]  William M. Grove,et al.  Normative comparisons in therapy outcome , 1988 .

[21]  Neil S. Jacobson,et al.  Statistics for assessing the clinical significance of psychotherapy techniques: Issues, problems, and new developments. , 1988 .

[22]  N. Jacobson,et al.  Psychotherapy outcome research: Methods for reporting variability and evaluating clinical significance , 1984 .

[23]  P. Kendall,et al.  Clinical Psychology: Scientific and Professional Dimensions , 1982 .

[24]  D. Shakow What is clinical psychology? , 1976, The American psychologist.

[25]  A. E. Kazlxn,et al.  Behavior Modification in Applied Settings , 1975 .

[26]  Paul A. Games,et al.  Robustness of the Analysis of Variance, the Welch Procedure and a Box Procedure to Heterogeneous Variances , 1974 .

[27]  K. Yuen,et al.  The two-sample trimmed t for unequal population variances , 1974 .

[28]  C. A. Boneau,et al.  The effects of violations of assumptions underlying the test. , 1960, Psychological bulletin.

[29]  Satterthwaite Fe An approximate distribution of estimates of variance components. , 1946 .

[30]  F. E. Satterthwaite An approximate distribution of estimates of variance components. , 1946, Biometrics.

[31]  B. L. Welch THE SIGNIFICANCE OF THE DIFFERENCE BETWEEN TWO MEANS WHEN THE POPULATION VARIANCES ARE UNEQUAL , 1938 .