The Researcher's Own Therapy Allegiances: A "Wild Card" in Comparisons of Treatment Efficacy

This report examines a possible distortion in the results of comparative treatment studies due to the association of the researcher's treatment allegiances with outcomes of those treatments. In eight past reviews a trend appeared for significant associations between the researcher's allegiance and outcomes of treatments compared. In a new review of 29 studies of treatment comparisons, a similar trend appeared. Allegiance ratings were based not only on the usual reprint method, but also on two new methods: ratings by colleagues who knew the researcher well, and self-ratings by the researchers themselves. The two new allegiance methods Interco related only moderately, but each allegiance measure correlated significantly with outcomes of the treatments compared, and when combined, the three measures explained 69% of the variance in outcomes Such an association can distort comparative treatment results. Our report concludes with how the researcher's allegiance may become associated with treatment outcomes and how studies should deal with these associations.

[1]  L Luborsky,et al.  Comparative studies of psychotherapies: is it true that "everybody has won and all must have prizes"? , 1975, Proceedings of the annual meeting of the American Psychopathological Association.

[2]  R. Moleski,et al.  Comparative psychotherapy: rational-emotive therapy versus systematic desensitization in the treatment of stuttering. , 1976, Journal of consulting and clinical psychology.

[3]  D. Klein,et al.  Behavior therapy, supportive psychotherapy, imipramine, and phobias. , 1978, Archives of general psychiatry.

[4]  Donald B. Rubin,et al.  Issues in summarizing the first 345 studies of interpersonal expectancy effects , 1978, Behavioral and Brain Sciences.

[5]  P. Lewinsohn,et al.  Nonspecific improvement effects in depression using interpersonal skills training, pleasant activity schedules, or cognitive training. , 1979, Journal of Consulting and Clinical Psychology.

[6]  G. Klerman,et al.  Differential symptom reduction by drugs and psychotherapy in acute depression. , 1979, Archives of general psychiatry.

[7]  R. Rosenthal.  The file drawer problem and tolerance for null results , 1979 .

[8]  A. Hakstian,et al.  Clinical depression: comparative efficacy of outpatient treatments. , 1979, Journal of consulting and clinical psychology.

[9]  Mia Biran,et al.  Treatment of phobic disorders using cognitive and exposure methods: a self-efficacy analysis. , 1981 .

[10]  I. Blackburn,et al.  The Efficacy of Cognitive Therapy in Depression: A Treatment Trial Using Cognitive Therapy and Pharmacotherapy, each Alone and in Combination , 1981, British Journal of Psychiatry.

[11]  D. Shapiro,et al.  Meta-analysis of comparative therapy outcome studies: a replication and refinement. , 1982, Psychological bulletin.

[12]  P. Sheehan,et al.  Short- and long-term follow-up of clients receiving insight-oriented therapy and behavior therapy. , 1982, Journal of consulting and clinical psychology.

[13]  L. Thompson,et al.  Treatment of major depressive disorder in older adult outpatients with brief psychotherapies. , 1982 .

[14]  A. Beck,et al.  Psychotherapy for opiate addicts. Does it help? , 1983, Archives of general psychiatry.

[15]  A. Simons,et al.  Cognitive therapy and pharmacotherapy. Singly and together in the treatment of depression. , 1984, Archives of general psychiatry.

[16]  P. Massman,et al.  Cognitive therapy versus systematic desensitization: is one treatment superior? , 1985, Psychological bulletin.

[17]  J. Horan,et al.  Attending to Experimental Construct Validity in the Evaluation of Cognitive and Behavioral Treatments for Depression. , 1985 .

[18]  Donald B. Rubin,et al.  Meta-Analytic Procedures for Combining Studies With Multiple Effect Sizes , 1986 .

[19]  L. Thompson,et al.  Comparative effectiveness of psychotherapies for depressed elders. , 1987, Journal of consulting and clinical psychology.

[20]  K. Power,et al.  A Controlled Comparison of Cognitive—Behaviour Therapy, Diazepam and Placebo in the Management of Generalized Anxiety , 1989, Behavioural Psychotherapy.

[21]  S D Imber,et al.  National Institute of Mental Health Treatment of Depression Collaborative Research Program. General effectiveness of treatments. , 1989, Archives of general psychiatry.

[22]  D. Brom,et al.  Brief psychotherapy for posttraumatic stress disorders. , 1989, Journal of consulting and clinical psychology.

[23]  K. Dobson.  A meta-analysis of the efficacy of cognitive therapy for depression. , 1989, Journal of consulting and clinical psychology.

[24]  R. Neimeyer,et al.  Psychotherapy for the treatment of depression: a comprehensive review of controlled outcome research. , 1990, Psychological bulletin.

[25]  M. Fennell,et al.  Comparison of behavior therapy and cognitive behavior therapy in the treatment of generalized anxiety disorder. , 1991, Journal of consulting and clinical psychology.

[26]  R. Rosenthal.  Quality-weighting of studies in meta-analytic research , 1991 .

[27]  R. Rapee,et al.  Behavioral treatment of generalized anxiety disorder , 1992 .

[28]  W M Grove,et al.  Cognitive therapy and pharmacotherapy for depression. Singly and in combination. , 1992, Archives of general psychiatry.

[29]  Blind man's bluff: effectiveness and significance of psychotherapy and pharmacotherapy blinding procedures in a clinical trial. , 1994 .

[30]  Donald B. Rubin,et al.  The Counternull Value of an Effect Size: A New Statistic , 1994 .

[31]  E. Gaffan,et al.  Researcher allegiance and meta-analysis: the case of cognitive therapy for depression. , 1995, Journal of consulting and clinical psychology.

[32]  M. Thase,et al.  The National Institute on Drug Abuse Collaborative Cocaine Treatment Study. Rationale and methods. , 1997, Archives of general psychiatry.