OBJECTIVES
It has been shown that human clinical trials that lack randomization (RND) or blinding (BLD) often overestimate the magnitude of treatment effects. However, no studies have evaluated the effect of RND and BLD on animal research. The authors' objectives were to determine the proportion of animal studies presented at a national academic emergency medicine meeting that utilize randomization, blinding, or both; and to determine whether failure to employ these techniques changes the likelihood of observing a difference between treatment groups.
METHODS
Two trained researchers reviewed abstracts presented at the 1997-2001 Society for Academic Emergency Medicine (SAEM) annual meetings using a standard data collection sheet. Studies that used an animal or cell line, compared two or more study groups, and measured an effect caused by the intervention or drugs were included. Studies were classified as randomized (RND+) if any part of the experiment involved random assignment of subjects to treatment groups, blinded (BLD+) if any assessment of the outcome was made by an investigator blinded to treatment group, and outcome-positive (Outcome+) if any difference between the study groups met the author's definition of significant. Following the initial review, differences in classification were resolved by consensus. The association between outcome and study methodology (RND, BLD or both) was measured using odds ratios (ORs) with 95% confidence intervals (95% CIs).
RESULTS
A total of 2,592 studies were published as abstracts. Three hundred eighty-nine were animal studies, and 290 of these studies had two or more study groups. RND- and BLD- studies were more likely to be Outcome+ than RND+ or BLD+ studies (OR = 3.4; 95% CI = 1.7 to 6.9 and OR = 3.2; 95% CI = 1.3 to 7.7, respectively). When studies that used both RND and BND were compared with studies that used neither, the OR for a positive study was 5.2 (95% CI = 2.0 to 13.5).
CONCLUSIONS
These results suggest that animal studies that do not utilize RND and BLD are more likely to report a difference between study groups than studies that employ these methods.
[1]
K. Schulz,et al.
Subverting randomization in controlled trials.
,
1995,
JAMA.
[2]
R. J. Hayes,et al.
Empirical evidence of bias. Dimensions of methodological quality associated with estimates of treatment effects in controlled trials.
,
1995,
JAMA.
[3]
Gordon H. Guyatt,et al.
Users' Guides to the Medical Literature: II. How to Use an Article About Therapy or Prevention B. What Were the Results and Will They Help Me in Caring for My Patients?
,
1994
.
[4]
T C Chalmers,et al.
Randomized versus historical controls for clinical trials.
,
1982,
The American journal of medicine.
[5]
F. Mosteller,et al.
How study design affects outcomes in comparisons of therapy. I: Medical.
,
1989,
Statistics in medicine.
[6]
G H Guyatt,et al.
Users' guides to the medical literature. II. How to use an article about therapy or prevention. B. What were the results and will they help me in caring for my patients? Evidence-Based Medicine Working Group.
,
1994,
JAMA.
[7]
D. Moher,et al.
The Revised CONSORT Statement for Reporting Randomized Trials: Explanation and Elaboration
,
2001,
Annals of Internal Medicine.
[8]
Jesse A Berlin,et al.
Does blinding of readers affect the results of meta-analyses?
,
1997,
The Lancet.
[9]
J. Ioannidis,et al.
Comparison of evidence of treatment effects in randomized and nonrandomized studies.
,
2001,
JAMA.