WHAT SHOULD BE INCLUDED IN META-ANALYSES?

Objective: To explore the impact of methodologic issues on the results of meta-analyses. The following issues were examined: the type of literature search strategy used; inclusion or exclusion of non–peer-reviewed studies; the inclusion or exclusion of non-English language publications; the effect of trial quality; and the inclusion or exclusion of non–placebo-controlled studies. Methods: The International Study of Perioperative Transfusion (ISPOT) meta-analyses were used to evaluate each of the methodologic issues. The 10 meta-analyses consisted of technologies to reduce the need for perioperative red blood cell transfusion. The number of trials for each of the meta-analyses varied from 2 to 45. Both EMBASE and MEDLINE searches were conducted, including the use of systematic search strategies. Results: MEDLINE identified the vast majority of trials. Alone, MEDLINE would have missed 8 studies compared to 10 for EMBASE. Use of the systematic search strategies greatly reduced the number of articles to be reviewed compared to open searches. Type of publication, country of study origin, inclusion of non-English publications, and trial quality had very little impact on the estimates of effect. The use of placebo versus open-label control affected the magnitude of the odds ratio for two of the meta-analyses. The results of the two meta-analyses were not statistically significant if only placebo-controlled trials were included. Conclusions: While methodologic issues had very little impact on the ISPOT meta-analyses, further studies are needed in a variety of other clinical settings. Because MEDLINE, coupled with a review of the references in the identified trials, identified the vast majority of trials, one needs to consider the costs and benefits of searching EMBASE and the pursuance of unpublished and unindexed trials.

[1]  D Menon,et al.  COMPARISON OF LITERATURE SEARCHES ON QUALITY AND COSTS FOR HEALTH TECHNOLOGY ASSESSMENT USING THE MEDLINE AND EMBASE DATABASES , 1999, International Journal of Technology Assessment in Health Care.

[2]  Laupacis,et al.  Erythropoietin to minimize perioperative blood transfusion: a systematic review of randomized trials , 1998, Transfusion medicine.

[3]  D. Cook,et al.  Does quality of reports of randomised trials affect estimates of intervention efficacy reported in meta-analyses? , 1998, The Lancet.

[4]  A. Laupacis,et al.  Preoperative autologous donation decreases allogeneic transfusion but increases exposure to all red blood cell transfusion: results of a meta-analysis. International Study of Perioperative Transfusion (ISPOT) Investigators. , 1998, Archives of internal medicine.

[5]  A. Laupacis,et al.  Does Acute Normovolemic Hemodilution Reduce Perioperative Allogeneic Transfusion? A Meta-Analysis , 1998, Anesthesia and analgesia.

[6]  F. Carballo,et al.  Prevalence and associated factors for gallstone disease: results of a population survey in Spain. , 1997, Journal of clinical epidemiology.

[7]  R. Simes,et al.  Publication bias: evidence of delayed publication in a cohort study of clinical research projects , 1997, BMJ.

[8]  G. Grégoire,et al.  Discrepancies between meta-analyses and subsequent large randomized, controlled trials. , 1997, The New England journal of medicine.

[9]  C. Lengeler,et al.  Language bias in randomised controlled trials published in English and German , 1997, The Lancet.

[10]  A. Jadad,et al.  The importance of quality of primary studies in producing unbiased systematic reviews. , 1996, Archives of internal medicine.

[11]  D. Moher,et al.  Completeness of reporting of trials published in languages other than English: implications for conduct and reporting of systematic reviews , 1996, The Lancet.

[12]  A R Jadad,et al.  Assessing the quality of reports of randomized clinical trials: is blinding necessary? , 1996, Controlled clinical trials.

[13]  M. Clarke,et al.  Identifying relevant studies for systematic reviews , 1995, BMJ.

[14]  R. J. Hayes,et al.  Empirical evidence of bias. Dimensions of methodological quality associated with estimates of treatment effects in controlled trials. , 1995, JAMA.

[15]  G. Grégoire,et al.  Selecting the language of the publications included in a meta-analysis: is there a Tower of Babel bias? , 1995, Journal of clinical epidemiology.

[16]  R. Brian Haynes,et al.  Developing optimal search strategies for detecting clinically sound studies in MEDLINE. , 1994, Journal of the American Medical Informatics Association : JAMIA.

[17]  I Olkin,et al.  Re: "A critical look at some popular meta-analytic methods". , 1994, American journal of epidemiology.

[18]  K. Dickersin,et al.  Full publication of results initially presented in abstracts. A meta-analysis. , 1994 .

[19]  K. Dickersin,et al.  Identifying relevant studies for systematic reviews. , 1994, BMJ.

[20]  Performance of online biomedical databases in rheumatology. , 1994, The Journal of rheumatology.

[21]  K. Dickersin,et al.  Publication Bias: The Problem That Won't Go Away , 1993, Annals of the New York Academy of Sciences.

[22]  W. McIlroy,et al.  Should unpublished data be included in meta-analyses? Current convictions and controversies. , 1993, JAMA.

[23]  Crossover Comparison of Drug Information Online Database Vendors: Dialog and Medlars , 1993, The Annals of pharmacotherapy.

[24]  T C Chalmers,et al.  Getting to grips with Archie Cochrane's agenda. , 1992, BMJ.

[25]  K. Dickersin,et al.  Factors influencing publication of research results. Follow-up of applications submitted to two institutional review boards. , 1992, JAMA.

[26]  J. Fleiss,et al.  Meta-analysis in epidemiology, with special reference to studies of the association between exposure to environmental tobacco smoke and lung cancer: a critique. , 1991, Journal of clinical epidemiology.

[27]  J D Emerson,et al.  An empirical study of the possible relation of treatment differences to quality scores in controlled randomized clinical trials. , 1990, Controlled clinical trials.

[28]  K. Dickersin.  The existence of publication bias and risk factors for its occurrence. , 1990, JAMA.

[29]  K. Dickersin,et al.  Publication bias and clinical trials. , 1987, Controlled clinical trials.

[30]  T C Chalmers,et al.  Meta-analysis of clinical trials as a scientific discipline. II: Replicate variability and comparison of studies that agree and disagree. , 1987, Statistics in medicine.

[31]  R. Simes,et al.  Confronting publication bias: a cohort design for meta-analysis. , 1987, Statistics in medicine.

[32]  T C Chalmers,et al.  Bias in treatment assignment in controlled clinical trials. , 1983, The New England journal of medicine.