Cochrane reviews compared with industry supported meta-analyses and other meta-analyses of the same drugs: systematic review

Abstract Objective To compare the methodological quality and conclusions in Cochrane reviews with those in industry supported meta-analyses and other meta-analyses of the same drugs. Design Systematic review comparing pairs of meta-analyses that studied the same two drugs in the same disease and were published within two years of each other. Data sources Cochrane Database of Systematic Reviews (2003, issue 1), PubMed, and Embase. Data extraction Two observers independently extracted data and used a validated scale to judge the methodological quality of the reviews. Results 175 of 1596 Cochrane reviews had a meta-analysis that compared two drugs. Twenty four meta-analyses that matched the Cochrane reviews were found: eight were industry supported, nine had undeclared support, and seven had no support or were supported by non-industry sources. On a 0-7 scale, the median quality score was 7 for Cochrane reviews and 3 for other reviews (P < 0.01). Compared with industry supported reviews and reviews with undeclared support, Cochrane reviews had more often considered the potential for bias in the review—for example, by describing the method of concealment of allocation and describing excluded patients or studies. The seven industry supported reviews that had conclusions recommended the experimental drug without reservations, compared with none of the Cochrane reviews (P = 0.02), although the estimated treatment effect was similar on average (z = 0.46, P = 0.64). Reviews with undeclared support and reviews with not for profit support or no support had conclusions that were similar in cautiousness to the Cochrane reviews. Conclusions Industry supported reviews of drugs should be read with caution as they were less transparent, had few reservations about methodological limitations of the included trials, and had more favourable conclusions than the corresponding Cochrane reviews.

[1]  M. Riachi,et al.  Crise d’épilepsie et vardenafil , 2006 .

[2]  H. Homayoun,et al.  The proconvulsant effect of sildenafil in mice: role of nitric oxide–cGMP pathway , 2006, British journal of pharmacology.

[3]  Alessandro Liberati,et al.  Assessment of methodological quality of primary studies by systematic reviews: results of the metaquality cross sectional study , 2005, BMJ : British Medical Journal.

[4]  M. Egger,et al.  Risk of cardiovascular events and rofecoxib: cumulative meta-analysis , 2004, The Lancet.

[5]  F. Montorsi,et al.  The efficacy and safety of flexible-dose vardenafil (levitra) in a broad population of European men. , 2004, European urology.

[6]  Christian Gluud,et al.  Association of funding and conclusions in randomized drug trials: a reflection of treatment effect or adverse events? , 2003, JAMA.

[7]  B. Djulbegovic,et al.  Pharmaceutical industry sponsorship and research outcome and quality: systematic review , 2003, BMJ : British Medical Journal.

[8]  C. Gross,et al.  Scope and impact of financial conflicts of interest in biomedical research: a systematic review. , 2003, JAMA.

[9]  S. Willich,et al.  How objective are systematic reviews? Differences between reviews on complementary medicine. , 2003, Journal of the Royal Society of Medicine.

[10]  G. Keating,et al.  Vardenafil: a review of its use in erectile dysfunction. , 2003, Drugs.

[11]  M. Egger,et al.  Allocation concealment in clinical trials. , 2002, JAMA.

[12]  Y. Lampl,et al.  Tonic-clonic seizures in patients taking sildenafil , 2002, BMJ : British Medical Journal.

[13]  A. Miller.  Screening for breast cancer with mammography , 2001, The Lancet.

[14]  R. Sperling,et al.  Cardiovascular Thrombotic Events in Controlled, Clinical Trials of Rofecoxib , 2001, Circulation.

[15]  P Middleton,et al.  Quality of Cochrane reviews: assessment of sample from 1998 , 2001, BMJ : British Medical Journal.

[16]  K. Hopayian,et al.  The need for caution in interpreting high quality systematic reviews , 2001, BMJ : British Medical Journal.

[17]  A R Jadad,et al.  Systematic reviews and meta-analyses on treatment of asthma: critical evaluation , 2000, BMJ : British Medical Journal.

[18]  D A Katerndahl,et al.  Variability in meta-analytic results concerning the value of cholesterol reduction in coronary heart disease: a meta-meta-analysis. , 1999, American journal of epidemiology.

[19]  A R Jadad,et al.  Methodology and reports of systematic reviews and meta-analyses: a comparison of Cochrane reviews with articles published in paper-based journals. , 1998, JAMA.

[20]  G. Smith,et al.  Meta-analysis: Unresolved issues and future developments , 1998, BMJ.

[21]  R. West.  Does blinding of readers affect results of meta-analyses? , 1997, The Lancet.

[22]  R. Simes,et al.  Publication bias: evidence of delayed publication in a cohort study of clinical research projects , 1997, BMJ.

[23]  D. Nutt,et al.  Adult night terrors and paroxetine , 1997, The Lancet.

[24]  Jesse A Berlin,et al.  Does blinding of readers affect the results of meta-analyses? , 1997, The Lancet.

[25]  G. Guyatt,et al.  Stress ulcer prophylaxis in critically ill patients. Resolving discordant meta-analyses. , 1996, JAMA.

[26]  E. Przegaliński,et al.  The role of nitric oxide in chemically- and electrically-induced seizures in mice , 1996, Neuroscience Letters.

[27]  A. Jadad,et al.  Meta-analyses to evaluate analgesic interventions: a systematic qualitative review of their methodology. , 1996, Journal of clinical epidemiology.

[28]  P. Gøtzsche.  Steroids and peptic ulcer: an end to the controversy? , 1994, Journal of internal medicine.

[29]  P. Rochon,et al.  A study of manufacturer-supported trials of nonsteroidal anti-inflammatory drugs in the treatment of arthritis. , 1994, Archives of internal medicine.

[30]  C. Katona,et al.  Effective and acceptable treatment for depression. , 1993, British medical journal.

[31]  F. Song,et al.  Selective serotonin reuptake inhibitors: meta-analysis of efficacy and acceptability. , 1993, BMJ.

[32]  P. Gøtzsche,et al.  Meta-analysis of second-line antirheumatic drugs: sample size bias and uncertain benefit. , 1992, Journal of clinical epidemiology.

[33]  G H Guyatt,et al.  Agreement among reviewers of review articles. , 1991, Journal of clinical epidemiology.

[34]  G. Guyatt,et al.  Validation of an index of the quality of review articles. , 1991, Journal of clinical epidemiology.

[35]  G H Guyatt,et al.  Guidelines for reading literature reviews. , 1988, CMAJ : Canadian Medical Association journal = journal de l'Association medicale canadienne.

[36]  T C Chalmers,et al.  Meta-analysis of clinical trials as a scientific discipline. II: Replicate variability and comparison of studies that agree and disagree. , 1987, Statistics in medicine.

[37]  E. Hemminki.  Study of information submitted by drug companies to licensing authorities. , 1980, British medical journal.