Trends in Worldwide Volume and Methodological Quality of Surgical Randomized Controlled Trials

Objective: To assess worldwide trends in volume and methodological quality of published surgical randomized controlled trials (RCTs) over the past decade. Background: Randomized controlled trials are essential for clinical decision making. It has repeatedly been suggested that surgical RCTs are scarce and of mediocre quality. Methods: We systematically searched PubMed for surgical RCTs published in 1999 and 2009. Characteristics and risks of bias were extracted. Trials where compared between study years and geographical regions. Primary outcome was “low risk of bias,” defined by all of the following: adequate allocation generation and concealment, intention-to-treat analysis, and adequate dropout handling. Results: The volume of published surgical RCTs increased by 50%, from 300 in 1999 to 450 in 2009. Volume increased in Europe (27% increase), Asia/Oceania (160% increase), and Africa/South America (416% increase) but decreased in North America (23% decrease), although the United States remained the country with the highest number of published RCTs. In 2009, methodological quality of surgical trials improved in terms of sample size calculation, adequate generation of randomization sequence, concealment of randomization sequence, and use of intention-to-treat analysis as compared with 1999 (P < 0.001 for all). The proportion of low risk of bias trials increased from 6% to 14% (prevalence ratio 2.59; 95% confidence interval 1.55–4.32). In 2009, the highest proportion of low risk of bias trials was from Europe (23%), whereas the lowest was from Asia/Oceania (5%). Conclusions: Volume and quality of surgical RCTs improved although striking differences exist between continents and countries. Structured education in trial methodology, enforced adherence to existing guidelines, and improved research infrastructure may guide further improvements.

[1]  Richard Horton,et al.  Surgical research or comic opera: questions, but few answers , 1996, The Lancet.

[2]  D G Altman,et al.  Assessing the quality of randomization from reports of controlled trials published in obstetrics and gynecology journals. , 1994, JAMA.

[3]  G. Mashour,et al.  Improvement in the Quality of Randomized Controlled Trials Among General Anesthesiology Journals 2000 to 2006: A 6-Year Follow-Up , 2009, Anesthesia and analgesia.

[4]  T. Reynolds.  Why randomized surgical oncology trials are so scarce. , 1999, Journal of the National Cancer Institute.

[5]  A. Walker,et al.  Improving the quality of reporting in randomised controlled trials. , 2004, Journal of wound care.

[6]  Jonathan A Cook,et al.  The challenges faced in the design, conduct and analysis of surgical randomised controlled trials , 2009, Trials.

[7]  D. Gouma,et al.  Impact of nationwide centralization of pancreaticoduodenectomy on hospital mortality , 2012, The British journal of surgery.

[8]  D. Banks,et al.  Educating medical students in evidence-based medicine: what we should expect as a starting point for our house officers. , 2012, Southern medical journal.

[9]  M. Egger,et al.  The hazards of scoring the quality of clinical trials for meta-analysis. , 1999, JAMA.

[10]  Douglas G Altman,et al.  Endorsement of the CONSORT statement by high impact medical journals: survey of instructions for authors , 2005, BMJ : British Medical Journal.

[11]  Vance W Berger,et al.  When can a clinical trial be called 'randomized'? , 2003, Vaccine.

[12]  Kenneth F Schulz,et al.  Blinding in randomised trials: hiding who got what , 2002, The Lancet.

[13]  G. Guyatt,et al.  Blinding of outcomes in trials of orthopaedic trauma: an opportunity to enhance the validity of clinical trials. , 2008, The Journal of bone and joint surgery. American volume.

[14]  D. Moher,et al.  The CONSORT statement: revised recommendations for improving the quality of reports of parallel-group randomized trials. , 2001, Journal of the American Podiatric Medical Association.

[15]  D. Chang,et al.  Why should surgeons care about clinical research methodology? , 2006, Journal of the American College of Surgeons.

[16]  I. Olkin,et al.  Improving the quality of reporting of randomized controlled trials. The CONSORT statement. , 1996, JAMA.

[17]  Douglas G Altman,et al.  Epidemiology and reporting of randomised trials published in PubMed journals , 2005, The Lancet.

[18]  A. Sheikh,et al.  Comparison of reports of randomized controlled trials and systematic reviews in surgical journals: literature review. , 2006, Journal of the Royal Society of Medicine.

[19]  David W. Taggart,et al.  No surgical innovation without evaluation: the IDEAL recommendations , 2009, The Lancet.

[20]  Jing Li,et al.  Chinese authors do need CONSORT: reporting quality assessment for five leading Chinese medical journals. , 2008, Contemporary clinical trials.

[21]  Darlene Chapman.  Health-related databases. , 2009, Journal of the Canadian Academy of Child and Adolescent Psychiatry = Journal de l'Academie canadienne de psychiatrie de l'enfant et de l'adolescent.

[22]  R. McLeod,et al.  Clinical studies in surgical journals—have we improved? , 1993, Diseases of the colon and rectum.

[23]  D. Moher,et al.  CONSORT 2010 Explanation and Elaboration: updated guidelines for reporting parallel group randomised trials , 2010, BMJ : British Medical Journal.

[24]  Douglas G Altman,et al.  Systematic reviews in health care: Assessing the quality of controlled clinical trials. , 2001, BMJ.

[25]  M. Kocher,et al.  Overcoming the funding challenge: the cost of randomized controlled trials in the next decade. , 2012, The Journal of bone and joint surgery. American volume.

[26]  A concept for trial institutions focussing on randomised controlled trials in surgery , 2008, Trials.

[27]  C. Dejong,et al.  Impact of hemostasis and blood loss on outcome after liver surgery , 2018 .

[28]  Mitsuru Sasako,et al.  Randomised trials in surgery: problems and possible solutions , 2002, BMJ : British Medical Journal.

[29]  D. Moher,et al.  Endorsement of the CONSORT Statement by high impact factor medical journals: a survey of journal editors and journal 'Instructions to Authors' , 2008, Trials.

[30]  T. Pawlik,et al.  Trends in the Quality of Highly Cited Surgical Research Over the Past 20 Years , 2009, Annals of surgery.

[31]  D. Moher,et al.  Use of the CONSORT statement and quality of reports of randomized trials: a comparative before-and-after evaluation. , 2001, JAMA.

[32]  R. Thisted,et al.  Clinical trials in general surgical journals: are methods better reported? , 1999, Surgery.

[33]  Robert L Kane,et al.  Reporting in randomized clinical trials improved after adoption of the CONSORT statement. , 2007, Journal of clinical epidemiology.

[34]  T. Venkatesan.  Probiotic Prophylaxis in Predicted Severe Acute Pancreatitis: A Randomised, Double-Blind, Placebo-Controlled Trial. , 2008, Nutrition in clinical practice : official publication of the American Society for Parenteral and Enteral Nutrition.

[35]  D. Moher,et al.  Does the CONSORT checklist improve the quality of reports of randomised controlled trials? A systematic review , 2006, The Medical journal of Australia.

[36]  G. Leung,et al.  Quality of Reporting of Key Methodological Items of Randomized Controlled Trials in Clinical Ophthalmic Journals , 2007, Ophthalmic epidemiology.

[37]  Michele Tarsilla.  Cochrane Handbook for Systematic Reviews of Interventions , 2010, Journal of MultiDisciplinary Evaluation.

[38]  B. Djulbegovic,et al.  Bad reporting does not mean bad methods for randomised trials: observational study of randomised controlled trials performed by the Radiation Therapy Oncology Group , 2004, BMJ : British Medical Journal.

[39]  J D Emerson,et al.  An empirical study of the possible relation of treatment differences to quality scores in controlled randomized clinical trials. , 1990, Controlled clinical trials.

[40]  G. Wang,et al.  The quality of reporting of randomized controlled trials of traditional Chinese medicine: a survey of 13 randomly selected journals from mainland China. , 2007, Clinical therapeutics.

[41]  D. Moher,et al.  Randomized trials published in some Chinese journals: how many are randomized? , 2009, Trials.

[42]  Gordon H Guyatt,et al.  An observational study found that authors of randomized controlled trials frequently use concealment of randomization and blinding, despite the failure to report these methods. , 2004, Journal of clinical epidemiology.

[43]  D. Moher,et al.  Opportunities and challenges for improving the quality of reporting clinical research: CONSORT and beyond , 2004, Canadian Medical Association Journal.

[44]  Sameer Desai,et al.  Adductor surgery to prevent hip displacement in children with cerebral palsy: the predictive role of the Gross Motor Function Classification System. , 2012, The Journal of bone and joint surgery. American volume.

[45]  M. Büchler,et al.  Perspectives of Evidence-Based Surgery , 2003, Digestive Surgery.

[46]  R. Agha,et al.  The reporting quality of randomised controlled trials in surgery: a systematic review. , 2007, International journal of surgery.

[47]  R. J. Hayes,et al.  Empirical evidence of bias. Dimensions of methodological quality associated with estimates of treatment effects in controlled trials. , 1995, JAMA.

[48]  Mohit Bhandari,et al.  Randomized Controlled Trials of Surgical Interventions , 2010, Annals of surgery.

[49]  M. Boermeester,et al.  A step-up approach or open necrosectomy for necrotizing pancreatitis. , 2010, The New England journal of medicine.

[50]  N. Black.  CONSORT , 1996, The Lancet.

[51]  Jane M Blazeby,et al.  Challenges in evaluating surgical innovation , 2009, The Lancet.

[52]  Douglas G Altman,et al.  Empirical evidence of bias in treatment effect estimates in controlled trials with different interventions and outcomes: meta-epidemiological study , 2008, BMJ : British Medical Journal.

[53]  S. Hopewell,et al.  Assessment of the Quality of Reporting in Abstracts of Randomized Controlled Trials Published in Five Leading Chinese Medical Journals , 2010, PloS one.

[54]  I. Autti-Rämö,et al.  Quality of Reporting of Randomized, Controlled Trials in Cerebral Palsy , 2006, Pediatrics.

[55]  Sally Hopewell,et al.  The quality of reports of randomised trials in 2000 and 2006: comparative study of articles indexed in PubMed , 2010, BMJ : British Medical Journal.