Outcomes in Cochrane Systematic Reviews Addressing Four Common Eye Conditions: An Evaluation of Completeness and Comparability

Introduction Choice of outcomes is critical for clinical trialists and systematic reviewers. It is currently unclear how systematic reviewers choose and pre-specify outcomes for systematic reviews. Our objective was to assess the completeness of pre-specification and comparability of outcomes in all Cochrane reviews addressing four common eye conditions. Methods We examined protocols for all Cochrane reviews as of June 2013 that addressed glaucoma, cataract, age-related macular degeneration (AMD), and diabetic retinopathy (DR). We assessed completeness and comparability for each outcome that was named in ≥25% of protocols on those topics. We defined a completely-specified outcome as including information about five elements: domain, specific measurement, specific metric, method of aggregation, and time-points. For each domain, we assessed comparability in how individual elements were specified across protocols. Results We identified 57 protocols addressing glaucoma (22), cataract (16), AMD (15), and DR (4). We assessed completeness and comparability for five outcome domains: quality-of-life, visual acuity, intraocular pressure, disease progression, and contrast sensitivity. Overall, these five outcome domains appeared 145 times (instances). Only 15/145 instances (10.3%) were completely specified (all five elements) (median = three elements per outcome). Primary outcomes were more completely specified than non-primary (median = four versus two elements). Quality-of-life was least completely specified (median = one element). Due to largely incomplete outcome pre-specification, conclusive assessment of comparability in outcome usage across the various protocols per condition was not possible. Discussion Outcome pre-specification was largely incomplete; we encourage systematic reviewers to consider all five elements. This will indicate the importance of complete specification to clinical trialists, on whose work systematic reviewers depend, and will indirectly encourage comparable outcome choice to reviewers undertaking related research questions. Complete pre-specification could improve efficiency and reduce bias in data abstraction and analysis during a systematic review. Ultimately, more completely specified and comparable outcomes could make systematic reviews more useful to decision-makers.

[1]  Jane M Blazeby,et al.  Developing core outcome sets for clinical trials: issues to consider , 2012, Trials.

[2]  C. Del Mar,et al.  Neuraminidase inhibitors for preventing and treating influenza in healthy adults and children , 2014, Sao Paulo medical journal = Revista paulista de medicina.

[3]  C. Adams,et al.  Content and quality of 2000 controlled trials in schizophrenia over 50 years , 1998, BMJ.

[4]  D. Altman,et al.  Identifying outcome reporting bias in randomised trials on PubMed: review of publications and survey of authors , 2005, BMJ : British Medical Journal.

[5]  J. Pell.  Clinical Trials Dictionary: Terminology and Usage Recommendations (2nd edition) , 2014 .

[6]  Thomas Kaiser,et al.  Impact of document type on reporting quality of clinical drug trials: a comparison of registry reports, clinical study reports, and journal publications , 2012, BMJ : British Medical Journal.

[7]  D G Altman,et al.  Estimating sample sizes for binary, ordered categorical, and continuous outcomes in two group comparisons , 1995, BMJ.

[8]  Beate Wieseler,et al.  Completeness of Reporting of Patient-Relevant Clinical Trial Outcomes: Comparison of Unpublished Clinical Study Reports with Publicly Available Data , 2013, PLoS medicine.

[9]  S. Salek.  Compendium of quality of life instruments , 1998 .

[10]  Michele Tarsilla.  Cochrane Handbook for Systematic Reviews of Interventions , 2010, Journal of MultiDisciplinary Evaluation.

[11]  M. Page,et al.  Many scenarios exist for selective inclusion and reporting of results in randomized trials and systematic reviews. , 2013, Journal of clinical epidemiology.

[12]  David Moher,et al.  Epidemiology and Reporting Characteristics of Systematic Reviews , 2007, PLoS medicine.

[13]  David Moher,et al.  Why prospective registration of systematic reviews makes sense , 2012, Systematic Reviews.

[14]  P. Tugwell,et al.  OMERACT: An international initiative to improve outcome measurement in rheumatology , 2007, Trials.

[15]  D. Moher,et al.  The nuts and bolts of PROSPERO: an international prospective register of systematic reviews , 2012, Systematic Reviews.

[16]  黄亚明（整理）,et al.  Equator network , 2012 .

[17]  Deborah M Caldwell,et al.  Simultaneous comparison of multiple treatments: combining direct and indirect evidence , 2005, BMJ : British Medical Journal.

[18]  Douglas G Altman,et al.  The impact of outcome reporting bias in randomised controlled trials on a cohort of systematic reviews , 2010, BMJ : British Medical Journal.

[19]  Mike Clarke,et al.  Standardising outcomes for clinical trials and systematic reviews , 2007, Trials.

[20]  A. Miller,et al.  Reporting results of cancer treatment , 1981, Cancer.

[21]  David Moher,et al.  Comparison of registered and published primary outcomes in randomized controlled trials. , 2009, JAMA.

[22]  A. Hrõbjartsson,et al.  Empirical evidence for selective reporting of outcomes in randomized trials: comparison of protocols to published articles. , 2004, JAMA.

[23]  Karen A Robinson,et al.  Development of a framework to identify research gaps from systematic reviews. , 2011, Journal of clinical epidemiology.

[24]  Laura A. Levit,et al.  Finding what works in health care : standards for systematic reviews , 2011 .

[25]  C. Meinert.  Clinical Trials Dictionary: Terminology and Usage Recommendations , 1996 .

[26]  D. Moher,et al.  PROSPERO at one year: an evaluation of its utility , 2013, Systematic Reviews.

[27]  Kristian Thorlund,et al.  Attention should be given to multiplicity issues in systematic reviews. , 2008, Journal of clinical epidemiology.

[28]  P. Enright,et al.  Mortality associated with tiotropium mist inhaler in patients with chronic obstructive pulmonary disease: systematic review and meta-analysis of randomised controlled trials , 2011, BMJ : British Medical Journal.

[29]  S. Goodman.  Meta-analysis and evidence. , 1989, Controlled clinical trials.

[30]  Kay Dickersin,et al.  Outcome reporting in industry-sponsored trials of gabapentin for off-label use. , 2009, The New England journal of medicine.

[31]  J. Farrar,et al.  Core outcome measures for chronic pain clinical trials: IMMPACT recommendations , 2003, Pain.

[32]  J. Higgins.  Cochrane handbook for systematic reviews of interventions. Version 5.1.0 [updated March 2011]. The Cochrane Collaboration , 2011 .

[33]  C. Meinert.  Clinical Trials Dictionary: Terminology and Usage Recommendations, Second Edition , 2012 .

[34]  D. Altman,et al.  Bias Due to Changes in Specified Outcomes during the Systematic Review Process , 2010, PloS one.

[35]  A R Jadad,et al.  Methodology and reports of systematic reviews and meta-analyses: a comparison of Cochrane reviews with articles published in paper-based journals. , 1998, JAMA.

[36]  Douglas G Altman,et al.  Comparison of protocols and registry entries to published reports for randomised controlled trials. , 2011, The Cochrane database of systematic reviews.

[37]  Tianjing Li,et al.  Network meta-analysis-highly attractive but more methodological research is needed , 2011, BMC medicine.

[38]  Nicholas C. Ide,et al.  The ClinicalTrials.gov results database--update and key issues. , 2011, The New England journal of medicine.