Comparison of Effects as Evidence Evolves From Single Trials to High-Quality Bodies of Evidence

The findings and conclusions in this document are those of the author(s), who are responsible for its content, and do not necessarily represent the views of AHRQ. Therefore, no statement in this report should be construed as an official position of AHRQ or of the U.S. Department of Health and Human Services. The information in this report is intended to help health care decisionmakers—patients and clinicians, health system leaders, and policymakers, among others—make well-informed decisions and thereby improve the quality of health care services. This report is not intended to be a substitute for the application of clinical judgment. Anyone who makes decisions concerning the provision of clinical care should consider this report in the same way as any medical reference and in conjunction with all other pertinent information, i.e., in the context of available resources and circumstances presented by individual patients. AHRQ or U.S. Department of Health and Human Services endorsement of any derivative products that may be developed from this report, such as clinical practice guidelines, other quality enhancement tools, or reimbursement or coverage policies may not be stated or implied. This document is in the public domain and may be used and reprinted without permission except those copyrighted materials that are clearly noted in the document. Further reproduction of those copyrighted materials is prohibited without the specific permission of copyright holders. Persons using assistive technology may not be able to fully access information in this report. For assistance contact EffectiveHealthCare@ahrq.hhs.gov. Drs. Gartlehner and Thaler are members of the GRADE Working Group. Drs. Gartlehner and Lohr are co-authors of the AHRQ guidance for grading the strength of evidence. None of the other authors have any affiliations or financial involvement disclosures to report. Preface The Agency for Healthcare Research and Quality (AHRQ), through its Evidence-Based Practice Centers (EPCs), sponsors the development of evidence reports and technology assessments to assist public-and private-sector organizations in their efforts to improve the quality of health care in the United States. The reports and assessments provide organizations with comprehensive, science-based information on common, costly medical conditions and new health care technologies. The EPCs systematically review the relevant scientific literature on topics assigned to them by AHRQ and conduct additional analyses when appropriate prior to developing their reports and assessments. To improve the scientific rigor of these evidence reports, AHRQ supports empiric research by the EPCs to help understand or improve complex methodologic issues in …

[1]  L. Amato,et al.  Antidepressants for cocaine dependence and problematic cocaine use. , 2011, The Cochrane database of systematic reviews.

[2]  L. Vidal,et al.  Anthracycline-containing regimens for treatment of follicular lymphoma in adults. , 2013, The Cochrane database of systematic reviews.

[3]  Thomas A Trikalinos,et al.  Early extreme contradictory estimates may appear in published research: the Proteus phenomenon in molecular genetics research and randomized trials. , 2005, Journal of clinical epidemiology.

[4]  M. Martinez-Alonso,et al.  Perioperative glycaemic control for diabetic patients undergoing surgery. , 2012, The Cochrane database of systematic reviews.

[5]  P. Glasziou,et al.  Antibiotics for sore throat. , 2004, The Cochrane database of systematic reviews.

[6]  P. Poole,et al.  Tiotropium versus placebo for chronic obstructive pulmonary disease. , 2012, The Cochrane database of systematic reviews.

[7]  R. D. de Haan,et al.  Intravenous immunoglobulin for chronic inflammatory demyelinating polyradiculoneuropathy. , 2009, The Cochrane database of systematic reviews.

[8]  K. Bassett,et al.  Blood pressure-lowering efficacy of monotherapy with thiazide diuretics for primary hypertension. , 2014, The Cochrane database of systematic reviews.

[9]  L. Ronfani,et al.  Oral zinc for treating diarrhoea in children. , 2012, The Cochrane database of systematic reviews.

[10]  F. Ducharme,et al.  Combined inhaled anticholinergics and short-acting beta2-agonists for initial treatment of acute asthma in children. , 2013, Paediatric respiratory reviews.

[11]  N. Gogtay,et al.  Artemisinin-based combination therapy for treating uncomplicated Plasmodium vivax malaria. , 2013, The Cochrane database of systematic reviews.

[12]  R. Nelson,et al.  Non surgical therapy for anal fissure. , 2012, The Cochrane database of systematic reviews.

[13]  O. Ozturk,et al.  Number of embryos for transfer following in-vitro fertilisation or intra-cytoplasmic sperm injection. , 2014, The Cochrane database of systematic reviews.

[14]  P. Poole,et al.  Combined corticosteroid and long-acting beta(2)-agonist in one inhaler versus inhaled corticosteroids alone for chronic obstructive pulmonary disease. , 2013, The Cochrane database of systematic reviews.

[15]  Xiaoxi Zeng,et al.  Omentoplasty for oesophagogastrostomy after oesophagectomy. , 2014, The Cochrane database of systematic reviews.

[16]  H. Bruschini,et al.  Traditional suburethral sling operations for urinary incontinence in women. , 2017, The Cochrane database of systematic reviews.

[17]  R. Moore,et al.  Sumatriptan plus naproxen for acute migraine attacks in adults. , 2013, The Cochrane database of systematic reviews.

[18]  M. V. van Driel,et al.  Antibiotics for clinically diagnosed acute rhinosinusitis in adults. , 2012, The Cochrane database of systematic reviews.

[19]  C. Mulrow,et al.  Pharmacotherapy for hypertension in the elderly , 1998 .

[20]  T. Braithwaite,et al.  Anti-vascular endothelial growth factor for macular oedema secondary to central retinal vein occlusion. , 2014, The Cochrane database of systematic reviews.

[21]  D. Young,et al.  Pulmonary artery catheters for adult patients in intensive care. , 2013, The Cochrane database of systematic reviews.

[22]  J. Ioannidis,et al.  Establishment of genetic associations for complex diseases is independent of early study findings , 2004, European Journal of Human Genetics.

[23]  M. Nitsche,et al.  Antibiotics for acute otitis media in children. , 2015, Medwave.

[24]  A. J. Kelly,et al.  Vaginal prostaglandin (PGE2 and PGF2a) for induction of labour at term. , 2014, The Cochrane database of systematic reviews.

[25]  C. Foster,et al.  Face-to-face interventions for promoting physical activity. , 2013, The Cochrane database of systematic reviews.

[26]  T. Wilt,et al.  Serenoa repens for benign prostatic hyperplasia. , 2000, The Cochrane database of systematic reviews.

[27]  A. Deorari,et al.  Light-emitting diode phototherapy for unconjugated hyperbilirubinaemia in neonates. , 2011, The Cochrane database of systematic reviews.

[28]  B. Rowe,et al.  Cochrane review: Holding chambers (spacers) versus nebulisers for beta‐agonist treatment of acute asthma , 2006 .

[29]  Kayleigh Kew,et al.  Inhaled steroids and risk of pneumonia for chronic obstructive pulmonary disease. , 2014, The Cochrane database of systematic reviews.

[30]  R. Nelson,et al.  Operative procedures for fissure in ano. , 2011, The Cochrane database of systematic reviews.

[31]  Claire Rickard,et al.  Clinically-indicated replacement versus routine replacement of peripheral venous catheters. , 2010, The Cochrane database of systematic reviews.

[32]  Rachel Churchill,et al.  Effect sizes in cumulative meta-analyses of mental health randomized trials evolved over time. , 2004, Journal of clinical epidemiology.

[33]  Antimicrobial prophylaxis in colorectal surgery. , 1998, Nursing times.

[34]  G. Norman,et al.  The truly remarkable universality of half a standard deviation: confirmation through another look , 2004, Expert review of pharmacoeconomics & outcomes research.

[35]  S. R. Lewis,et al.  Nonsteroidal anti-inflammatory drugs and perioperative bleeding in paediatric tonsillectomy. , 2013, The Cochrane database of systematic reviews.

[36]  L. Amato,et al.  Methadone at tapered doses for the management of opioid withdrawal. , 2003, The Cochrane database of systematic reviews.

[37]  M. Prins,et al.  Duration of treatment with vitamin K antagonists in symptomatic venous thromboembolism. , 2014, The Cochrane database of systematic reviews.

[38]  L. Dans,et al.  Intravenous immunoglobulin for treating sepsis, severe sepsis and septic shock. , 2013, The Cochrane database of systematic reviews.

[39]  Howard Balshem,et al.  GRADE guidelines: 3. Rating the quality of evidence. , 2011, Journal of clinical epidemiology.

[40]  P. Middleton,et al.  Effects of sevoflurane versus other general anaesthesia on emergence agitation in children. , 2014, The Cochrane database of systematic reviews.

[41]  J. Kavanagh,et al.  Vaginal prostaglandin (PGE2 and PGF2a) for induction of labour at term. , 2003, The Cochrane database of systematic reviews.

[42]  B. Feagan,et al.  Oral 5-aminosalicylic acid for maintenance of remission in ulcerative colitis. , 2012, The Cochrane database of systematic reviews.

[43]  N. Foster OF ACUPUNCTURE FOR PERIPHERAL JOINT OSTEOARTHRITIS , 2015 .

[44]  K. Lohr,et al.  Grades for quality of evidence were associated with distinct likelihoods that treatment effects will remain stable. , 2015, Journal of clinical epidemiology.

[45]  B. Hemmingsen Targeting intensive glycaemic control versus targeting conventional glycaemic control for type 2 diabetes mellitus , 2011, The Cochrane database of systematic reviews.

[46]  P. Tugwell,et al.  Exercise for preventing and treating osteoporosis in postmenopausal women. , 2011, The Cochrane database of systematic reviews.

[47]  J. Kwong,et al.  Factor Xa inhibitors for acute coronary syndromes. , 2011, The Cochrane database of systematic reviews.

[48]  S. Mcconnell,et al.  Exercise for osteoarthritis of the hip. , 2009, The Cochrane database of systematic reviews.

[49]  X. Montalban,et al.  Interferon beta in secondary progressive multiple sclerosis , 2007, Journal of Neurology.

[50]  R. Moore,et al.  Zolmitriptan for acute migraine attacks in adults. , 2014, The Cochrane database of systematic reviews.

[51]  T. Lasserson,et al.  Addition to inhaled corticosteroids of long-acting beta2-agonists versus anti-leukotrienes for chronic asthma. , 2011, The Cochrane database of systematic reviews.

[52]  Cindy Stern,et al.  Infraclavicular brachial plexus block for regional anaesthesia of the lower arm. , 2015, International journal of evidence-based healthcare.

[53]  W. Baine,et al.  The Agency for Healthcare Research and Quality , 2006, Italian Journal of Public Health.

[54]  L. Vidal,et al.  Colony-stimulating factors for prevention and treatment of infectious complications in patients with acute myelogenous leukemia. , 2012, The Cochrane database of systematic reviews.

[55]  R. Moore,et al.  Single dose oral etoricoxib for acute postoperative pain in adults. , 2009, The Cochrane database of systematic reviews.

[56]  G. Gartlehner,et al.  Household interventions for preventing domestic lead exposure in children. , 2016, The Cochrane database of systematic reviews.

[57]  P. Lachenbruch Statistical Power Analysis for the Behavioral Sciences (2nd ed.) , 1989 .

[58]  A. Berghold,et al.  Intensive glucose control versus conventional glucose control for type 1 diabetes mellitus. , 2014, The Cochrane database of systematic reviews.

[59]  F. Ducharme,et al.  Addition to inhaled corticosteroids of long-acting beta2-agonists versus anti-leukotrienes for chronic asthma. , 2014, The Cochrane database of systematic reviews.

[60]  L. Amato,et al.  Psychosocial combined with agonist maintenance treatments versus agonist maintenance treatments alone for treatment of opioid dependence. , 2004, The Cochrane database of systematic reviews.

[61]  S. Straus,et al.  Hospital at home: home-based end of life care. , 2011, The Cochrane database of systematic reviews.

[62]  D. Massel,et al.  Antiplatelet and anticoagulation for patients with prosthetic heart valves. , 2013, The Cochrane database of systematic reviews.

[63]  K. Wilhelmus Antiviral treatment and other therapeutic interventions for herpes simplex virus epithelial keratitis. , 2010, The Cochrane database of systematic reviews.

[64]  David W. Johnson,et al.  Biocompatible dialysis fluids for peritoneal dialysis. , 2014, The Cochrane database of systematic reviews.

[65]  T. Poynard,et al.  Interferon for interferon nonresponding and relapsing patients with chronic hepatitis C. , 2002, The Cochrane database of systematic reviews.

[66]  E. Akl,et al.  Parenteral anticoagulation in patients with cancer who have no therapeutic or prophylactic indication for anticoagulation. , 2011, The Cochrane database of systematic reviews.

[67]  R. Hughes,et al.  Intravenous immunoglobulin for Guillain-Barré syndrome. , 2006, The Cochrane database of systematic reviews.

[68]  M. Wallen,et al.  Botulinum toxin A as an adjunct to treatment in the management of the upper limb in children with spastic cerebral palsy (UPDATE). , 2010, The Cochrane database of systematic reviews.

[69]  M. McDonagh,et al.  Grading the Strength of a Body of Evidence When Assessing Health Care Interventions for the Effective Health Care Program of the Agency for Healthcare Research and Quality: An Update , 2013 .

[70]  A. Ciapponi,et al.  High versus low positive end-expiratory pressure (PEEP) levels for mechanically ventilated adult patients with acute lung injury and acute respiratory distress syndrome. , 2013, The Cochrane database of systematic reviews.

[71]  Xi-Jing Chen,et al.  Music therapy for people with schizophrenia and schizophrenia-like disorders. , 2011, The Cochrane database of systematic reviews.

[72]  K. Wilhelmus Therapeutic interventions for herpes simplex virus epithelial keratitis. , 2007, The Cochrane database of systematic reviews.

[73]  T. Lasserson,et al.  Regular treatment with formoterol for chronic asthma: serious adverse events. , 2008, The Cochrane database of systematic reviews.

[74]  Stephen J. Gentles,et al.  The GRADE approach is reproducible in assessing the quality of evidence of quantitative evidence syntheses. , 2013, Journal of clinical epidemiology.

[75]  V. Musini,et al.  Effect of early treatment with anti-hypertensive drugs on short and long-term mortality in patients with an acute cardiovascular event. , 2009, The Cochrane database of systematic reviews.

[76]  L. Amato,et al.  Dopamine agonists for the treatment of cocaine dependence. , 2015, The Cochrane database of systematic reviews.

[77]  J. Ioannidis,et al.  Benzodiazepines for alcohol withdrawal. , 2005, The Cochrane database of systematic reviews.

[78]  M. Bulsara,et al.  Venom immunotherapy for preventing allergic reactions to insect stings. , 2012, The Cochrane database of systematic reviews.

[79]  S. Fisher,et al.  Stem cell treatment for acute myocardial infarction. , 2012, The Cochrane database of systematic reviews.

[80]  James M Wright,et al.  Angiotensin converting enzyme (ACE) inhibitors versus angiotensin receptor blockers for primary hypertension. , 2014, The Cochrane database of systematic reviews.

[81]  Jacob Cohen Statistical Power Analysis for the Behavioral Sciences , 1969, The SAGE Encyclopedia of Research Design.

[82]  F. Qadri,et al.  Oral vaccines for preventing cholera. , 2011, The Cochrane database of systematic reviews.

[83]  J. Utzinger,et al.  Drugs for treating urinary schistosomiasis , 2014, The Cochrane database of systematic reviews.

[84]  J. DeVries,et al.  Insulin detemir versus insulin glargine for type 2 diabetes mellitus. , 2011, The Cochrane database of systematic reviews.

[85]  L. Lin,et al.  A concordance correlation coefficient to evaluate reproducibility. , 1989, Biometrics.

[86]  R. Perera,et al.  Corticosteroids as standalone or add-on treatment for sore throat. , 2012, The Cochrane database of systematic reviews.

[87]  G. Wassmer,et al.  Preoperative statin therapy for patients undergoing cardiac surgery. , 2012, The Cochrane database of systematic reviews.

[88]  B. Vandermeer,et al.  Glucocorticoids for acute viral bronchiolitis in infants and young children. , 2013, The Cochrane database of systematic reviews.

[89]  Peter Tugwell,et al.  Methotrexate for treating rheumatoid arthritis. , 2014, The Cochrane database of systematic reviews.

[90]  N. Chavannes,et al.  Integrated disease management interventions for patients with chronic obstructive pulmonary disease. , 2013, The Cochrane database of systematic reviews.

[91]  P. Coulthard,et al.  Ibuprofen and/or paracetamol (acetaminophen) for pain relief after surgical removal of lower wisdom teeth. , 2013, The Cochrane database of systematic reviews.

[92]  J. Holden,et al.  Hepatitis A immunisation in persons not previously exposed to hepatitis A. , 2012, The Cochrane database of systematic reviews.

[93]  P. Poole,et al.  Phosphodiesterase 4 inhibitors for chronic obstructive pulmonary disease. , 2017, The Cochrane database of systematic reviews.

[94]  Anne Eisinga,et al.  Hormone therapy for preventing cardiovascular disease in post-menopausal women. , 2015, The Cochrane database of systematic reviews.

[95]  E. Hodson,et al.  Frequency of administration of erythropoiesis-stimulating agents for the anaemia of end-stage kidney disease in dialysis patients. , 2014, The Cochrane database of systematic reviews.

[96]  P. Alonso-Coello,et al.  Bronchial thermoplasty for moderate or severe persistent asthma in adults. , 2014, The Cochrane database of systematic reviews.

[97]  Karl Schurr,et al.  Stretch for the treatment and prevention of contractures. , 2010, The Cochrane database of systematic reviews.

[98]  John P A Ioannidis,et al.  Empirical evaluation of very large treatment effects of medical interventions. , 2012, JAMA.

[99]  A. Webster,et al.  Antiviral medications for preventing cytomegalovirus disease in solid organ transplant recipients. , 2005, The Cochrane database of systematic reviews.

[100]  N. Skoetz,et al.  Comparison of chemotherapy including escalated BEACOPP versus chemotherapy including ABVD for patients with early unfavourable or advanced stage Hodgkin lymphoma. , 2011, The Cochrane database of systematic reviews.

[101]  G. Guyatt,et al.  GRADE guidelines: 1. Introduction-GRADE evidence profiles and summary of findings tables. , 2011, Journal of clinical epidemiology.

[102]  L. Moja,et al.  Trastuzumab containing regimens for early breast cancer. , 2012, The Cochrane database of systematic reviews.

[103]  R. Ali,et al.  Buprenorphine for the management of opioid withdrawal. , 2009, The Cochrane database of systematic reviews.

[104]  P. Lumbiganon,et al.  Paracervical local anaesthesia for cervical dilatation and uterine intervention. , 2009, The Cochrane database of systematic reviews.

[105]  K. Soares-Weiser,et al.  Vaccines for preventing rotavirus diarrhoea: vaccines in use. , 2019, The Cochrane database of systematic reviews.

[106]  M. Davey,et al.  Interventions for supporting pregnant women's decision-making about mode of birth after a caesarean. , 2013, The Cochrane database of systematic reviews.

[107]  Kayleigh Kew,et al.  Combination formoterol and budesonide as maintenance and reliever therapy versus combination inhaler maintenance for chronic asthma in adults and children. , 2013, The Cochrane database of systematic reviews.

[108]  L. Leibovici,et al.  Beta-lactam versus beta-lactam-aminoglycoside combination therapy in cancer patients with neutropenia. , 2013, The Cochrane database of systematic reviews.

[109]  L. Trinquart,et al.  Influence of trial sample size on treatment effect estimates: meta-epidemiological study , 2013, BMJ : British Medical Journal.

[110]  P. Wiffen SEROTONIN AND NORADRENALINE REUPTAKE INHIBITORS (SNRIs) FOR FIBROMYALGIA SYNDROME , 2013 .

[111]  V. Welch,et al.  Oral or transdermal opioids for osteoarthritis of the knee or hip. , 2009, The Cochrane database of systematic reviews.

[112]  Atle Fretheim,et al.  Flexible sigmoidoscopy versus faecal occult blood testing for colorectal cancer screening in asymptomatic individuals. , 2013, The Cochrane database of systematic reviews.

[113]  S. Saint,et al.  Catheter impregnation, coating or bonding for reducing central venous catheter-related infections in adults. , 2013, The Cochrane database of systematic reviews.

[114]  K. Charoenkwan,et al.  Retroperitoneal drainage versus no drainage after pelvic lymphadenectomy for the prevention of lymphocyst formation in patients with gynaecological malignancies. , 2010, The Cochrane database of systematic reviews.

[115]  T. Stroup,et al.  Paliperidone palmitate for schizophrenia. , 2012, The Cochrane database of systematic reviews.

[116]  L. Leibovici,et al.  Antibiotic prophylaxis for bacterial infections in afebrile neutropenic patients following chemotherapy. , 2005, The Cochrane database of systematic reviews.