PRISMA harms checklist: improving harms reporting in systematic reviews

Introduction For any health intervention, accurate knowledge of both benefits and harms is needed. Systematic reviews often compound poor reporting of harms in primary studies by failing to report harms or doing so inadequately. While the PRISMA statement (Preferred Reporting Items for Systematic reviews and Meta-Analyses) helps systematic review authors ensure complete and transparent reporting, it is focused mainly on efficacy. Thus, a PRISMA harms checklist has been developed to improve harms reporting in systematic reviews, promoting a more balanced assessment of benefits and harms. Methods A development strategy, endorsed by the EQUATOR Network and existing reporting guidelines (including the PRISMA statement, PRISMA for abstracts, and PRISMA for protocols), was used. After the development of a draft checklist of items, a modified Delphi process was initiated. The Delphi consisted of three rounds of electronic feedback followed by an in-person meeting. Results The PRISMA harms checklist contains four essential reporting elements to be added to the original PRISMA statement to improve harms reporting in reviews. These are reported in the title (“Specifically mention ‘harms’ or other related terms, or the harm of interest in the review”), synthesis of results (“Specify how zero events were handled, if relevant”), study characteristics (“Define each harm addressed, how it was ascertained (eg, patient report, active search), and over what time period”), and synthesis of results (“Describe any assessment of possible causality”). Additional guidance regarding existing PRISMA items was developed to demonstrate relevance when synthesising information about harms. Conclusion The PRISMA harms checklist identifies a minimal set of items to be reported when reviewing adverse events. This guideline extension is intended to improve harms reporting in systematic reviews, whether harms are a primary or secondary outcome.

[1]  Jonathan J Shuster,et al.  Empirical vs natural weighting in random effects meta‐analysis , 2009, Statistics in medicine.

[2]  Su Golder,et al.  THE CONTRIBUTION OF DIFFERENT INFORMATION SOURCES FOR ADVERSE EFFECTS DATA , 2012, International Journal of Technology Assessment in Health Care.

[3]  J. Ioannidis,et al.  Challenges in meta-analysis of randomized clinical trials for rare harmful cardiovascular events: the case of rosiglitazone. , 2008, American heart journal.

[4]  A. B. Hill,et al.  "The Environment and Disease: Association or Causation?" (1965), by Austin Bradford Hill , 2017 .

[5]  B. Guglielmo,et al.  The FDA extended warning for intravenous haloperidol and torsades de pointes: how should institutions respond? , 2010, Journal of hospital medicine.

[6]  I. Olkin,et al.  Meta-analysis of observational studies in epidemiology - A proposal for reporting , 2000 .

[7]  D. Altman,et al.  Outcome reporting bias in randomized trials funded by the Canadian Institutes of Health Research , 2004, Canadian Medical Association Journal.

[8]  P. Middleton,et al.  Maternal adverse effects of different antenatal magnesium sulphate regimens for improving maternal and infant outcomes: a systematic review , 2013, BMC Pregnancy and Childbirth.

[9]  Y. Loke,et al.  Bi‐directional interaction between hypoglycaemia and cognitive impairment in elderly patients treated with glucose‐lowering agents: a systematic review and meta‐analysis , 2016, Diabetes, obesity & metabolism.

[10]  D. Sargent,et al.  Adverse-event rates: journals versus databases , 2007, The Lancet.

[11]  Theo Stijnen,et al.  Random effects meta‐analysis of event outcome in the framework of the generalized linear mixed model with applications in sparse data , 2010, Statistics in medicine.

[12]  S. Golder,et al.  Unpublished data can be of value in systematic reviews of adverse effects: methodological overview. , 2010, Journal of clinical epidemiology.

[13]  A. B. Hill The Environment and Disease: Association or Causation? , 1965, Proceedings of the Royal Society of Medicine.

[14]  Sally Hopewell,et al.  Reporting of adverse events in systematic reviews can be improved: survey results. , 2008, Journal of clinical epidemiology.

[15]  David Moher,et al.  Increasing value and reducing waste in biomedical research: who's listening? , 2016, The Lancet.

[16]  Y. Loke,et al.  If nothing happens, is everything all right? Distinguishing genuine reassurance from a false sense of security , 2015, Canadian Medical Association Journal.

[17]  J. Gray,et al.  Evidence-Based Healthcare: How to Make Health Policy and Management Decisions , 2001 .

[18]  I. Olkin,et al.  The case of the misleading funnel plot , 2006, BMJ : British Medical Journal.

[19]  John P A Ioannidis,et al.  Availability of large-scale evidence on specific harms from systematic reviews of randomized trials. , 2004, The American journal of medicine.

[20]  Su Golder,et al.  Failure or success of electronic search strategies to identify adverse effects data. , 2012, Journal of the Medical Library Association : JMLA.

[21]  Jonathan J Shuster,et al.  Rebuttal to Carpenter et al. comments on ‘Fixed vs. random effects meta‐analysis in rare event studies: The rosiglitazone link with myocardial infarction and cardiac death’ , 2008 .

[22]  John P A Ioannidis,et al.  Reasons or excuses for avoiding meta-analysis in forest plots , 2008, BMJ : British Medical Journal.

[23]  Katja Jasinskaja,et al.  Elaboration and Explanation ⋆ , 2011 .

[24]  Lisa Hartling,et al.  Quality of reporting in systematic reviews of adverse events: systematic review , 2014, BMJ : British Medical Journal.

[25]  Isabelle Boutron,et al.  Reporting of safety results in published reports of randomized controlled trials. , 2009, Archives of internal medicine.

[26]  David Moher,et al.  Does use of the CONSORT Statement impact the completeness of reporting of randomised controlled trials published in medical journals? A Cochrane reviewa , 2012, Systematic Reviews.

[27]  T. Michele,et al.  The safety of tiotropium--the FDA's conclusions. , 2010, The New England journal of medicine.

[28]  M. Schulzer,et al.  The effect of a bolus dose of etomidate on cortisol levels, mortality, and health services utilization: a systematic review. , 2010, Annals of emergency medicine.

[29]  John P A Ioannidis,et al.  Adverse events in randomized trials: neglected, restricted, distorted, and silenced. , 2009, Archives of internal medicine.

[30]  T. Einarson,et al.  The safety of anti-tumour necrosis factor treatments in rheumatoid arthritis: meta and exposure-adjusted pooled analyses of serious adverse events , 2008, Annals of the rheumatic diseases.

[31]  Sally Hopewell,et al.  PRISMA for Abstracts: Reporting Systematic Reviews in Journal and Conference Abstracts , 2013, PLoS medicine.

[32]  Douglas G Altman,et al.  The impact of outcome reporting bias in randomised controlled trials on a cohort of systematic reviews , 2010, BMJ : British Medical Journal.

[33]  S. Golder,et al.  Poor reporting and inadequate searches were apparent in systematic reviews of adverse effects. , 2008, Journal of clinical epidemiology.

[34]  Y. Yazici Some concerns about adverse event reporting in randomized clinical trials. , 2008, Bulletin of the NYU hospital for joint diseases.

[35]  J. Ioannidis,et al.  Recommendations for examining and interpreting funnel plot asymmetry in meta-analyses of randomised controlled trials , 2011, BMJ : British Medical Journal.

[36]  D. Altman,et al.  Selective reporting bias of harm outcomes within studies: findings from a cohort of systematic reviews , 2014, BMJ : British Medical Journal.

[37]  John P A Ioannidis,et al.  Optimal type I and type II error pairs when the available sample size is fixed. , 2013, Journal of clinical epidemiology.

[38]  Anne-Marie Bagnall,et al.  Bmc Medical Research Methodology Open Access Assessing Harmful Effects in Systematic Reviews , 2022 .

[39]  Sheena Derry,et al.  BMC Clinical Pharmacology BioMed Central BMC 1 2001, Clinical Pharmacology , 2001 .

[40]  E. Ernst,et al.  Assessment of therapeutic safety in systematic reviews: literature review , 2001, BMJ : British Medical Journal.

[41]  T. Einarson,et al.  Long-Term Developmental Outcome of Children of Women with Epilepsy, Unexposed or Exposed Prenatally to Antiepileptic Drugs , 2010, Drug safety.

[42]  S. Golder,et al.  Dexamethasone and haemorrhage risk in paediatric tonsillectomy: a systematic review and meta-analysis. , 2014, British journal of anaesthesia.

[43]  A. Idrovo,et al.  Safety of botulinum toxin type A among children with spasticity secondary to cerebral palsy: a systematic review of randomized clinical trials , 2009, Clinical rehabilitation.

[44]  J. Ioannidis,et al.  Safety of Medical Interventions in Children Versus Adults , 2014, Pediatrics.

[45]  D. Moher,et al.  Guidance for Developers of Health Research Reporting Guidelines , 2010, PLoS medicine.

[46]  S. Shakir,et al.  Systematic reviews of adverse effects of drug interventions: a survey of their conduct and reporting quality , 2009, Pharmacoepidemiology and drug safety.

[47]  S. Haque Ethics approval This study was conducted with the approval of the East London and City Health Authority Ethic Committee. Provenance and peer review Not commissioned; externally peer reviewed. , 2011 .

[48]  P. Shekelle,et al.  Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015: elaboration and explanation , 2016, British Medical Journal.

[49]  A. Hrõbjartsson,et al.  Empirical evidence for selective reporting of outcomes in randomized trials: comparison of protocols to published articles. , 2004, JAMA.

[50]  S. Bangalore,et al.  Angiotensin receptor blockers and risk of myocardial infarction: meta-analyses and trial sequential analyses of 147 020 patients from randomised trials , 2011, BMJ : British Medical Journal.

[51]  P. Enright,et al.  Mortality associated with tiotropium mist inhaler in patients with chronic obstructive pulmonary disease: systematic review and meta-analysis of randomised controlled trials , 2011, BMJ : British Medical Journal.

[52]  G. D. Dal Pan,et al.  Reporting of meta-analyses of randomized controlled trials with a focus on drug safety: An empirical assessment , 2013, Clinical trials.

[53]  Andrew Herxheimer,et al.  Systematic reviews of adverse effects: framework for a structured approach , 2007, BMC medical research methodology.

[54]  D. Moher,et al.  Improving the reporting and usability of research studies , 2013, Canadian Journal of Anesthesia/Journal canadien d'anesthésie.

[55]  K. Pilkington,et al.  Complementary medicine and safety: a systematic investigation of design and reporting of systematic reviews. , 2012, Complementary therapies in medicine.

[56]  Published Online Biomedical research: increasing value, reducing waste , 2014 .

[57]  J. Ioannidis,et al.  Better Reporting of Harms in Randomized Trials: An Extension of the CONSORT Statement , 2004, Annals of Internal Medicine.

[58]  Andrew Forbes,et al.  Bias due to selective inclusion and reporting of outcomes and analyses in systematic reviews of randomised trials of healthcare interventions. , 2014, The Cochrane database of systematic reviews.

[59]  S. Golder,et al.  Comprehensive evaluations of the adverse effects of drugs: importance of appropriate study selection and data sources , 2011, Therapeutic advances in drug safety.

[60]  G. Hejblum,et al.  A systematic review of adult admissions to ICUs related to adverse drug events , 2014, Critical Care.

[61]  R. Chou,et al.  Challenges in Systematic Reviews That Assess Treatment Harms , 2005, Annals of Internal Medicine.

[62]  J. Ioannidis,et al.  Completeness of safety reporting in randomized trials: an evaluation of 7 medical areas. , 2001, JAMA.

[63]  Alexander J Sutton,et al.  What to add to nothing? Use and avoidance of continuity corrections in meta-analysis of sparse data. , 2004, Statistics in medicine.

[64]  D. Spiegelhalter,et al.  Consensus development methods, and their use in clinical guideline development. , 1998, Health technology assessment.

[65]  A. Sandgren,et al.  Adverse Events in Healthy Individuals and MDR-TB Contacts Treated with Anti-Tuberculosis Drugs Potentially Effective for Preventing Development of MDR-TB: A Systematic Review , 2013, PloS one.

[66]  R. O’Neill,et al.  Does Research Synthesis Have a Place in Drug Regulatory Policy? Synopsis of Issues: Assessment of Safety and Postmarketing Surveillance , 1996 .

[67]  D. Sackett,et al.  Choosing the best research design for each question , 1997, BMJ.

[68]  Amy T. Wang,et al.  Clinical review 1: Adverse effects of testosterone therapy in adult men: a systematic review and meta-analysis. , 2010, The Journal of clinical endocrinology and metabolism.

[69]  Su Golder,et al.  Some improvements are apparent in identifying adverse effects in systematic reviews from 1994 to 2011. , 2013, Journal of clinical epidemiology.

[70]  A. Colevas,et al.  Adverse event reporting in publications compared with sponsor database for cancer clinical trials. , 2006, Journal of clinical oncology : official journal of the American Society of Clinical Oncology.

[71]  Ingram Olkin,et al.  Why add anything to nothing? The arcsine difference as a measure of treatment effect in meta‐analysis with zero cells , 2009, Statistics in medicine.

[72]  J. Higgins Cochrane handbook for systematic reviews of interventions. Version 5.1.0 [updated March 2011]. The Cochrane Collaboration , 2011 .

[73]  J. Aronson,et al.  BMC Medical Research Methodology BioMed Central BMC 1 2001, Medical Research Methodology , 2001 .

[74]  W. Prendiville,et al.  Perinatal mortality and other severe adverse pregnancy outcomes associated with treatment of cervical intraepithelial neoplasia: meta-analysis , 2008, BMJ : British Medical Journal.

[75]  David Moher,et al.  Reducing waste from incomplete or unusable reports of biomedical research , 2014, The Lancet.

[76]  J Hartley,et al.  Clarifying the abstracts of systematic literature reviews. , 2000, Bulletin of the Medical Library Association.

[77]  R. Chou,et al.  Methodological shortcomings predicted lower harm estimates in one of two sets of studies of clinical interventions. , 2007, Journal of clinical epidemiology.

[78]  Jonathan J Deeks,et al.  Much ado about nothing: a comparison of the performance of meta‐analytical methods with rare events , 2007, Statistics in medicine.

[79]  H. Gooszen,et al.  Robustness Assessments Are Needed to Reduce Bias in Meta-Analyses That Include Zero-Event Randomized Trials , 2009, The American Journal of Gastroenterology.

[80]  D. Moher,et al.  Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. , 2010, International journal of surgery.

[81]  David Moher,et al.  AHRQ series paper 4: assessing harms when comparing medical interventions: AHRQ and the effective health-care program. , 2010, Journal of clinical epidemiology.

[82]  Yoon K Loke,et al.  Drug safety assessment in clinical trials: methodological challenges and opportunities , 2012, Trials.

[83]  R. O’Neill,et al.  Does Research Synthesis Have a Place in Drug Regulatory Policy? Synopsis of Issues: Assessment of Efficacy and Drug Approval , 1996 .

[84]  Sonal Singh,et al.  Comparative cardiovascular effects of thiazolidinediones: systematic review and meta-analysis of observational studies , 2011, BMJ : British Medical Journal.

[85]  S. Reichenbach,et al.  Metamizole-Associated Adverse Events: A Systematic Review and Meta-Analysis , 2015, PloS one.

[86]  S. Golder,et al.  Bmc Medical Research Methodology Open Access Room for Improvement? a Survey of the Methods Used in Systematic Reviews of Adverse Effects , 2006 .