Development of the Grading of Recommendations Assessment, Development and Evaluation (GRADE) Summary of Findings (SoF) Table for Network Meta-analysis.

Abstract Objectives The aim of the study was to develop a Grading of Recommendations, Assessment, Development and Evaluation (GRADE) summary of findings (SoF) table format that displays the critical information from a network meta-analysis (NMA). Study Design and Setting We applied a user experience model for data analysis based on four rounds of semistructured interviews. Results We interviewed 32 stakeholders who conduct or use MA. Four rounds of interviews produced six candidate NMA-SoF tables. Users found a final NMA-SoF table that included the following components highly acceptable: (1) details of the clinical question (PICO), (2) a plot depicting network geometry, (3) relative and absolute effect estimates, (4) certainty of evidence, (5) ranking of treatments, and (6) interpretation of findings. Conclusion Using stakeholder feedback, we developed a new GRADE NMA-SoF table that includes the relevant components that facilitate understanding NMA findings and health decision-making.

[1]  Joseph Beyene,et al.  Bias in identification of the best treatment in a Bayesian network meta-analysis for binary outcome: a simulation study , 2014, Clinical epidemiology.

[2]  A. Oxman,et al.  Chapter 12: Interpreting results and drawing conclusions , 2011 .

[3]  J. Paling.  Strategies to help patients understand risks , 2003, BMJ : British Medical Journal.

[4]  Kristian Thorlund,et al.  The PRISMA Extension Statement for Reporting of Systematic Reviews Incorporating Network Meta-analyses of Health Care Interventions: Checklist and Explanations , 2015, Annals of Internal Medicine.

[5]  Helvi Kyngäs,et al.  The qualitative content analysis process. , 2008, Journal of advanced nursing.

[6]  Claire Glenton,et al.  Summary-of-findings tables in Cochrane reviews improved understanding and rapid retrieval of key information. , 2010, Journal of clinical epidemiology.

[7]  H. Schünemann,et al.  Improving GRADE evidence tables part 2: a systematic survey of explanatory notes shows more guidance is needed. , 2016, Journal of clinical epidemiology.

[8]  Alex J. Sutton,et al.  Evidence Synthesis for Decision Making 7 , 2013, Medical decision making : an international journal of the Society for Medical Decision Making.

[9]  G. Guyatt,et al.  GRADE guidelines: 13. Preparing summary of findings tables and evidence profiles-continuous outcomes. , 2013, Journal of clinical epidemiology.

[10]  John P.A. Ioannidis,et al.  Integration of evidence from multiple meta-analyses: a primer on umbrella reviews, treatment networks and multiple treatments meta-analyses , 2009, Canadian Medical Association Journal.

[11]  Phyllis Butow,et al.  A systematic review on communicating with patients about evidence. , 2006, Journal of evaluation in clinical practice.

[12]  G. Gigerenzer,et al.  Simple tools for understanding risks: from innumeracy to insight , 2003, BMJ : British Medical Journal.

[13]  Philippe Ravaud,et al.  Reporting of results from network meta-analyses: methodological systematic review , 2014, BMJ : British Medical Journal.

[14]  Robert A. Virzi,et al.  Refining the Test Phase of Usability Evaluation: How Many Subjects Is Enough? , 1992 .

[15]  V. Reyna.  CURRENT DIRECTIONS IN PSYCHOLOGICAL SCIENCE How People Make Decisions That Involve Risk A Dual-Processes Approach , 2022 .

[16]  Naihua Duan,et al.  Purposeful Sampling for Qualitative Data Collection and Analysis in Mixed Method Implementation Research , 2015, Administration and Policy in Mental Health and Mental Health Services Research.

[17]  Claire Glenton,et al.  User experiences of evidence-based online resources for health professionals: User testing of The Cochrane Library , 2008, BMC Medical Informatics Decis. Mak..

[18]  Hsiu-Fang Hsieh,et al.  Three Approaches to Qualitative Content Analysis , 2005, Qualitative health research.

[19]  Alex J. Sutton,et al.  Evidence Synthesis for Decision Making 2 , 2013, Medical decision making : an international journal of the Society for Medical Decision Making.

[20]  Mohammad Hassan Murad,et al.  A GRADE Working Group approach for rating the quality of treatment effect estimates from network meta-analysis , 2014, BMJ : British Medical Journal.

[21]  Claire Glenton,et al.  User testing and stakeholder feedback contributed to the development of understandable and useful Summary of Findings tables for Cochrane reviews. , 2010, Journal of clinical epidemiology.

[22]  Andrea Cipriani,et al.  Conceptual and Technical Challenges in Network Meta-analysis , 2013, Annals of Internal Medicine.

[23]  Deborah M Caldwell,et al.  Simultaneous comparison of multiple treatments: combining direct and indirect evidence , 2005, BMJ : British Medical Journal.

[24]  I. Lipkus.  Numeric, Verbal, and Visual Formats of Conveying Health Risks: Suggested Best Practices and Future Recommendations , 2007, Medical decision making : an international journal of the Society for Medical Decision Making.

[25]  Carl Heneghan,et al.  What is missing from descriptions of treatment in trials and reviews? , 2008, BMJ : British Medical Journal.

[26]  David C Hoaglin,et al.  Conducting indirect-treatment-comparison and network-meta-analysis studies: report of the ISPOR Task Force on Indirect Treatment Comparisons Good Research Practices: part 2. , 2011, Value in health : the journal of the International Society for Pharmacoeconomics and Outcomes Research.

[27]  Matthew B. Miles,et al.  Qualitative Data Analysis: An Expanded Sourcebook , 1994 .

[28]  M. Sandelowski.  Focus on Research Methods Whatever Happened to Qualitative Description? , 2022 .

[29]  Gordon H Guyatt,et al.  Advances in the GRADE approach to rate the certainty in estimates from a network meta-analysis. , 2018, Journal of clinical epidemiology.

[30]  Gordon H. Guyatt,et al.  Approaches to interpreting and choosing the best treatments in network meta-analyses , 2017, Systematic Reviews.

[31]  G. Guyatt,et al.  Improving GRADE evidence tables part 1: a randomized trial shows improved understanding of content in summary of findings tables with a new format. , 2016, Journal of clinical epidemiology.

[32]  Sylwia Bujkiewicz,et al.  Presentational approaches used in the UK for reporting evidence synthesis using indirect and mixed treatment comparisons , 2013, Journal of health services research & policy.

[33]  Philippe Ravaud,et al.  Uncertainty in Treatment Rankings: Reanalysis of Network Meta-analyses of Randomized Trials , 2016, Annals of Internal Medicine.

[34]  S. Sullivan,et al.  What Guidance Are Researchers Given on How to Present Network Meta-Analyses to End-Users such as Policymakers and Clinicians? A Systematic Review , 2014, PloS one.