Advancing the WHO-INTEGRATE Framework as a Tool for Evidence-Informed, Deliberative Decision-Making Processes: Exploring the Views of Developers and Users of WHO Guidelines

BACKGROUND Decision-making on matters of public health and health policy is a deeply value-laden process. The World Health Organization (WHO)-INTEGRATE framework was proposed as a new evidence-to-decision (EtD) framework to support guideline development from a complexity perspective, notably in relation to public health and health system interventions, and with a foundation in WHO norms and values. This study was conducted as part of the development of the framework to assess its comprehensiveness and usefulness for public health and health policy decision-making. METHODS We conducted a qualitative study comprising nine key informant interviews (KIIs) with experts involved in WHO guideline development and four focus group discussions (FGDs) with a total of forty health decision-makers from Brazil, Germany, Nepal and Uganda. Transcripts were analyzed using MAXQDA12 and qualitative content analysis. RESULTS Most key informants and participants in the FGDs appreciated the framework for its relevance to real-world decision-making on four widely differing health topics. They praised its broad perspective and comprehensiveness with respect to new or expanded criteria, notably regarding societal implications, equity considerations, and acceptability. Some guideline developers questioned the value of the framework beyond current practice and were concerned with the complexity of applying such a broad range of criteria in guideline development processes. Participants made concrete suggestions for improving the wording and definitions of criteria as well as their grouping, for covering missing aspects, and for addressing overlap between criteria. CONCLUSION The framework was well-received by health decision-makers as well as the developers of WHO guidelines and appears to capture all relevant considerations discussed in four distinct real-world decision processes that took place on four different continents. Guidance is needed on how to apply the framework in guideline processes that are both transparent and participatory. A set of suggestions for improvement provides a valuable starting point for advancing the framework towards version 2.0.

[1]  P. Mayring.  Qualitative content analysis: theoretical foundation, basic procedures and software solution , 2014 .

[2]  G. Gigerenzer.  Decision Making: Nonrational Theories , 2001 .

[3]  Marcia Tummers,et al.  Value Assessment Frameworks for HTA Agencies: The Organization of Evidence-Informed Deliberative Processes. , 2017, Value in health : the journal of the International Society for Pharmacoeconomics and Outcomes Research.

[4]  David B Allison,et al.  White Hat Bias: Examples of its Presence in Obesity Research and a Call for Renewed Commitment to Faithfulness in Research Reporting , 2009, International Journal of Obesity.

[5]  Un Desa.  Transforming our world : The 2030 Agenda for Sustainable Development , 2016 .

[6]  Wolfgang Greiner,et al.  Barriers and Strategies in Guideline Implementation—A Scoping Review , 2016, Healthcare.

[7]  Renee F Wilson,et al.  Assessing the Impact of Economic Evidence on Policymakers in Health Care—A Systematic Review , 2012 .

[8]  Rob Baltussen,et al.  Balancing efficiency, equity and feasibility of HIV treatment in South Africa – development of programmatic guidance , 2013, Cost Effectiveness and Resource Allocation.

[9]  Sarah E. Rosenbaum,et al.  Policymakers’ and other stakeholders’ perceptions of key considerations for health system decisions and the presentation of evidence to inform those considerations: an international survey , 2013, Health Research Policy and Systems.

[10]  M. Petticrew,et al.  The need for a complex systems model of evidence for public health , 2017, The Lancet.

[11]  Rob Baltussen,et al.  Priority Setting for Universal Health Coverage: We Need Evidence-Informed Deliberative Processes, Not Just More Evidence on Cost-Effectiveness , 2016, International journal of health policy and management.

[12]  Mario Giampietro,et al.  What is wrong with evidence based policy, and how can it be improved? , 2017 .

[13]  Jane Noyes,et al.  Implications of a complexity perspective for systematic reviews and guideline development in health decision making , 2019, BMJ Global Health.

[14]  T. Tyler,et al.  Psychological perspectives on legitimacy and legitimation. , 2006, Annual review of psychology.

[15]  Lydia Kapiriri,et al.  Beyond cost-effectiveness, morbidity and mortality: a comprehensive evaluation of priority setting for HIV programming in Uganda , 2019, BMC Public Health.

[16]  Jerald Greenberg,et al.  Procedural Justice, Participation, and the Fair Process Effect in Groups and Organizations , 1983 .

[17]  H. Schünemann,et al.  [GRADE Evidence to Decision (EtD) frameworks: a systematic and transparent approach to making well informed healthcare choices. 1: Introduction.] , 2017, Recenti progressi in medicina.

[18]  Rob Baltussen,et al.  Translating international HIV treatment guidelines into local priorities in Indonesia , 2018, Tropical medicine & international health : TM & IH.

[19]  N. Daniels,et al.  The ethics of accountability in managed care reform. , 1998, Health affairs.

[20]  Alan Shiell,et al.  Theorising Interventions as Events in Systems , 2009, American journal of community psychology.

[21]  Sofia Tranæus,et al.  FRAMEWORK FOR SYSTEMATIC IDENTIFICATION OF ETHICAL ASPECTS OF HEALTHCARE TECHNOLOGIES: THE SBU APPROACH , 2015, International Journal of Technology Assessment in Health Care.

[22]  Narasimhan Manjulaa,et al.  Consolidated guideline on sexual and reproductive health and rights of women living with HIV. , 2017 .

[23]  Susan L Norris,et al.  Complex health interventions in complex systems: improving the process and methods for evidence-informed health decisions , 2019, BMJ Global Health.

[24]  Yuan Zhang,et al.  Development of rapid guidelines: 2. A qualitative study with WHO guideline developers , 2018, Health Research Policy and Systems.

[25]  I. Sanderson,et al.  Intelligent Policy Making for a Complex World: Pragmatism, Evidence and Learning , 2009 .

[26]  P. C. Nutt.  Surprising but true: Half the decisions in organizations fail , 1999 .

[27]  R. Baltussen,et al.  Development of the WHO-INTEGRATE evidence-to-decision framework: an overview of systematic reviews of decision criteria for health decision-making , 2020, Cost effectiveness and resource allocation : C/E.

[28]  Douglas K. Martin,et al.  Priority setting for new technologies in medicine: qualitative case study , 2000, BMJ : British Medical Journal.

[29]  R Robinson,et al.  Limits to rationality: economics, economists and priority setting. , 1999, Health policy.

[30]  Marc Fleurbaey,et al.  Guidance on priority setting in health care (GPS-Health): the inclusion of equity criteria not captured by cost-effectiveness analysis , 2014, Cost Effectiveness and Resource Allocation.

[31]  Steve McDonald,et al.  Towards a new model for producing evidence-based guidelines: a qualitative study of current approaches and opportunities for innovation among Australian guideline developers , 2019, F1000Research.

[32]  Becky Skidmore,et al.  Priority setting for health technology assessments: A systematic review of current practical approaches , 2007, International Journal of Technology Assessment in Health Care.

[33]  R. Baltussen,et al.  Priority setting of health interventions: the need for multi-criteria decision analysis , 2006, Cost effectiveness and resource allocation : C/E.

[34]  N. Daniels,et al.  Accountability for reasonableness. , 2000, BMJ.

[35]  C. Mitton,et al.  ‘Real-world’ health care priority setting using explicit decision criteria: a systematic review of the literature , 2015, BMC Health Services Research.

[36]  Marisa A. Ryan.  Adherence to Clinical Practice Guidelines , 2017, Otolaryngology--head and neck surgery : official journal of American Academy of Otolaryngology-Head and Neck Surgery.

[37]  C Ham,et al.  Priority setting in health care: learning from international experience. , 1997, Health policy.

[38]  J. P. Peña-Rosas,et al.  WHO recommendations on antenatal care for a positive pregnancy experience—going beyond survival , 2017, BJOG : an international journal of obstetrics and gynaecology.

[39]  Elly Stolk,et al.  Towards a multi-criteria approach for priority setting: an application to Ghana. , 2006, Health economics.

[40]  Douglas K. Martin,et al.  A Strategy to Improve Priority Setting in Developing Countries , 2007, Health Care Analysis.

[41]  Claire Glenton,et al.  The GRADE Evidence to Decision (EtD) framework for health system and public health decisions , 2018, Health Research Policy and Systems.

[42]  Sarah E. Rosenbaum,et al.  Developing and evaluating communication strategies to support informed decisions and practice based on evidence (DECIDE): protocol and preliminary results , 2013, Implementation Science.

[43]  M. Krahn,et al.  Health technology assessment: A comprehensive framework for evidence-based recommendations in Ontario , 2009, International Journal of Technology Assessment in Health Care.

[44]  T. Turner.  Developing evidence-based clinical practice guidelines in hospitals in Australia, Indonesia, Malaysia, the Philippines and Thailand: values, requirements and barriers , 2009, BMC health services research.

[45]  Lydia Kapiriri,et al.  Setting priorities for health interventions in developing countries: a review of empirical studies , 2009, Tropical medicine & international health : TM & IH.

[46]  Gordon H Guyatt,et al.  Progress in evidence-based medicine: a quarter century on , 2017, The Lancet.

[47]  Susan Michie,et al.  Assessing the complexity of interventions within systematic reviews: development, content and use of a new tool (iCAT_SR) , 2017, BMC Medical Research Methodology.

[48]  Gerald Gartlehner,et al.  [Communicating Risk in Public Health Emergencies: A WHO Guideline for Emergency Risk Communication (Erc) Policy and Practice]. , 2019, Gesundheitswesen (Bundesverband der Arzte des Offentlichen Gesundheitsdienstes (Germany)).

[49]  S Holm,et al.  The second phase of priority setting. Goodbye to the simple solutions: the second phase of priority setting in health care. , 1998, BMJ.

[50]  and Quality.  Assessing the Impact of Economic Evidence on Policymakers In Health Care - A Systematic Review , 2013 .

[51]  S. Hoffman,et al.  Institutionalising an evidence-informed approach to guideline development: progress and challenges at the World Health Organization , 2018, BMJ Global Health.

[52]  Paul Kind,et al.  From efficacy to equity: Literature review of decision criteria for resource allocation and healthcare decisionmaking , 2012, Cost Effectiveness and Resource Allocation.

[53]  Alex M. Friedman,et al.  Beyond Accountability for Reasonableness , 2008, Bioethics.

[54]  R. Baltussen,et al.  Priority setting using multiple criteria: should a lung health programme be implemented in Nepal? , 2007, Health policy and planning.

[55]  Arielle Lasry,et al.  Allocating funds for HIV/AIDS: a descriptive study of KwaDukuza, South Africa. , 2011, Health policy and planning.

[56]  A. Zwi,et al.  Factors influencing resource allocation decisions and equity in the health system of Ghana. , 2009, Public health.

[57]  N. Daniels.  Just Health: Meeting Health Needs Fairly , 2007 .

[58]  J. Bartram,et al.  Global Water, Sanitation, and Hygiene Research Priorities and Learning Challenges under Sustainable Development Goal 6 , 2018 .

[59]  Susan L Norris,et al.  The WHO-INTEGRATE evidence to decision framework version 1.0: integrating WHO norms and values and a complexity perspective , 2019, BMJ Global Health.