Development of a universal benefit-risk assessment framework and its application for regulatory agencies

The assessment of medicines has moved from efficacy and safety to that of a benefit-risk balance and regulatory agencies and pharmaceutical companies are improving their processes in order to achieve greater consistency and transparency in decision-making. However, their efforts are largely independent and do not address the lack of consistency in decisions by different countries, albeit for the same medicine, resulting in the potential inaccessibility of important medicines. The aim of this study was the development and validation of a universal benefit-risk framework for use by regulatory authorities. A questionnaire, specifically developed for this study, was used to evaluate the current approaches to benefit-risk assessment of medicines by 14 regulatory agencies and 24 pharmaceutical companies. None of the 11 agencies (79%) and 20 companies (83%) that responded used a fully quantitative approach, but the majority used a qualitative system for benefit-risk assessment. The development of a universal benefit-risk framework for use by both regulators and industry, with the involvement of all stakeholders, was supported by the study participants. A comparison of the existing benefit-risk assessment frameworks used by agencies and companies identified the common elements. As no major differences were observed, an 8-step universal framework was developed which incorporated the other frameworks. To support the framework in the assessment of benefits and risks, a template for documenting the benefit-risk decision together with a user manual was also developed. Four regulatory agencies conducted a retrospective pilot study to investigate the feasibility of this framework, the benefit-risk template and user manual. Subsequently, a prospective study was conducted by TGA of Australia, Health Canada and HSA of Singapore. The agencies found the benefit-risk template was ‘fit for purpose’ in terms of the relevance of information supporting the benefit-risk decision, the documentation and communication and the relative importance and values of the benefits and risks. The results showed that the benefit-risk summary template was adequate to document benefits and risks, relevant summaries and ii conclusions for the emerging markets. The applicability and validity of the summary component of the benefit-risk template was evaluated by sixteen HSA clinical reviewers in a retrospective study. They found that the BR Summary Template was adequate to document benefits, risks, relevant summaries and conclusions. However, a revision of the BR Summary Template should include technical improvements and more details of safety information. The BR Summary Template was thought to be a useful tool for communicating benefit-risk decisions to a variety of stakeholders. The formats of publicly available reports from major regulatory agencies were compared and found to be generally similar. When compared to the BR Template, the listing of benefits and risks, assigning of weights and values, visualisation and a more detailed, systematic standardised structure were found to be absent. This research has demonstrated that the 8-step universal framework is of value for the assessment of benefits and risks of medicines by regulatory agencies and the template was found to be useful for documenting and communicating benefit-risk decisions.

[1]  D L Sackett,et al.  An assessment of clinically useful measures of the consequences of treatment. , 1988, The New England journal of medicine.

[2]  B. Zikmund‐Fisher,et al.  The Effect of Format on Parents' Understanding of the Risks and Benefits of Clinical Research: A Comparison Between Text, Tables, and Graphics , 2010, Journal of health communication.

[3]  E. Kaplan,et al.  Nonparametric Estimation from Incomplete Observations , 1958 .

[4]  Rebecca Noel,et al.  Application of the Benefit-Risk Action Team (BRAT) Framework in Pharmaceutical R&D: Results From a Pilot Program , 2012 .

[5]  D. Sackett,et al.  The number needed to treat: a clinically useful measure of treatment effect , 1995, BMJ.

[6]  Mandy Ryan,et al.  Using discrete choice experiments to value health and health care , 2008 .

[7]  Theodor J. Stewart,et al.  Multiple Criteria Decision Analysis , 2001 .

[8]  L. Phillips,et al.  Benefits-risk assessment model for medicines: developing a structured approach to decision making , 2006 .

[9]  Paul Goodwin,et al.  Decision Analysis for Management Judgment , 1998 .

[10]  Mark J Sculpher,et al.  Using the incremental net benefit framework for quantitative benefit-risk analysis in regulatory decision-making--a case study of alosetron in irritable bowel syndrome. , 2010, Value in health : the journal of the International Society for Pharmacoeconomics and Outcomes Research.

[11]  W. Aspinall A route to more tractable expert advice , 2010, Nature.

[12]  J. Figueira,et al.  A survey on stochastic multicriteria acceptability analysis methods , 2008 .

[13]  Suzanne D. Pawlowski,et al.  The Delphi method as a research tool: an example, design considerations and applications , 2004, Inf. Manag..

[14]  M. Weinstein,et al.  QALYs: the basics. , 2009, Value in health : the journal of the International Society for Pharmacoeconomics and Outcomes Research.

[15]  H. Eichler,et al.  Innovative methods in drug regulatory sciences. , 2011, Drug discovery today. Technologies.

[16]  B S Levitan,et al.  Development of a Framework for Enhancing the Transparency, Reproducibility and Communication of the Benefit–Risk Balance of Medicines , 2011, Clinical pharmacology and therapeutics.

[17]  G. Naglie,et al.  How Well Do Guidelines Incorporate Evidence on Patient Preferences? , 2009, Journal of General Internal Medicine.

[18]  I. Lipkus Numeric, Verbal, and Visual Formats of Conveying Health Risks: Suggested Best Practices and Future Recommendations , 2007, Medical decision making : an international journal of the Society for Medical Decision Making.

[19]  Juhaeri Juhaeri,et al.  Benefit‐risk analysis: a proposal using quantitative methods , 2003, Pharmacoepidemiology and drug safety.

[20]  Juhaeri Juhaeri,et al.  Benefit‐risk analysis: examples using quantitative methods , 2003, Pharmacoepidemiology and drug safety.

[21]  Murray Turoff,et al.  The Delphi Method: Techniques and Applications , 1976 .

[22]  Larry D Lynd,et al.  Advances in risk-benefit evaluation using probabilistic simulation methods: an application to the prophylaxis of deep vein thrombosis. , 2004, Journal of clinical epidemiology.

[23]  Sam Salek,et al.  Retracted:A quantitative approach to benefit‐risk assessment of medicines — part 2: the practical application of a new model , 2007, Pharmacoepidemiology and drug safety.

[24]  J R Beck,et al.  Markov Models in Medical Decision Making , 1993, Medical decision making : an international journal of the Society for Medical Decision Making.

[25]  J. Landeta Current validity of the Delphi method in social sciences , 2006 .

[26]  Esther F. Schmid,et al.  Communicating the risks and benefits of medicines. , 2007, Drug discovery today.

[27]  S. Walker,et al.  Is there a need for a universal benefit–risk assessment framework for medicines? Regulatory and industry perspectives , 2013, Pharmacoepidemiology and drug safety.

[28]  R. Gelber,et al.  Evaluation of effectiveness: Q-TWiST , 1993 .

[29]  Jeremy E. Oakley,et al.  Uncertain Judgements: Eliciting Experts' Probabilities , 2006 .

[30]  Louis P Garrison,et al.  Assessing a structured, quantitative health outcomes approach to drug risk-benefit analysis. , 2007, Health affairs.

[31]  G. Lu,et al.  Combination of direct and indirect evidence in mixed treatment comparisons , 2004, Statistics in medicine.

[32]  I Zineh,et al.  Clinical Pharmacology and the Catalysis of Regulatory Science: Opportunities for the Advancement of Drug Development and Evaluation , 2013, Clinical pharmacology and therapeutics.

[33]  M Ryan,et al.  Methodological issues in the application of conjoint analysis in health care. , 1998, Health economics.

[34]  Andrew Stranieri,et al.  Does the Delphi process lead to increased accuracy in group-based judgmental forecasts or does it simply induce consensus amongst judgmental forecasters ? , 2011 .

[35]  K. Watterberg,et al.  Joint distribution approaches to simultaneously quantifying benefit and risk , 2006 .

[36]  Sam Salek,et al.  Benefit-Risk Appraisal of Medicines: A Systematic Approach to Decision-making , 2010 .

[37]  A. Dobson,et al.  Impact numbers in health policy decisions , 2002, Journal of epidemiology and community health.

[38]  Baruch Nevo,et al.  Face validity revisited. , 1985 .

[39]  John Doyle,et al.  A review of quantitative risk-benefit methodologies for assessing drug safety and efficacy-report of the ISPOR risk-benefit management working group. , 2010, Value in health : the journal of the International Society for Pharmacoeconomics and Outcomes Research.

[40]  Ralph L. Keeney,et al.  Book Reviews : Scientific Opportunities and Public Needs: Improv ing Priority Setting and Public Input at the National Institutes of Health. Institute of Medicine. Washington, DC: National Academy Press, 1998, 136 pages, $26.00 , 1998 .

[41]  Jeffrey S. Stonebraker How Bayer Makes Decisions to Develop New Drugs , 2002, Interfaces.

[42]  Carlos Martinez,et al.  Concepts in Risk-Benefit Assessment , 1996, Drug safety.

[43]  Gregory J. Skulmoski,et al.  Journal of Information Technology Education the Delphi Method for Graduate Research , 2022 .