Eliciting judgements about dependent quantities of interest: The SHeffield ELicitation Framework extension and copula methods illustrated using an asthma case study

Pharmaceutical companies regularly need to make decisions about drug development programs based on the limited knowledge from early stage clinical trials. In this situation, eliciting the judgements of experts is an attractive approach for synthesising evidence on the unknown quantities of interest. When calculating the probability of success for a drug development program, multiple quantities of interest—such as the effect of a drug on different endpoints—should not be treated as unrelated. We discuss two approaches for establishing a multivariate distribution for several related quantities within the SHeffield ELicitation Framework (SHELF). The first approach elicits experts' judgements about a quantity of interest conditional on knowledge about another one. For the second approach, we first elicit marginal distributions for each quantity of interest. Then, for each pair of quantities, we elicit the concordance probability that both lie on the same side of their respective elicited medians. This allows us to specify a copula to obtain the joint distribution of the quantities of interest. We show how these approaches were used in an elicitation workshop that was performed to assess the probability of success of the registrational program of an asthma drug. The judgements of the experts, which were obtained prior to completion of the pivotal studies, were well aligned with the final trial results.

[1]  Björn Holzhauer,et al.  Improving the assessment of the probability of success in late stage drug development , 2021, Pharmaceutical statistics.

[2]  E. Bleecker,et al.  Effectiveness of fevipiprant in reducing exacerbations in patients with severe asthma (LUSTER-1 and LUSTER-2): two phase 3 randomised controlled trials. , 2020, The Lancet. Respiratory medicine.

[3]  Roger M. Cooke,et al.  Ice sheet contributions to future sea-level rise from structured expert judgment , 2019, Proceedings of the National Academy of Sciences.

[4]  Anthony O’Hagan,et al.  Expert Knowledge Elicitation: Subjective but Scientific , 2019, The American Statistician.

[5]  B. Knorr.  LUSTER-1 and -2: Two randomized controlled trials of the prostaglandin D2 receptor 2 antagonist, fevipiprant, in asthma , 2019 .

[6]  J. Corren,et al.  Tezepelumab in Adults with Uncontrolled Asthma. , 2019, The New England journal of medicine.

[7]  V. Backer,et al.  Effect of tralokinumab, an interleukin-13 neutralising monoclonal antibody, on eosinophilic airway inflammation in uncontrolled moderate-to-severe asthma (MESOS): a multicentre, double-blind, randomised, placebo-controlled phase 2 trial. , 2018, The Lancet. Respiratory medicine.

[8]  C. Brightling,et al.  Tralokinumab for severe, uncontrolled asthma (STRATOS 1 and STRATOS 2): two randomised, double-blind, placebo-controlled, phase 3 clinical trials. , 2018, The Lancet. Respiratory medicine.

[9]  I. Pavord,et al.  Dupilumab Efficacy and Safety in Moderate‐to‐Severe Uncontrolled Asthma , 2018, The New England journal of medicine.

[10]  Heinz Schmidli,et al.  Evidence synthesis from aggregate recurrent event data for clinical trial design and analysis , 2018, Statistics in medicine.

[11]  Nicky Best,et al.  Better decision making in drug development through adoption of formal prior elicitation , 2017, Pharmaceutical statistics.

[12]  Laura Bojke,et al.  Expert Elicitation to Inform Health Technology Assessment , 2018 .

[13]  John Paul Gosling,et al.  SHELF: The Sheffield Elicitation Framework , 2018 .

[14]  Oswaldo Morales-Nápoles,et al.  Eliciting Multivariate Uncertainty from Experts: Considerations and Approaches Along the Expert Judgement Process , 2018 .

[15]  A model to account for data dependency when estimating floral cover in different land use types over a season , 2017, Environmental and Ecological Statistics.

[16]  J. Oakley,et al.  EVidence Synthesis For Health Technology Assessment With Limited Studies , 2017 .

[17]  E. Bateman,et al.  Fevipiprant, an oral prostaglandin DP2 receptor (CRTh2) antagonist, in allergic asthma uncontrolled on low-dose inhaled corticosteroids , 2017, European Respiratory Journal.

[18]  Herbert Hoijtink,et al.  Application and Evaluation of an Expert Judgment Elicitation Procedure for Correlations , 2017, Front. Psychol..

[19]  E. Bleecker,et al.  Efficacy and safety of benralizumab for patients with severe asthma uncontrolled with high-dosage inhaled corticosteroids and long-acting β2-agonists (SIROCCO): a randomised, multicentre, placebo-controlled phase 3 trial , 2016, The Lancet.

[20]  W. Busse,et al.  Benralizumab, an anti-interleukin-5 receptor α monoclonal antibody, as add-on treatment for patients with severe, uncontrolled, eosinophilic asthma (CALIMA): a randomised, double-blind, placebo-controlled phase 3 trial , 2016, The Lancet.

[21]  Baldur P Magnusson,et al.  The oral CRTh2 antagonist QAW039 (fevipiprant): A phase II study in uncontrolled allergic asthma. , 2016, Pulmonary pharmacology & therapeutics.

[22]  E. Bateman,et al.  Reslizumab for inadequately controlled asthma with elevated blood eosinophil counts: results from two multicentre, parallel, double-blind, randomised, placebo-controlled, phase 3 trials. , 2015, The Lancet. Respiratory medicine.

[23]  Ian D Pavord,et al.  Mepolizumab treatment in patients with severe eosinophilic asthma. , 2014, The New England journal of medicine.

[24]  L. Boulet,et al.  Effects of an anti-TSLP antibody on allergen-induced asthmatic responses. , 2014, The New England journal of medicine.

[25]  Anthony O'Hagan,et al.  Eliciting expert judgements about a set of proportions , 2014 .

[26]  W. Busse,et al.  Effects of benralizumab on airway eosinophils in asthmatic patients with sputum eosinophilia. , 2013, The Journal of allergy and clinical immunology.

[27]  Ruksha Bhadresha.  Dupilumab in persistent asthma with elevated eosinophil levels , 2013, Thorax.

[28]  Will Usher,et al.  An expert elicitation of climate, energy and economic uncertainties , 2013 .

[29]  Fadlalla G. Elfadaly,et al.  Eliciting Dirichlet and Connor–Mosimann prior distributions for multinomial models , 2013 .

[30]  Simon Day,et al.  Structured approach to the elicitation of expert beliefs for a Bayesian‐designed clinical trial: a case study , 2013, Pharmaceutical statistics.

[31]  Gerard B. M. Heuvelink,et al.  Web-based tool for expert elicitation of the variogram , 2013, Comput. Geosci..

[32]  Ian D Pavord,et al.  Mepolizumab for severe eosinophilic asthma (DREAM): a multicentre, double-blind, placebo-controlled trial , 2012, The Lancet.

[33]  Andy Hart,et al.  Quantifying Experts’ Uncertainty About the Future Cost of Exotic Diseases , 2012, Risk Analysis.

[34]  N. Wilson,et al.  Use of sputum eosinophil counts to guide management in children with severe asthma , 2011, Thorax.

[35]  A. Chang,et al.  A systematic review and meta-analysis: tailoring asthma treatment on eosinophilic markers (exhaled nitric oxide or sputum eosinophils) , 2010, Thorax.

[36]  D. Spiegelhalter,et al.  Summarizing historical information on controls in clinical trials , 2010, Clinical trials.

[37]  Parameswaran Nair,et al.  Mepolizumab for prednisone-dependent asthma with sputum eosinophilia. , 2009, The New England journal of medicine.

[38]  John Quigley,et al.  Modelling the reliability of search and rescue operations with Bayesian Belief Networks , 2008, Reliab. Eng. Syst. Saf..

[39]  Pravin K. Trivedi,et al.  Copula Modeling: An Introduction for Practitioners , 2007 .

[40]  Jeremy E. Oakley,et al.  Uncertain Judgements: Eliciting Experts' Probabilities , 2006 .

[41]  J. Vondráček,et al.  Strategy Aimed at Reduction of Sputum Eosinophils Decreases Exacerbation Rate in Patients with Asthma , 2006, The Journal of international medical research.

[42]  L. Boulet,et al.  Determining asthma treatment by monitoring sputum cell counts: effect on exacerbations , 2006, European Respiratory Journal.

[43]  Roger M. Cooke,et al.  Probability Density Decomposition for Conditionally Dependent Random Variables Modeled by Vines , 2001, Annals of Mathematics and Artificial Intelligence.

[44]  I. Pavord,et al.  Asthma exacerbations and sputum eosinophil counts: a randomised controlled trial , 2002, The Lancet.

[45]  John Quigley,et al.  Bayesian belief nets for managing expert judgement and modelling reliability , 2001 .

[46]  Anthony O'Hagan,et al.  Quantifying expert opinion in the UK water industry: an experimental study , 2000 .

[47]  R. Pauwels,et al.  GLOBAL STRATEGY FOR ASTHMA MANAGEMENT AND PREVENTION , 1996 .

[48]  D. V. Gokhale,et al.  Assessment of a Prior Distribution for the Correlation Coefficient in a Bivariate Normal Distribution , 1982 .