A guide to value of information methods for prioritising research in health impact modelling

Health impact simulation models are used to predict how a proposed intervention or scenario will affect public health outcomes, based on available data and knowledge of the process. The outputs of these models are uncertain due to uncertainty in the structure and inputs to the model. In order to assess the extent of uncertainty in the outcome we must quantify all potentially relevant uncertainties. Then to reduce uncertainty we should obtain and analyse new data, but it may be unclear which parts of the model would benefit from such extra research. This paper presents methods for uncertainty quantification and research prioritisation in health impact models based on Value of Information (VoI) analysis. Specifically, we 1. discuss statistical methods for quantifying uncertainty in this type of model, given the typical kinds of data that are available, which are often weaker than the ideal data that are desired; 2. show how the expected value of partial perfect information (EVPPI) can be calculated to compare how uncertainty in each model parameter influences uncertainty in the output; 3. show how research time can be prioritised efficiently, in the light of which components contribute most to outcome uncertainty. The same methods can be used whether the purpose of the model is to estimate quantities of interest to a policy maker, or to explicitly decide between policies. We demonstrate how these methods might be used in a model of the impact of air pollution on health outcomes.

[1]  Elisabeth Fenwick,et al.  Value of Information Analysis for Research Decisions-An Introduction: Report 1 of the ISPOR Value of Information Analysis Emerging Good Practices Task Force. , 2020, Value in health : the journal of the International Society for Pharmacoeconomics and Outcomes Research.

[2]  Andrew Gelman,et al.  Data Analysis Using Regression and Multilevel/Hierarchical Models , 2006 .

[3]  Paola Annoni,et al.  Variance based sensitivity analysis of model output. Design and estimator for the total sensitivity index , 2010, Comput. Phys. Commun..

[4]  Karl Claxton,et al.  A Framework for Addressing Structural Uncertainty in Decision Models , 2011, Medical decision making : an international journal of the Society for Medical Decision Making.

[5]  Mark Strong,et al.  Computing the Expected Value of Sample Information Efficiently: Practical Guidance and Recommendations for Four Model-Based Methods. , 2020, Value in health : the journal of the International Society for Pharmacoeconomics and Outcomes Research.

[6]  I. Sobola,et al.  Global sensitivity indices for nonlinear mathematical models and their Monte Carlo estimates , 2001 .

[7]  M. Givoni,et al.  Health Impact Modelling of Active Travel Visions for England and Wales Using an Integrated Transport and Health Impact Modelling Tool (ITHIM) , 2013, PloS one.

[8]  A. de Nazelle,et al.  Estimating traffic contribution to particulate matter concentration in urban areas using a multilevel Bayesian meta-regression approach , 2020, Environment International.

[9]  Michael Brauer,et al.  Data integration model for air quality: a hierarchical approach to the global estimation of exposures to ambient air pollution , 2016, 1609.00141.

[10]  M. Sculpher,et al.  Decision Modelling for Health Economic Evaluation , 2006 .

[11]  Adalgiza Fornaro,et al.  Vehicle emissions and PM2.5 mass concentrations in six Brazilian cities , 2010, Air Quality, Atmosphere & Health.

[12]  Elisabeth Fenwick,et al.  Value of Information Analytical Methods: Report 2 of the ISPOR Value of Information Analysis Emerging Good Practices Task Force. , 2020, Value in health : the journal of the International Society for Pharmacoeconomics and Outcomes Research.

[13]  Carlos Monteiro,et al.  Health impact modelling of different travel patterns on physical activity, air pollution and road injuries for São Paulo, Brazil , 2017, Environment international.

[14]  M. Jetté,et al.  Metabolic equivalents (METS) in exercise testing, exercise prescription, and evaluation of functional capacity , 1990, Clinical cardiology.

[15]  Jiqiang Guo,et al.  Stan: A Probabilistic Programming Language. , 2017, Journal of statistical software.

[16]  Alan D. Lopez,et al.  A comparative risk assessment of burden of disease and injury attributable to 67 risk factors and risk factor clusters in 21 regions, 1990–2010: a systematic analysis for the Global Burden of Disease Study 2010 , 2012, The Lancet.

[17]  Anna Goodman,et al.  The current and potential health benefits of the National Health Service Health Check cardiovascular disease prevention programme in England: A microsimulation study , 2018, PLoS medicine.

[18]  James Woodcock,et al.  Physical activity and incident type 2 diabetes mellitus: a systematic review and dose–response meta-analysis of prospective cohort studies , 2016, Diabetologia.

[19]  Fumie Yokota,et al.  Value of Information Analysis in Environmental Health Risk Management Decisions: Past, Present, and Future , 2004, Risk analysis : an official publication of the Society for Risk Analysis.

[20]  Joseph Hilbe,et al.  Data Analysis Using Regression and Multilevel/Hierarchical Models , 2009 .

[21]  Luc Int Panis,et al.  Health impact assessment of active transportation: A systematic review. , 2015, Preventive medicine.

[22]  Gianluca Baio,et al.  Estimating the expected value of partial perfect information in health economic evaluations using integrated nested Laplace approximation , 2015, Statistics in medicine.

[23]  Raymond J Carroll,et al.  A Bayesian multilevel model for estimating the diet/disease relationship in a multicenter study with exposures measured with error: The EPIC study , 2008, Statistics in medicine.

[24]  Markus Amann,et al.  Contributions to cities' ambient particulate matter (PM): a systematic review of local source contributions at global level , 2015 .

[25]  Jonathan Karnon,et al.  Model Parameter Estimation and Uncertainty: a Report of the Ispor-smdm Modeling Good Research Practices Task Force-6 Background to the Task Forcemodel-parameter-estimation-and- Uncertainty-analysis.asp). a Summary of These Articles Was Pre- Sented at a Plenary Session at the Ispor 16th Annual Intern , 2022 .

[26]  Emanuele Borgonovo,et al.  Probabilistic sensitivity measures as information value , 2021, Eur. J. Oper. Res..

[27]  James Damsere-Derry,et al.  Health impacts of changes in travel patterns in Greater Accra Metropolitan Area, Ghana. , 2021, Environment international.

[28]  John S. Evans,et al.  Uncertainty in mortality response to airborne fine particulate matter: Combining European air pollution experts , 2008, Reliab. Eng. Syst. Saf..

[29]  A. O'Hagan,et al.  Probabilistic sensitivity analysis of complex models: a Bayesian approach , 2004 .

[30]  R. Burnett,et al.  Confounding and exposure measurement error in air pollution epidemiology , 2011, Air Quality, Atmosphere & Health.

[31]  Saravanan Arunachalam,et al.  Bayesian maximum entropy integration of ozone observations and model predictions: an application for attainment demonstration in North Carolina. , 2010, Environmental science & technology.

[32]  M. Drummond,et al.  A rational framework for decision making by the National Institute For Clinical Excellence (NICE) , 2002, The Lancet.

[33]  S. Wood Generalized Additive Models: An Introduction with R, Second Edition , 2017 .

[34]  G. Guyatt,et al.  GRADE guidelines: 8. Rating the quality of evidence--indirectness. , 2011, Journal of clinical epidemiology.

[35]  Santos Junior,et al.  Emissões veiculares em São Paulo: quantificação de fontes com modelos receptores e caracterização do material carbonáceo , 2015 .

[36]  J. Friedman Multivariate adaptive regression splines , 1990 .

[37]  M. A. Best Bayesian Approaches to Clinical Trials and Health‐Care Evaluation , 2005 .

[38]  Mike Paulden,et al.  One-Way Sensitivity Analysis for Probabilistic Cost-Effectiveness Analysis: Conditional Expected Incremental Net Benefit , 2019, PharmacoEconomics.

[39]  S Greenland,et al.  Basic methods for sensitivity analysis of biases. , 1996, International journal of epidemiology.

[40]  Jeremy E. Oakley,et al.  Uncertain Judgements: Eliciting Experts' Probabilities , 2006 .

[41]  S. Chick,et al.  A taxonomy of model structures for economic evaluation of health technologies. , 2006, Health economics.

[42]  Anthony N. Pettitt,et al.  A Review of Modern Computational Algorithms for Bayesian Optimal Design , 2016 .

[43]  Andrew B. Lawson,et al.  Bayesian Disease Mapping: Hierarchical Modeling in Spatial Epidemiology , 2008 .

[44]  Sylvia Richardson,et al.  Bayesian hierarchical models in ecological studies of health–environment effects , 2003 .

[45]  Yizheng Wu,et al.  Modeling health equity in active transportation planning , 2019, Transportation Research Part D: Transport and Environment.

[46]  Joel Schwartz,et al.  Measurement error caused by spatial misalignment in environmental epidemiology. , 2009, Biostatistics.

[47]  David J Spiegelhalter,et al.  Bias modelling in evidence synthesis , 2009, Journal of the Royal Statistical Society. Series A,.

[48]  Richard F MacLehose,et al.  Good practices for quantitative bias analysis. , 2014, International journal of epidemiology.

[49]  Jeremy E. Oakley,et al.  Estimating the Expected Value of Sample Information Using the Probabilistic Sensitivity Analysis Sample , 2015, Medical decision making : an international journal of the Society for Medical Decision Making.

[50]  Oliver O’Brien,et al.  Health effects of the London bicycle sharing system: health impact modelling study , 2014, BMJ : British Medical Journal.

[51]  Michael Brauer,et al.  An Integrated Risk Function for Estimating the Global Burden of Disease Attributable to Ambient Fine Particulate Matter Exposure , 2014, Environmental health perspectives.

[52]  Mark Jit,et al.  Quantifying Parameter and Structural Uncertainty of Dynamic Disease Transmission Models Using MCMC , 2015, Medical decision making : an international journal of the Society for Medical Decision Making.

[53]  Andrew B Lawson,et al.  Bayesian latent structure modeling of walking behavior in a physical activity intervention , 2016, Statistical methods in medical research.

[54]  J. Wakefield,et al.  Spatial epidemiology: methods and applications. , 2000 .

[55]  Jeremy E. Oakley,et al.  Managing structural uncertainty in health economic decision models: a discrepancy approach , 2012 .

[56]  Saltelli Andrea,et al.  Global Sensitivity Analysis: The Primer , 2008 .

[57]  Jeremy E. Oakley,et al.  When Is a Model Good Enough? Deriving the Expected Value of Model Improvement via Specifying Internal Model Discrepancies , 2014, SIAM/ASA J. Uncertain. Quantification.

[58]  James Woodcock,et al.  Can air pollution negate the health benefits of cycling and walking? , 2016, Preventive medicine.

[59]  Christopher Jackson,et al.  Value of Information: Sensitivity Analysis and Research Design in Bayesian Evidence Synthesis , 2017, Journal of the American Statistical Association.