A Framework for Addressing Structural Uncertainty in Decision Models

Decision analytic models used for health technology assessment are subject to uncertainties. These uncertainties can be quantified probabilistically, by placing distributions on model parameters and simulating from these to generate estimates of cost-effectiveness. However, many uncertain model choices, often termed structural assumptions, are usually only explored informally by presenting estimates of cost-effectiveness under alternative scenarios. The authors show how 2 recent research proposals represent parts of a framework to formally account for all common structural uncertainties. First, the model is expanded to include parameters that encompass all possible structural choices. Uncertainty can then arise because these parameters are estimated imprecisely from data, for example, a treatment effect of doubtful significance. Uncertainty can also arise if there are no relevant data. If there are relevant data, uncertainty can be addressed by averaging expected costs and effects generated from probabilistic analysis of the models with and without the parameter. The weights used for averaging are related to the predictive ability of each model, assessed against the data. If there are no data, additional parameters can often be informed by eliciting expert beliefs as probability distributions. These ideas are illustrated in decision models for antiplatelet therapies for vascular disease and new biologic drugs for the treatment of active psoriatic arthritis.

[1]  Jonathan Karnon,et al.  A Critique and Impact Analysis of Decision Modeling Assumptions , 2007, Medical decision making : an international journal of the Society for Medical Decision Making.

[2]  Jonathan Karnon,et al.  Alternative decision modelling techniques for the evaluation of health care technologies: Markov processes versus discrete event simulation. , 2003, Health economics.

[3]  L. Sharples,et al.  Bayesian evidence synthesis to extrapolate survival estimates in cost‐effectiveness studies , 2006, Statistics in medicine.

[4]  Miguel A Negrín,et al.  Incorporating model uncertainty in cost-effectiveness analysis: a Bayesian model averaging approach. , 2008, Journal of health economics.

[5]  Karl Claxton,et al.  Eliciting distributions to populate decision analytic models. , 2010, Value in health : the journal of the International Society for Pharmacoeconomics and Outcomes Research.

[6]  Louise B Russell,et al.  Comparing Model Structures in Cost-Effectiveness Analysis , 2005, Medical decision making : an international journal of the Society for Medical Decision Making.

[7]  David R. Anderson,et al.  Model selection and multimodel inference : a practical information-theoretic approach , 2003 .

[8]  Deborah M Caldwell,et al.  Simultaneous comparison of multiple treatments: combining direct and indirect evidence , 2005, BMJ : British Medical Journal.

[9]  David Draper,et al.  Assessment and Propagation of Model Uncertainty , 2011 .

[10]  M. Drummond,et al.  A rational framework for decision making by the National Institute For Clinical Excellence (NICE) , 2002, The Lancet.

[11]  Roger M. Cooke,et al.  Expert judgment in maintenance optimization , 1992 .

[12]  A A Stinnett,et al.  Estimating CE Ratios under Second-order Uncertainty , 1997, Medical decision making : an international journal of the Society for Medical Decision Making.

[13]  Karl Claxton,et al.  Characterizing structural uncertainty in decision analytic models: a review and application of methods. , 2009, Value in health : the journal of the International Society for Pharmacoeconomics and Outcomes Research.

[14]  M. Sculpher,et al.  Probabilistic sensitivity analysis for NICE technology assessment: not an optional extra. , 2005, Health economics.

[15]  S. Palmer,et al.  Clinical effectiveness and cost-effectiveness of clopidogrel and modified-release dipyridamole in the secondary prevention of occlusive vascular events: a systematic review and economic evaluation. , 2004, Health technology assessment.

[16]  Andrea Tancredi,et al.  A Bayesian model averaging approach for cost-effectiveness analyses. , 2009, Health economics.

[17]  Simon G Thompson,et al.  Structural and parameter uncertainty in Bayesian cost-effectiveness models , 2010, Journal of the Royal Statistical Society. Series C, Applied statistics.

[18]  H. Akaike,et al.  Information Theory and an Extension of the Maximum Likelihood Principle , 1973 .

[19]  M. Granger Morgan,et al.  Graphical Communication of Uncertain Quantities to Nontechnical People , 1987 .

[20]  Milton C Weinstein,et al.  Principles of good practice for decision analytic modeling in health-care evaluation: report of the ISPOR Task Force on Good Research Practices--Modeling Studies. , 2003, Value in health : the journal of the International Society for Pharmacoeconomics and Outcomes Research.

[21]  F. Harrell,et al.  Prognostic/Clinical Prediction Models: Multivariable Prognostic Models: Issues in Developing Models, Evaluating Assumptions and Adequacy, and Measuring and Reducing Errors , 2005 .

[22]  K Claxton,et al.  The irrelevance of inference: a decision-making approach to the stochastic evaluation of health care technologies. , 1999, Journal of health economics.

[23]  L. Sharples,et al.  Survival Models in Health Economic Evaluations: Balancing Fit and Parsimony to Improve Prediction , 2010, The international journal of biostatistics.

[24]  M. Clyde,et al.  Model Uncertainty , 2003 .

[25]  S. Chick,et al.  A taxonomy of model structures for economic evaluation of health technologies. , 2006, Health economics.

[26]  Simon G Thompson,et al.  Accounting for uncertainty in health economic decision models by using model averaging , 2009, Journal of the Royal Statistical Society. Series A,.

[27]  David J Spiegelhalter,et al.  Bayesian approaches to multiple sources of evidence and uncertainty in complex cost‐effectiveness modelling , 2003, Statistics in medicine.

[28]  R. Cooke Experts in Uncertainty: Opinion and Subjective Probability in Science , 1991 .

[29]  D. Clement A randomised, blinded, trial of Clopidogrel versus aspirin in patients at risk of ischaemic events (CAPRIE) , 1996 .

[30]  David J Spiegelhalter,et al.  Bias modelling in evidence synthesis , 2009, Journal of the Royal Statistical Society. Series A,.

[31]  Masahiro Kokai,et al.  Synapse alteration in hippocampal CA3 field following entorhinal cortex lesion , 1997, Journal of the Neurological Sciences.

[32]  Christian Genest,et al.  Combining Probability Distributions: A Critique and an Annotated Bibliography , 1986 .

[33]  Andrew H. Briggs,et al.  Handling Uncertainty in Cost-Effectiveness Models , 2000, PharmacoEconomics.

[34]  Simon G Thompson,et al.  Uncertainty and validation of health economic decision models. , 2009, Health economics.

[35]  Hans-Christoph Diener,et al.  European Stroke Prevention Study 2. Efficacy and safety data. , 1997, Journal of the neurological sciences.

[36]  David Schultz,et al.  A randomised, blinded, trial of clopidogrel versus aspirin in patients at risk of ischaemic events (CAPRIE) , 1996, The Lancet.

[37]  M. Sculpher,et al.  Decision Modelling for Health Economic Evaluation , 2006 .

[38]  David Turner,et al.  Comprehensive decision analytical modelling in economic evaluation: a Bayesian approach. , 2004, Health economics.

[39]  M Sculpher,et al.  Etanercept and infliximab for the treatment of psoriatic arthritis: a systematic review and economic evaluation. , 2006, Health technology assessment.