Sensitivity Analysis in Cost-Effectiveness Studies

Cost-effectiveness analysis (CEA) is one of the main tools of economic evaluation. Every CEA is based on a number of assumptions, some of which may not be accurate, introducing uncertainty. Sensitivity analysis (SA) formalizes ways to measure and evaluate this uncertainty. Specific sources of uncertainty in CEA have been noted by various researchers. In this work, we consolidate across all sources of uncertainty, discuss the imbalanced attention to SA across different sources, and discuss criteria for conducting and reporting SA to help bridge the gap between guidelines and practice.Guidelines on how to perform SA have been published for many years in response to requests for greater standardization among researchers. Decision makers tasked with reviewing new health technologies also seem to appreciate the additional information conveyed by a robust SA, including the attention to important patient subgroups. Yet, past reviews have shown that there is a substantial gap between the guidelines’ suggestions and the quality of SA in the field. Past reviews have also focused on one or two but not all three sources of uncertainty. The objective of our work is to comprehensively review all different sources of uncertainty and provide a concise set of criteria for conducting and presenting SA, stratified by common modelling approaches, including decision analysis and regression models.We first provide an overview of the three sources of uncertainty in a CEA (parameter, structural and methodological), including patient heterogeneity. We then present results from a literature review of the conduct and reporting of SA based on 406 CEA articles published between 2000 and mid-2009. We find that a minority of papers addressed at least two of the three sources of uncertainty, with no change over time. On the other hand, the use of some sophisticated techniques, such as probabilistic SA, has surged over the past 10 years. Lastly, we identify criteria for reporting uncertainty-robust SA and also discuss how to conduct SA and how to improve the reporting of SA for decision makers. We recommend that researchers take a more comprehensive view of uncertainty when planning SA for an economic evaluation.

[1]  S. Pauker,et al.  The Markov Process in Medical Prognosis , 1983, Medical decision making : an international journal of the Society for Medical Decision Making.

[2]  A Briggs,et al.  Uncertainty in the economic evaluation of health care technologies: the role of sensitivity analysis. , 1994, Health economics.

[3]  A H Briggs,et al.  Handling uncertainty when performing economic evaluation of healthcare interventions. , 1999, Health technology assessment.

[4]  M. Parsonage,et al.  Discounting and health benefits. , 1992, Health economics.

[5]  Andrew H Briggs,et al.  Thinking outside the box: recent advances in the analysis and presentation of uncertainty in cost-effectiveness studies. , 2002, Annual review of public health.

[6]  M F Drummond,et al.  Users' Guides to the Medical Literature: XIII. How to Use an Article on Economic Analysis of Clinical Practice A. Are the Results of the Study Valid? , 1997 .

[7]  David J. Pasta,et al.  Probabilistic Sensitivity Analysis Incorporating the Bootstrap , 1999, Medical decision making : an international journal of the Society for Medical Decision Making.

[8]  Jing Jing Li,et al.  Using effectiveness and cost-effectiveness to make drug coverage decisions: a comparison of Britain, Australia, and Canada. , 2009, JAMA.

[9]  A. Briggs,et al.  Something old, something new, something borrowed, something blue: a framework for the marriage of health econometrics and cost-effectiveness analysis. , 2002, Health economics.

[10]  B. Schackman,et al.  How Often Do Sensitivity Analyses for Economic Parameters Change Cost-Utility Analysis Conclusions? , 2012, PharmacoEconomics.

[11]  M. Weinstein,et al.  Foundations of cost-effectiveness analysis for health and medical practices. , 1977, The New England journal of medicine.

[12]  San Antonio Annual Meeting of the Southwest Educational Research Association , 1999 .

[13]  A. Briggs Statistical approaches to handling uncertainty in health economic evaluation , 2004, European journal of gastroenterology & hepatology.

[14]  A Briggs,et al.  Sensitivity analysis in economic evaluation: a review of published studies. , 1995, Health economics.

[15]  W. Edmunds,et al.  Impact of Model, Methodological, and Parameter Uncertainty in the Economic Analysis of Vaccination Programs , 2006, Medical decision making : an international journal of the Society for Medical Decision Making.

[16]  J Lipscomb,et al.  Time Preference for Health in Cost-Effectiveness Analysis , 1989, Medical care.

[17]  P Barton,et al.  Sensitivity analysis in economic evaluation: an audit of NICE current practice and a review of its use and value in decision-making. , 2009, Health technology assessment.

[18]  J. B. Ramsey,et al.  Tests for Specification Errors in Classical Linear Least‐Squares Regression Analysis , 1969 .

[19]  Andrew H. Briggs,et al.  Handling Uncertainty in Cost-Effectiveness Models , 2000, PharmacoEconomics.

[20]  J. Richard Landis,et al.  Large sample variance of kappa in the case of different sets of raters. , 1979 .

[21]  Karl Claxton,et al.  Assessing Quality in Decision Analytic Cost-Effectiveness Models , 2000, PharmacoEconomics.

[22]  J. Seymour,et al.  Raising the quality of cost-utility analyses: lessons learnt and still to learn. , 1999, Health policy.

[23]  R. Engle Autoregressive conditional heteroscedasticity with estimates of the variance of United Kingdom inflation , 1982 .

[24]  Dennis G. Fryback,et al.  Assessing the effectiveness of health interventions for cost-effectiveness analysis , 1997, Journal of General Internal Medicine.

[25]  S. Goldfeld,et al.  Some Tests for Homoscedasticity , 1965 .

[26]  A F Lehman,et al.  Cost-effectiveness of assertive community treatment for homeless persons with severe mental illness , 1999, British Journal of Psychiatry.

[27]  S. R. Hill BMed PhD Fafphm,et al.  Use of pharmacoeconomics in prescribing research. Part 5: modelling – beyond clinical trials , 2003 .

[28]  M. Mcgrath Cost Effectiveness in Health and Medicine. , 1998 .

[29]  M G Myriam Hunink,et al.  Uncertainty and Patient Heterogeneity in Medical Decision Models , 2010, Medical decision making : an international journal of the Society for Medical Decision Making.

[30]  B. McNeil,et al.  Probabilistic Sensitivity Analysis Using Monte Carlo Simulation , 1985, Medical decision making : an international journal of the Society for Medical Decision Making.

[31]  S Bryan,et al.  The use of economic evaluations in NHS decision-making: a review and empirical investigation. , 2008, Health technology assessment.

[32]  M. Drummond,et al.  Users' guides to the medical literature. XIII. How to use an article on economic analysis of clinical practice. A. Are the results of the study valid? Evidence-Based Medicine Working Group. , 1997, JAMA.

[33]  M. Sculpher,et al.  The effectiveness and cost-effectiveness of minimal access surgery amongst people with gastro-oesophageal reflux disease - a UK collaborative study. The REFLUX trial. , 2008, Health technology assessment.

[34]  S. Golder,et al.  Good Practice Guidelines for Decision-Analytic Modelling in Health Technology Assessment , 2012, PharmacoEconomics.

[35]  T. Breurch,et al.  A simple test for heteroscedasticity and random coefficient variation (econometrica vol 47 , 1979 .

[36]  Peter J. Neumann,et al.  Using cost-effectiveness analysis to improve health care: opportunities and barriers. , 2004 .

[37]  Anil K. Bera,et al.  Efficient tests for normality, homoscedasticity and serial independence of regression residuals , 1980 .

[38]  Javier Soto,et al.  HEALTH ECONOMIC EVALUATIONS USING DECISION ANALYTIC MODELING , 2002, International Journal of Technology Assessment in Health Care.

[39]  T. Einarson,et al.  Sensitivity Analysis in Health Economic and Pharmacoeconomic Studies , 2012, PharmacoEconomics.

[40]  R. Tibshirani,et al.  The Measurement of Patients' Values in Medicine , 1982, Medical decision making : an international journal of the Society for Medical Decision Making.

[41]  R. D'Agostino,et al.  A Suggestion for Using Powerful and Informative Tests of Normality , 1990 .

[42]  M. Drummond,et al.  Economic Evaluation in Health Care: Merging Theory with Practice , 2002 .

[43]  Mark Sculpher,et al.  Subgroups and Heterogeneity in Cost-Effectiveness Analysis , 2012, PharmacoEconomics.

[44]  Joachim Zietz Heteroskedasticity and Neglected Parameter Heterogeneity , 2001 .

[45]  Jeffrey M. Woodbridge Econometric Analysis of Cross Section and Panel Data , 2002 .

[46]  M. Goetghebeur,et al.  Towards a European consensus on conducting and reporting health economic evaluations--a report from the ISPOR Inaugural European Conference. , 1999, Value in health : the journal of the International Society for Pharmacoeconomics and Outcomes Research.

[47]  Karl Claxton,et al.  Characterizing structural uncertainty in decision analytic models: a review and application of methods. , 2009, Value in health : the journal of the International Society for Pharmacoeconomics and Outcomes Research.

[48]  G A Colditz,et al.  Cost-Effectiveness and Cost-Benefit Analyses in the Medical Literature , 1992, Annals of Internal Medicine.

[49]  C. Dewa,et al.  Lessons from Trial-Based Cost-Effectiveness Analyses of Mental Health Interventions , 2012, PharmacoEconomics.

[50]  A E Ades,et al.  Evidence synthesis, parameter correlation and probabilistic sensitivity analysis. , 2006, Health economics.

[51]  Jeong-Wook Seo,et al.  The Korean journal of pathology is selected for coverage in science citation index expanded and journal citation reports by Thomson Reuters , 2008 .

[52]  Milton C Weinstein,et al.  Principles of good practice for decision analytic modeling in health-care evaluation: report of the ISPOR Task Force on Good Research Practices--Modeling Studies. , 2003, Value in health : the journal of the International Society for Pharmacoeconomics and Outcomes Research.

[53]  S. Sonnad,et al.  Methodologic Quality of Cost-Effectiveness Analyses of Surgical Procedures , 2007, Annals of surgery.

[54]  S Bryan,et al.  Rapid testing for group B streptococcus during labour: a test accuracy study with evaluation of acceptability and cost-effectiveness. , 2009, Health technology assessment.

[55]  J Hjelmgren,et al.  Health economic guidelines--similarities, differences and some implications. , 2001, Value in health : the journal of the International Society for Pharmacoeconomics and Outcomes Research.

[56]  T O Jefferson,et al.  Guidelines for authors and peer reviewers of economic submissions to the BMJ , 1996, BMJ.

[57]  G. Torrance Measurement of health state utilities for economic appraisal. , 1986, Journal of health economics.

[58]  S. Wilson Methods for the economic evaluation of health care programmes , 1987 .

[59]  Joachim Zietz Detecting neglected parameter heterogeneity with Chow tests , 2006 .

[60]  C. Bell,et al.  The Quality of Reporting in Published Cost-Utility Analyses, 19761997 , 2000, Annals of Internal Medicine.

[61]  R. Lopert,et al.  Use of pharmacoeconomics in prescribing research. Part 5: modelling – beyond clinical trials , 2003, Journal of clinical pharmacy and therapeutics.