Adjusting Estimates of the Expected Value of Information for Implementation

Background: Value of information (VoI) calculations give the expected benefits of decision making under perfect information (EVPI) or sample information (EVSI), typically on the premise that any treatment recommendations made in light of this information will be implemented instantly and fully. This assumption is unlikely to hold in health care; evidence shows that obtaining further information typically leads to “improved” rather than “perfect” implementation. Objectives: To present a method of calculating the expected value of further research that accounts for the reality of improved implementation. Methods: This work extends an existing conceptual framework by introducing additional states of the world regarding information (sample information, in addition to current and perfect information) and implementation (improved implementation, in addition to current and optimal implementation). The extension allows calculating the “implementation-adjusted” EVSI (IA-EVSI), a measure that accounts for different degrees of implementation. Calculations of implementation-adjusted estimates are illustrated under different scenarios through a stylized case study in non–small cell lung cancer. Results: In the particular case study, the population values for EVSI and IA-EVSI were £25 million and £8 million, respectively; thus, a decision assuming perfect implementation would have overestimated the expected value of research by about £17 million. IA-EVSI was driven by the assumed time horizon and, importantly, the specified rate of change in implementation: the higher the rate, the greater the IA-EVSI and the lower the difference between IA-EVSI and EVSI. Conclusions: Traditionally calculated measures of population VoI rely on unrealistic assumptions about implementation. This article provides a simple framework that accounts for improved, rather than perfect, implementation and offers more realistic estimates of the expected value of research.

[1]  David O Meltzer,et al.  Minimal Modeling Approaches to Value of Information Analysis for Health Research , 2011, Medical decision making : an international journal of the Society for Medical Decision Making.

[2]  N. Waugh,et al.  A rapid and systematic review of the clinical effectiveness and cost-effectiveness of paclitaxel, docetaxel, gemcitabine and vinorelbine in non-small-cell lung cancer. , 2001, Health technology assessment.

[3]  Karl Claxton,et al.  Unrelated Future Costs and Unrelated Future Benefits: Reflections on NICE Guide to the Methods of Technology Appraisal. , 2016, Health economics.

[4]  D. Lowe,et al.  To what extent are national guidelines for the management of urinary incontinence in women adhered? Data from a national audit , 2011, BJOG : an international journal of obstetrics and gynaecology.

[5]  A. Jemal,et al.  Cancer statistics, 2015 , 2015, CA: a cancer journal for clinicians.

[6]  Rahber Thariani,et al.  Value-of-Information Analysis within a Stakeholder-Driven Research Prioritization Process in a US Setting: An Application in Cancer Genomics , 2013, Medical decision making : an international journal of the Society for Medical Decision Making.

[7]  D. Cahill,et al.  Trends in procedures for infertility and caesarean sections: was NICE disinvestment guidance implemented? NICE recommendation reminders , 2013, BMC Public Health.

[8]  Jonathan Karnon,et al.  Planning the efficient allocation of research funds: an adapted application of a non-parametric Bayesian value of information analysis. , 2002, Health policy.

[9]  Karl Claxton,et al.  The Value of Implementation and the Value of Information: Combined and Uneven Development , 2008, Medical decision making : an international journal of the Society for Medical Decision Making.

[10]  D. Hosmer,et al.  The impact of clinical trials on the use of medications for acute myocardial infarction. Results of a community-based study. , 1996, Archives of internal medicine.

[11]  K Claxton,et al.  The irrelevance of inference: a decision-making approach to the stochastic evaluation of health care technologies. , 1999, Journal of health economics.

[12]  L. Jackson,et al.  Multi-level factors influence the implementation and use of complex innovations in cancer care: a multiple case study of synoptic reporting , 2014, Implementation Science.

[13]  M. Drummond,et al.  Assessing the costs and benefits of medical research: the diabetic retinopathy study. , 1992, Social science & medicine.

[14]  Karl Claxton,et al.  Value of information and value of implementation: application of an analytic framework to inform resource allocation decisions in metastatic hormone-refractory prostate cancer. , 2009, Value in health : the journal of the International Society for Pharmacoeconomics and Outcomes Research.

[15]  Sox Hc,et al.  Setting Priorities for Health Technology Assessment: A Model Process , 1992 .

[16]  P. Jones.  Making Decisions , 1971, Nature.

[17]  N. Dubrawsky.  Cancer statistics , 1989, CA: a cancer journal for clinicians.

[18]  G. Lamas,et al.  Do the results of randomized clinical trials of cardiovascular drugs influence medical practice? The SAVE Investigators. , 1992, The New England journal of medicine.

[19]  A E Ades,et al.  Expected Value of Sample Information Calculations in Medical Decision Modeling , 2004, Medical decision making : an international journal of the Society for Medical Decision Making.

[20]  D M Eddy,et al.  Selecting Technologies for Assessment , 1989, International Journal of Technology Assessment in Health Care.

[21]  M. Aristides,et al.  Retrospective cost analysis of gemcitabine in combination with cisplatin in non-small cell lung cancer compared to other combination therapies in Europe. , 2004, Lung cancer.

[22]  J Chilcott,et al.  The role of modelling in prioritising and planning clinical trials. , 2003, Health technology assessment.

[23]  M. Sculpher,et al.  Using Value of Information Analysis to Prioritise Health Research , 2012, PharmacoEconomics.

[24]  A. Rubinstein,et al.  Influence of new evidence on prescription patterns. , 2002, The Journal of the American Board of Family Practice.

[25]  J. Grimshaw,et al.  From best evidence to best practice: effective implementation of change in patients' care , 2003, The Lancet.

[26]  D. Berwick.  Disseminating innovations in health care. , 2003, JAMA.

[27]  Rachael L Fleurence,et al.  Setting priorities for research: a practical application of 'payback' and expected value of information. , 2007, Health economics.

[28]  E. Rogers,et al.  Diffusion of innovations , 1964, Encyclopedia of Sport Management.

[29]  The influence of NICE guidance on the uptake of laparoscopic surgery for colorectal cancer. , 2009, Journal of public health.

[30]  D. Meltzer,et al.  Toward a Science of Research Prioritization? The Use of Value of Information by Multidisciplinary Stakeholder Groups , 2013, Medical decision making : an international journal of the Society for Medical Decision Making.

[31]  M S Thompson,et al.  Decision-Analytic Determination of Study Size , 1981, Medical decision making : an international journal of the Society for Medical Decision Making.

[32]  Louise Fitzgerald,et al.  Innovation in healthcare: how does credible evidence influence professionals? , 2003, Health & social care in the community.

[33]  B. McCormack,et al.  An exploration of the factors that influence the implementation of evidence into practice. , 2004, Journal of clinical nursing.

[34]  Howard Raiffa,et al.  Decision Analysis: Introductory Lectures on Choices Under Uncertainty , 1968 .

[35]  T. Dean,et al.  An evaluation of the impact of NICE guidance on GP prescribing. , 2004, The British journal of general practice : the journal of the Royal College of General Practitioners.

[36]  P. Zatloukal,et al.  Gemcitabine plus cisplatin vs. gemcitabine plus carboplatin in stage IIIb and IV non-small cell lung cancer: a phase III randomized trial. , 2003, Lung cancer.

[37]  M. Stafford,et al.  Health state utilities for non small cell lung cancer , 2008, Health and quality of life outcomes.

[38]  M. Drummond,et al.  PRIORITIZING INVESTMENTS IN HEALTH TECHNOLOGY ASSESSMENT , 2000, International Journal of Technology Assessment in Health Care.

[39]  R. Grol.  Successes and Failures in the Implementation of Evidence-Based Guidelines for Clinical Practice , 2001, Medical care.

[40]  Andrew R Willan,et al.  Optimal clinical trial design using value of information methods with imperfect implementation. , 2009, Health economics.

[41]  Igor Linkov,et al.  The value of information for managing contaminated sediments. , 2014, Environmental science & technology.

[42]  H Raiffa,et al.  Decision Analysis: Introductory Lectures on Choices under Uncertainty. , 1969 .

[43]  A S Detsky,et al.  Using cost-effectiveness analysis to improve the efficiency of allocating funds to clinical trials. , 1990, Statistics in medicine.

[44]  J Townsend,et al.  Cost effectiveness scenario analysis for a proposed trial of hormone replacement therapy. , 1997, Health policy.

[45]  J M Grimshaw,et al.  Effectiveness and efficiency of guideline dissemination and implementation strategies , 2004, International Journal of Technology Assessment in Health Care.

[46]  Howard Raiffa,et al.  Decision analysis: introductory lectures on choices under uncertainty. 1968. , 1969, M.D.Computing.