Calculating Expected Value of Sample Information Adjusting for Imperfect Implementation

Background: The Expected Value of Sample Information (EVSI) calculates the value of collecting additional information through a study with a given design. Standard EVSI analyses assume that the treatment recommendations based on the new information will be implemented immediately and completely once the study has finished. However, treatment implementation is often slow and incomplete, giving a biased estimation of the study value. Previous methods have adjusted for this bias, but they typically make the unrealistic assumption that the study outcomes do not impact the implementation. One method does assume that the implementation is related to the strength of evidence in favour of the treatment but this method uses analytical results, which require alternative restrictive assumptions. Methods: We develop two implementation-adjusted EVSI calculation methods that relax these assumptions. The first method uses computationally demanding nested simulations, based on the definition of the implementation-adjusted EVSI. The second method aims to facilitate the computation by adapting a recently developed efficient EVSI computation method to adjust for imperfect implementation. The implementation-adjusted EVSI is then calculated with the two methods across three examples. Results: The maximum difference between the two methods remains at most 6% in all examples. The efficient computation method is between 6 and 60 times faster than the nested simulation method in this case study and could be used in practice. Conclusions: The methods developed in this paper calculate implementation-adjusted EVSI using realistic assumptions. The efficient estimation method is accurate and can estimate the implementation-adjusted EVSI in practice. By adapting standard EVSI estimation methods, we ensure that accurate adjustments for imperfect implementation can be made with the same computational cost as a standard EVSI analysis.