Dimensions of Design Space: A Decision-Theoretic Approach to Optimal Research Design

Bayesian decision theory can be used not only to establish the optimal sample size and its allocation in a single clinical study but also to identify an optimal portfolio of research combining different types of study design. Within a single study, the highest societal payoff to proposed research is achieved when its sample sizes and allocation between available treatment options are chosen to maximize the expected net benefit of sampling (ENBS). Where a number of different types of study informing different parameters in the decision problem could be conducted, the simultaneous estimation of ENBS across all dimensions of the design space is required to identify the optimal sample sizes and allocations within such a research portfolio. This is illustrated through a simple example of a decision model of zanamivir for the treatment of influenza. The possible study designs include: 1) a single trial of all the parameters, 2) a clinical trial providing evidence only on clinical endpoints, 3) an epidemiological study of natural history of disease, and 4) a survey of quality of life. The possible combinations, samples sizes, and allocation between trial arms are evaluated over a range of cost-effectiveness thresholds. The computational challenges are addressed by implementing optimization algorithms to search the ENBS surface more efficiently over such large dimensions.

[1]  David B. Dunson,et al.  Bayesian Data Analysis , 2010 .

[2]  Sylvia Richardson,et al.  Markov Chain Monte Carlo in Practice , 1997 .

[3]  A A Stinnett,et al.  Net Health Benefits , 1998, Medical decision making : an international journal of the Society for Medical Decision Making.

[4]  A E Ades,et al.  Expected Value of Sample Information Calculations in Medical Decision Modeling , 2004, Medical decision making : an international journal of the Society for Medical Decision Making.

[5]  F. J. Anscombe,et al.  Probability and Statistics for Business Decisions: An Introduction to Managerial Economics under Uncertainty. , 1959 .

[6]  K. Chaloner,et al.  Bayesian Experimental Design: A Review , 1995 .

[7]  Laura Bojke,et al.  Priority setting for research in health care: An application of value of information analysis to glycoprotein IIb/IIIa antagonists in non-ST elevation acute coronary syndrome , 2006, International Journal of Technology Assessment in Health Care.

[8]  George Woodworth.  Modeling in Medical Decision Making: A Bayesian Approach , 2007 .

[9]  Robert Schlaifer.  Student's manual for probability and statistics for business decisions , 1959 .

[10]  J. Berger.  Statistical Decision Theory and Bayesian Analysis , 1988 .

[11]  James K. Hammitt,et al.  Research Planning for Food Safety , 1989 .

[12]  P. Armitage.  The search for optimality in clinical trials , 1985 .

[13]  K Claxton,et al.  The irrelevance of inference: a decision-making approach to the stochastic evaluation of health care technologies. , 1999, Journal of health economics.

[14]  K Claxton,et al.  Bayesian approaches to the value of information: implications for the regulation of new pharmaceuticals. , 1999, Health economics.

[15]  Fumie Yokota,et al.  Value of Information Literature Analysis: A Review of Applications in Health Risk Management , 2004, Medical decision making : an international journal of the Society for Medical Decision Making.

[16]  P. Younger.  The Elgar Companion to Health Economics , 2006 .

[17]  Andrew R Willan,et al.  Expected value of information and decision making in HTA. , 2007, Health economics.

[18]  Karl Claxton,et al.  Identifying Research Priorities: The Value of Information Associated with Repeat Screening for Age-Related Macular Degeneration , 2008, Medical decision making : an international journal of the Society for Medical Decision Making.

[19]  Andrew M. Jones.  The Elgar Companion to Health Economics , 2007 .

[20]  Ronald A. Howard,et al.  Information Value Theory , 1966, IEEE Trans. Syst. Sci. Cybern..

[21]  Peter Littlejohns,et al.  Making a decision to wait for more evidence: when the National Institute for Health and Clinical Excellence recommends a technology only in the context of research. , 2007, Journal of the Royal Society of Medicine.

[22]  Mark J Sculpher,et al.  Dangerous omissions: the consequences of ignoring decision uncertainty. , 2011, Health economics.

[23]  Howard Raiffa,et al.  Applied Statistical Decision Theory. , 1961 .

[24]  Stephen J. Wright,et al.  Numerical Optimization , 2018, Fundamental Statistical Inference.

[25]  Jeremy E. Oakley,et al.  Decision-Theoretic Sensitivity Analysis for Complex Computer Models , 2009, Technometrics.

[26]  Andrew R Willan,et al.  Time and expected value of sample information wait for no patient. , 2008, Value in health : the journal of the International Society for Pharmacoeconomics and Outcomes Research.

[27]  H. Raiffa,et al.  Introduction to Statistical Decision Theory , 1996 .

[28]  Gordon B. Hazen,et al.  Sensitivity Analysis and the Expected Value of Perfect Information , 1998, Medical decision making : an international journal of the Society for Medical Decision Making.

[29]  K M Thompson,et al.  A dynamic programming approach to the efficient design of clinical trials. , 2001, Journal of health economics.

[30]  M J Al,et al.  Costs, effects and C/E-ratios alongside a clinical trial. , 1994, Health economics.

[31]  Hamid Pezeshk,et al.  Bayesian techniques for sample size determination in clinical trials: a short review , 2003, Statistical methods in medical research.

[32]  Nicky J Welton,et al.  Preventive strategies for group B streptococcal and other bacterial infections in early infancy: cost effectiveness and value of information analyses , 2007, BMJ : British Medical Journal.

[33]  Eric R. Ziegel,et al.  Modeling in Medical Decision Making , 2002, Technometrics.

[34]  M D Stevenson,et al.  Gaussian Process Modeling in Conjunction with Individual Patient Simulation Modeling: A Case Study Describing the Calculation of Cost-Effectiveness Ratios for the Treatment of Established Osteoporosis , 2004, Medical decision making : an international journal of the Society for Medical Decision Making.

[35]  Karl Claxton,et al.  An Iterative Bayesian Approach to Health Technology Assessment: Application to a Policy of Preoperative Optimization for Patients Undergoing Major Elective Surgery , 2006, Medical decision making : an international journal of the Society for Medical Decision Making.

[36]  C. Gamble.  Zanamivir for the treatment of influenza in adults: a systematic review and economic evaluation , 2002 .

[37]  John D. Graham,et al.  Going beyond the single number: Using probabilistic risk assessment to improve risk management , 1996 .

[38]  M. Drummond,et al.  Health Care Technology: Effectiveness, Efficiency and Public Policy@@@Methods for the Economic Evaluation of Health Care Programmes , 1988 .

[39]  M. Sculpher,et al.  Representing uncertainty: the role of cost-effectiveness acceptability curves. , 2001, Health economics.

[40]  T. Jefferson,et al.  Population-Based Study on Incidence, Risk Factors, Clinical Complications and Drug Utilisation Associated with Influenza in the United Kingdom , 2000, European Journal of Clinical Microbiology and Infectious Diseases.

[41]  Alan Brennan,et al.  Efficient computation of partial expected value of sample information using Bayesian approximation. , 2007, Journal of health economics.

[42]  J. Bather.  Randomized Allocation of Treatments in Sequential Experiments , 1981 .

[43]  K Claxton,et al.  An economic approach to clinical trial design and research priority-setting. , 1996, Health economics.

[44]  Giovanni Parmigiani,et al.  Modeling in Medical Decision Making: A Bayesian Approach , 2002 .

[45]  Stephen Palmer,et al.  The Half-Life of Truth: What Are Appropriate Time Horizons for Research Decisions? , 2008, Medical decision making : an international journal of the Society for Medical Decision Making.

[46]  M. Zelen,et al.  Play the Winner Rule and the Controlled Clinical Trial , 1969 .

[47]  Alan Brennan,et al.  Expected value of sample information for Weibull survival data. , 2007, Health economics.

[48]  Alexander Shapiro,et al.  Stochastic programming by Monte Carlo simulation methods , 2000 .

[49]  Howard Raiffa,et al.  Applied Statistical Decision Theory. , 1961 .

[50]  Nicky J Welton,et al.  Exploring the Research Decision Space: The Expected Value of Information for Sequential Research Designs , 2010, Medical decision making : an international journal of the Society for Medical Decision Making.

[51]  M. Sculpher,et al.  Decision Modelling for Health Economic Evaluation , 2006 .

[52]  M C Weinstein,et al.  The FDA's regulation of health economic information. , 2000, Health affairs.

[53]  Kimberly M. Thompson,et al.  The Value of Improved National Exposure Information for Perchloroethylene (Perc): A Case Study for Dry Cleaners , 1997 .

[54]  Andrew R Willan,et al.  The value of information and optimal clinical trial design , 2005, Statistics in medicine.

[55]  Matt Stevenson,et al.  Monte Carlo probabilistic sensitivity analysis for patient level simulation models: efficient estimation of mean and variance using ANOVA. , 2007, Health economics.

[56]  Joshua T. Cohen,et al.  When is evidence sufficient? , 2005, Health affairs.

[57]  S. Palmer,et al.  Incorporating option values into the economic evaluation of health care technologies. , 2000, Journal of health economics.