On Prediction and Tolerance Intervals for Dynamic Treatment Regimes

We develop and evaluate tolerance interval methods for dynamic treatment regimes (DTRs) that can provide more detailed prognostic information to patients who will follow an estimated optimal regime. Although the problem of constructing confidence intervals for DTRs has been extensively studied, prediction and tolerance intervals have received little attention. We begin by reviewing in detail different interval estimation and prediction methods and then adapting them to the DTR setting. We illustrate some of the challenges associated with tolerance interval estimation stemming from the fact that we do not typically have data that were generated from the estimated optimal regime. We give an extensive empirical evaluation of the methods and discussed several practical aspects of method choice, and we present an example application using data from a clinical trial. Finally, we discuss future directions within this important emerging area of DTR research.

[1]  E. Moodie,et al.  A note on the variance of doubly-robust G-estimators , 2009 .

[2]  Peter F Thall,et al.  Optimization of multi‐stage dynamic treatment regimes utilizing accumulated data , 2015, Statistics in medicine.

[3]  Aurélien Garivier,et al.  On the Complexity of Best-Arm Identification in Multi-Armed Bandit Models , 2014, J. Mach. Learn. Res..

[4]  B. Chakraborty,et al.  Statistical Methods for Dynamic Treatment Regimes: Reinforcement Learning, Causal Inference, and Personalized Medicine , 2013 .

[5]  Inbal Nahum-Shani,et al.  Q-learning: a data analysis method for constructing adaptive interventions. , 2012, Psychological methods.

[6]  Erica E M Moodie,et al.  Estimating optimal shared‐parameter dynamic regimens with application to a multistage depression clinical trial , 2016, Biometrics.

[7]  Eric B. Laber,et al.  Dynamic treatment regimes: Technical challenges and applications , 2014 .

[8]  Susan A. Murphy,et al.  A-Learning for approximate planning , 2004 .

[9]  Yingqi Zhao,et al.  Inference for Optimal Dynamic Treatment Regimes Using an Adaptive m‐Out‐of‐n Bootstrap Scheme , 2013, Biometrics.

[10]  Luisa T. Fernholz,et al.  Content-Corrected Tolerance Limits Based on the Bootstrap , 2001, Technometrics.

[11]  Eric B. Laber,et al.  Inference about the expected performance of a data-driven dynamic treatment regime , 2014, Clinical trials.

[12]  Inbal Nahum-Shani,et al.  Optimization of behavioral dynamic treatment regimens based on the sequential, multiple assignment, randomized trial (SMART) , 2014, Clinical trials.

[13]  P. J. Huber.  The behavior of maximum likelihood estimates under nonstandard conditions , 1967 .

[14]  Derek S. Young,et al.  Regression Tolerance Intervals , 2013, Commun. Stat. Simul. Comput..

[15]  D. Kupfer,et al.  Sequenced treatment alternatives to relieve depression (STAR*D): rationale and design. , 2004, Controlled clinical trials.

[16]  J. Neter,et al.  Applied Linear Regression Models , 1983 .

[17]  James M Robins,et al.  The International Journal of Biostatistics CAUSAL INFERENCE Dynamic Regime Marginal Structural Mean Models for Estimation of Optimal Dynamic Treatment Regimes , Part II : Proofs of Results , 2011 .

[18]  Peter Dalgaard,et al.  R Development Core Team (2010): R: A language and environment for statistical computing , 2010 .

[19]  H. White.  A Heteroskedasticity-Consistent Covariance Matrix Estimator and a Direct Test for Heteroskedasticity , 1980 .

[20]  S. S. Wilks.  Determination of Sample Sizes for Setting Tolerance Limits , 1941 .

[21]  Susan A. Murphy,et al.  Efficient Reinforcement Learning with Multiple Reward Functions for Randomized Controlled Trial Analysis , 2010, ICML.

[22]  Anastasios A. Tsiatis,et al.  Q- and A-learning Methods for Estimating Optimal Dynamic Treatment Regimes , 2012, Statistical science : a review journal of the Institute of Mathematical Statistics.

[23]  Thomas Mathew,et al.  Statistical Tolerance Regions: Theory, Applications, and Computation , 2009 .

[24]  Daniel J. Lizotte,et al.  Set‐valued dynamic treatment regimes for competing outcomes , 2012, Biometrics.

[25]  Nir Friedman,et al.  Probabilistic Graphical Models - Principles and Techniques , 2009 .

[26]  Susan Murphy,et al.  Inference for non-regular parameters in optimal dynamic treatment regimes , 2010, Statistical methods in medical research.

[27]  Olli Saarela,et al.  Predictive Bayesian inference and dynamic treatment regimes , 2015, Biometrical journal. Biometrische Zeitschrift.

[28]  D. Ghosh.  Propensity score modelling in observational studies using dimension reduction methods. , 2011, Statistics & probability letters.

[29]  S. Murphy,et al.  Experimental design and primary data analysis methods for comparing adaptive interventions. , 2012, Psychological methods.

[30]  Joelle Pineau,et al.  Informing sequential clinical decision-making through reinforcement learning: an empirical study , 2010, Machine Learning.

[31]  Susan A. Murphy,et al.  Linear fitted-Q iteration with multiple reward functions , 2013, J. Mach. Learn. Res..

[32]  Nir Friedman,et al.  Probabilistic Graphical Models: Principles and Techniques - Adaptive Computation and Machine Learning , 2009 .

[33]  W Wallis,et al.  Tolerance Intervals for Linear Regression , 1951 .

[34]  J. Markowitz,et al.  The 16-Item quick inventory of depressive symptomatology (QIDS), clinician rating (QIDS-C), and self-report (QIDS-SR): a psychometric evaluation in patients with chronic major depression , 2003, Biological Psychiatry.

[35]  Donglin Zeng,et al.  New Statistical Learning Methods for Estimating Optimal Dynamic Treatment Regimes , 2015, Journal of the American Statistical Association.

[36]  Stephen B. Vardeman,et al.  What about the other Intervals , 1992 .

[37]  Eric B. Laber,et al.  Estimation of optimal dynamic treatment regimes , 2014, Clinical trials.

[38]  Jessica K. Barrett,et al.  Doubly Robust Estimation of Optimal Dynamic Treatment Regimes , 2014, Statistics in biosciences.

[39]  M. Barry,et al.  Shared decision making--pinnacle of patient-centered care. , 2012, The New England journal of medicine.