Bayesian Nonparametric Estimation for Dynamic Treatment Regimes With Sequential Transition Times

ABSTRACT We analyze a dataset arising from a clinical trial involving multi-stage chemotherapy regimes for acute leukemia. The trial design was a 2 × 2 factorial for frontline therapies only. Motivated by the idea that subsequent salvage treatments affect survival time, we model therapy as a dynamic treatment regime (DTR), that is, an alternating sequence of adaptive treatments or other actions and transition times between disease states. These sequences may vary substantially between patients, depending on how the regime plays out. To evaluate the regimes, mean overall survival time is expressed as a weighted average of the means of all possible sums of successive transitions times. We assume a Bayesian nonparametric survival regression model for each transition time, with a dependent Dirichlet process prior and Gaussian process base measure (DDP-GP). Posterior simulation is implemented by Markov chain Monte Carlo (MCMC) sampling. We provide general guidelines for constructing a prior using empirical Bayes methods. The proposed approach is compared with inverse probability of treatment weighting, including a doubly robust augmented version of this approach, for both single-stage and multi-stage regimes with treatment assignment depending on baseline covariates. The simulations show that the proposed nonparametric Bayesian approach can substantially improve inference compared to existing methods. An R program for implementing the DDP-GP-based Bayesian nonparametric analysis is freely available at www.ams.jhu.edu/yxu70. Supplementary materials for this article are available online.

[1]  Michael R. Kosorok,et al.  Causal nearest neighbor rules for optimal treatment regimes , 2017, 1711.08451.

[2]  Wenbin Lu,et al.  Optimal treatment regimes for survival endpoints using a locally-efficient doubly-robust estimator from a classification perspective , 2017, Lifetime data analysis.

[3]  Wenbin Lu,et al.  Change-Plane Analysis for Subgroup Detection and Sample Size Calculation , 2017, Journal of the American Statistical Association.

[4]  Thomas A Murray,et al.  Robust Treatment Comparison Based on Utilities of Semi-Competing Risks in Non-Small-Cell Lung Cancer , 2017, Journal of the American Statistical Association.

[5]  Michael R Kosorok,et al.  Residual Weighted Learning for Estimating Individualized Treatment Rules , 2015, Journal of the American Statistical Association.

[6]  Wenbin Lu,et al.  On estimation of optimal treatment regimes for maximizing t‐year survival probability , 2014, Journal of the Royal Statistical Society. Series B, Statistical methodology.

[7]  P. Diggle,et al.  Latent Variable Modeling and Applications to Causality , 2017 .

[8]  Peter F Thall,et al.  Bayesian group sequential clinical trial design using total toxicity burden and progression‐free survival , 2016, Journal of the Royal Statistical Society. Series C, Applied statistics.

[9]  Trippa Lorenzo,et al.  Bayesian Nonparametric Estimation for Dynamic Treatment Regimes With Sequential Transition Times Comment , 2016 .

[10]  Eric B. Laber,et al.  Tree-based methods for individualized treatment regimes. , 2015, Biometrika.

[11]  Marie Davidian,et al.  Using decision lists to construct interpretable and parsimonious treatment regimes , 2015, Biometrics.

[12]  Peter F Thall,et al.  Bayesian Dose-Finding in Two Treatment Cycles Based on the Joint Utility of Efficacy and Toxicity , 2015, Journal of the American Statistical Association.

[13]  Donglin Zeng,et al.  New Statistical Learning Methods for Estimating Optimal Dynamic Treatment Regimes , 2015, Journal of the American Statistical Association.

[14]  Iven Van Mechelen,et al.  A Novel Method for Estimating Optimal Tree-Based Treatment Regimes in Randomized Clinical Trials , 2015 .

[15]  Eric B. Laber,et al.  Doubly Robust Learning for Estimating Individualized Treatment with Censored Data. , 2015, Biometrika.

[16]  Wenting Cheng,et al.  Reader reaction to “A robust method for estimating optimal treatment regimes” by Zhang et al. (2012) , 2015, Biometrics.

[17]  Xuming He,et al.  Inference for Subgroup Analysis With a Structured Logistic-Normal Mixture Model , 2015 .

[18]  Eric B. Laber,et al.  Interactive model building for Q-learning. , 2014, Biometrika.

[19]  Nema Dean,et al.  Q-Learning: Flexible Learning About Useful Utilities , 2013, Statistics in Biosciences.

[20]  Holly Janes,et al.  Combining biomarkers to optimize patient treatment recommendations , 2014, Biometrics.

[21]  C. Carvalho,et al.  Decoupling Shrinkage and Selection in Bayesian Linear Models: A Posterior Summary Perspective , 2014, 1408.0464.

[22]  Eric B. Laber,et al.  Inference about the expected performance of a data-driven dynamic treatment regime , 2014, Clinical trials.

[23]  Anastasios A. Tsiatis,et al.  Q- and A-learning Methods for Estimating Optimal Dynamic Treatment Regimes , 2012, Statistical science : a review journal of the Institute of Mathematical Statistics.

[24]  Eric B. Laber,et al.  Dynamic treatment regimes: Technical challenges and applications , 2014 .

[25]  Marie Davidian,et al.  Robust estimation of optimal dynamic treatment regimes for sequential treatment decisions. , 2013, Biometrika.

[26]  Yingqi Zhao,et al.  Inference for Optimal Dynamic Treatment Regimes Using an Adaptive m‐Out‐of‐n Bootstrap Scheme , 2013, Biometrics.

[27]  Peter F Thall,et al.  Using Joint Utilities of the Times to Response and Toxicity to Adaptively Optimize Schedule–Dose Regimes , 2013, Biometrics.

[28]  Abdus S Wahed,et al.  Evaluating joint effects of induction–salvage treatment regimes on overall survival in acute leukaemia , 2013, Journal of the Royal Statistical Society. Series C, Applied statistics.

[29]  Riten Mitra,et al.  Bayesian Nonparametric Inference - Why and How. , 2013, Bayesian analysis.

[30]  Eric B. Laber,et al.  A Robust Method for Estimating Optimal Treatment Regimes , 2012, Biometrics.

[31]  Donglin Zeng,et al.  Estimating Individualized Treatment Rules Using Outcome Weighted Learning , 2012, Journal of the American Statistical Association.

[32]  Peter F Thall,et al.  Evaluation of Viable Dynamic Treatment Regimes in a Sequentially Randomized Trial of Advanced Prostate Cancer , 2012, Journal of the American Statistical Association.

[33]  Stephen G. Walker,et al.  A Bayesian nonparametric causal model , 2012 .

[34]  M. Kosorok,et al.  Q-LEARNING WITH CENSORED DATA. , 2012, Annals of statistics.

[35]  Min Zhang,et al.  Estimating optimal treatment regimes from a classification perspective , 2012, Stat.

[36]  Jennifer L. Hill,et al.  Bayesian Nonparametric Modeling for Causal Inference , 2011 .

[37]  Tristan Zajonc,et al.  Bayesian Inference for Dynamic Treatment Regimes: Mobility, Equity, and Efficiency in Student Tracking , 2010 .

[38]  J. Robins,et al.  The International Journal of Biostatistics CAUSAL INFERENCE Dynamic Regime Marginal Structural Mean Models for Estimation of Optimal Dynamic Treatment Regimes , Part I : Main Content , 2011 .

[39]  Carl E. Rasmussen,et al.  Gaussian processes for machine learning , 2005, Adaptive computation and machine learning.

[40]  A. P. Dawid,et al.  Regression and Classification Using Gaussian Process Priors , 2009 .

[41]  Radford M. Neal Regression and Classification Using Gaussian Process Priors , 2009 .

[42]  M. Kosorok,et al.  Reinforcement learning design for cancer clinical trials , 2009, Statistics in medicine.

[43]  J. Robins,et al.  Estimation and extrapolation of optimal treatment and testing strategies , 2008, Statistics in medicine.

[44]  Peter F Thall,et al.  Bayesian and frequentist two‐stage treatment strategies based on sequential failure times subject to interval censoring , 2007, Statistics in medicine.

[45]  P. Thall,et al.  Adaptive therapy for androgen-independent prostate cancer: a randomized selection trial of four regimens. , 2007, Journal of the National Cancer Institute.

[46]  J Q Shi,et al.  Gaussian Process Functional Regression Modeling for Batch Data , 2007, Biometrics.

[47]  Erica E M Moodie,et al.  Demystifying Optimal Dynamic Treatment Regimes , 2007, Biometrics.

[48]  Susan A Murphy,et al.  Customizing treatment to the patient: adaptive treatment strategies. , 2007, Drug and alcohol dependence.

[49]  S. Murphy,et al.  Developing adaptive treatment strategies in substance abuse research. , 2007, Drug and alcohol dependence.

[50]  M. J. van der Laan,et al.  The International Journal of Biostatistics Causal Effect Models for Realistic Individualized Treatment and Intention to Treat Rules , 2011 .

[51]  G. Bernstein,et al.  Practice parameter for the assessment and treatment of children and adolescents with anxiety disorders. , 2007, Journal of the American Academy of Child and Adolescent Psychiatry.

[52]  J. McClellan,et al.  Practice parameter for the assessment and treatment of children and adolescents with bipolar disorder. , 2007, Journal of the American Academy of Child and Adolescent Psychiatry.

[53]  K. Hirano,et al.  Asymptotics for Statistical Treatment Rules , 2009 .

[54]  A. Tsiatis Semiparametric Theory and Missing Data , 2006 .

[55]  Anastasios A. Tsiatis,et al.  Semiparametric efficient estimation of survival distributions in two-stage randomisation designs in clinical trials with censored data , 2006 .

[56]  S. Murphy,et al.  An experimental design for the development of adaptive treatment strategies , 2005, Statistics in medicine.

[57]  P. Rothwell Subgroup analysis in randomised controlled trials: importance, indications, and interpretation , 2005, The Lancet.

[58]  V. De Gruttola,et al.  Reader Reaction , 2004 .

[59]  P. Lavori,et al.  Placebo‐free designs for evaluating new mental health treatments: the use of adaptive treatment strategies , 2004, Statistics in medicine.

[60]  P. Thall,et al.  Dose‐Finding Based on Efficacy–Toxicity Trade‐Offs , 2004, Biometrics.

[61]  Sara T Brookes,et al.  Subgroup analyses in randomized trials: risks of subgroup-specific analyses; power and sample size for the interaction test. , 2004, Journal of clinical epidemiology.

[62]  James M. Robins,et al.  Optimal Structural Nested Models for Optimal Sequential Decisions , 2004 .

[63]  S. Murphy,et al.  Optimal dynamic treatment regimes , 2003 .

[64]  H. Sung,et al.  Selecting Therapeutic Strategies Based on Efficacy and Death in Multicourse Clinical Trials , 2002 .

[65]  Marie Davidian,et al.  Estimation of Survival Distributions of Treatment Policies in Two‐Stage Randomization Designs in Clinical Trials , 2002, Biometrics.

[66]  J M Robins,et al.  Marginal Mean Models for Dynamic Regimes , 2001, Journal of the American Statistical Association.

[67]  Lancelot F. James,et al.  Gibbs Sampling Methods for Stick-Breaking Priors , 2001 .

[68]  J. Robins,et al.  Marginal structural models to estimate the causal effect of zidovudine on the survival of HIV-positive men. , 2000, Epidemiology.

[69]  J. Robins,et al.  Marginal Structural Models and Causal Inference in Epidemiology , 2000, Epidemiology.

[70]  Radford M. Neal Markov Chain Sampling Methods for Dirichlet Process Mixture Models , 2000 .

[71]  H. Sung,et al.  Evaluating multiple treatment courses in clinical trials. , 2000, Statistics in medicine.

[72]  Philip W. Lavori,et al.  A design for testing clinical strategies: biased adaptive within‐subject randomization , 2000 .

[73]  J. Robins,et al.  Adjusting for Nonignorable Drop-Out Using Semiparametric Nonresponse Models , 1999 .

[74]  P F Thall,et al.  Randomized phase II study of fludarabine + cytosine arabinoside + idarubicin +/- all-trans retinoic acid +/- granulocyte colony-stimulating factor in poor prognosis newly diagnosed acute myeloid leukemia and myelodysplastic syndrome. , 1999, Blood.

[75]  Christopher K. I. Williams Prediction with Gaussian Processes: From Linear Regression to Linear Prediction and Beyond , 1999, Learning in Graphical Models.

[76]  S. MacEachern,et al.  Estimating mixture of dirichlet process models , 1998 .

[77]  James M. Robins,et al.  Causal Inference from Complex Longitudinal Data , 1997 .

[78]  Geoffrey E. Hinton,et al.  Bayesian Learning for Neural Networks , 1995 .

[79]  J. Robins,et al.  Recovery of Information and Adjustment for Dependent Censoring Using Surrogate Markers , 1992 .

[80]  J. Wittes,et al.  Analysis and interpretation of treatment effects in subgroups of patients in randomized clinical trials. , 1991, JAMA.

[81]  J. Sethuraman A CONSTRUCTIVE DEFINITION OF DIRICHLET PRIORS , 1991 .

[82]  D. Rubin [On the Application of Probability Theory to Agricultural Experiments. Essay on Principles. Section 9.] Comment: Neyman (1923) and Causal Inference in Experiments and Observational Studies , 1990 .

[83]  Aad Van Der Vbart,et al.  ON DIFFERENTIABLE FUNCTIONALS , 1988 .

[84]  C. N. Morris,et al.  The calculation of posterior distributions by data augmentation , 1987 .

[85]  J. Robins Addendum to “a new approach to causal inference in mortality studies with a sustained exposure period—application to control of the healthy worker survivor effect” , 1987 .

[86]  J. Robins A new approach to causal inference in mortality studies with a sustained exposure period—application to control of the healthy worker survivor effect , 1986 .

[87]  A. O'Hagan,et al.  Curve Fitting and Optimal Design for Prediction , 1978 .

[88]  Donald B. Rubin,et al.  Bayesian Inference for Causal Effects: The Role of Randomization , 1978 .

[89]  T. Ferguson A Bayesian Analysis of Some Nonparametric Problems , 1973 .