Targeted Learning of the Mean Outcome under an Optimal Dynamic Treatment Rule

Abstract We consider estimation of and inference for the mean outcome under the optimal dynamic two time-point treatment rule defined as the rule that maximizes the mean outcome under the dynamic treatment, where the candidate rules are restricted to depend only on a user-supplied subset of the baseline and intermediate covariates. This estimation problem is addressed in a statistical model for the data distribution that is nonparametric beyond possible knowledge about the treatment and censoring mechanism. This contrasts from the current literature that relies on parametric assumptions. We establish that the mean of the counterfactual outcome under the optimal dynamic treatment is a pathwise differentiable parameter under conditions, and develop a targeted minimum loss-based estimator (TMLE) of this target parameter. We establish asymptotic linearity and statistical inference for this estimator under specified conditions. In a sequentially randomized trial the statistical inference relies upon a second-order difference between the estimator of the optimal dynamic treatment and the optimal dynamic treatment to be asymptotically negligible, which may be a problematic condition when the rule is based on multivariate time-dependent covariates. To avoid this condition, we also develop TMLEs and statistical inference for data adaptive target parameters that are defined in terms of the mean outcome under the estimate of the optimal dynamic treatment. In particular, we develop a novel cross-validated TMLE approach that provides asymptotic inference under minimal conditions, avoiding the need for any empirical process conditions. We offer simulation results to support our theoretical findings.

[1]  D. Berry,et al.  Statistical models in epidemiology, the environment, and clinical trials , 2000 .

[2]  M. J. van der Laan,et al.  The International Journal of Biostatistics Targeted Maximum Likelihood Learning , 2011 .

[3]  M. J. van der Laan,et al.  The International Journal of Biostatistics Causal Effect Models for Realistic Individualized Treatment and Intention to Treat Rules , 2011 .

[4]  S. Murphy,et al.  A "SMART" design for building individualized treatment sequences. , 2012, Annual review of clinical psychology.

[5]  J. Robins.  A new approach to causal inference in mortality studies with a sustained exposure period—application to control of the healthy worker survivor effect , 1986 .

[6]  J. Robins,et al.  Estimation and extrapolation of optimal treatment and testing strategies , 2008, Statistics in medicine.

[7]  M. J. Laan,et al.  Targeted Minimum Loss Based Estimation of an Intervention Specific Mean Outcome , 2011 .

[8]  J. Robins,et al.  The International Journal of Biostatistics CAUSAL INFERENCE Dynamic Regime Marginal Structural Mean Models for Estimation of Optimal Dynamic Treatment Regimes , Part I : Main Content , 2011 .

[9]  J Mark,et al.  The Construction and Analysis of Adaptive Group Sequential Designs , 2008 .

[10]  James M. Robins,et al.  Marginal Structural Models versus Structural nested Models as Tools for Causal inference , 2000 .

[11]  James M. Robins,et al.  Optimal Structural Nested Models for Optimal Sequential Decisions , 2004 .

[12]  Aad van der Vaart,et al.  Higher order influence functions and minimax estimation of nonlinear functionals , 2008, 0805.3040.

[13]  P. Bickel.  Efficient and Adaptive Estimation for Semiparametric Models , 1993 .

[14]  M. J. Laan,et al.  Targeted Learning: Causal Inference for Observational and Experimental Data , 2011 .

[15]  J. Robins,et al.  Estimating causal effects from epidemiological data , 2006, Journal of Epidemiology and Community Health.

[16]  Brian T. Austin,et al.  Improving chronic illness care: translating evidence into action. , 2001, Health affairs.

[17]  S. Murphy,et al.  An experimental design for the development of adaptive treatment strategies , 2005, Statistics in medicine.

[18]  S. Murphy,et al.  Optimal dynamic treatment regimes , 2003 .

[19]  Alan E. Hubbard,et al.  Statistical Inference for Data Adaptive Target Parameters , 2016, The international journal of biostatistics.

[20]  M. J. Laan,et al.  Targeted Learning of an Optimal Dynamic Treatment, and Statistical Inference for its Mean Outcome , 2014 .

[21]  Robin Henderson,et al.  Estimation of optimal dynamic anticoagulation regimes from observational data: a regret-based approach. , 2006, Statistics in medicine.

[22]  M. J. van der Laan,et al.  Targeted Minimum Loss Based Estimation of Causal Effects of Multiple Time Point Interventions , 2012, The international journal of biostatistics.

[23]  Susan Murphy,et al.  Inference for non-regular parameters in optimal dynamic treatment regimes , 2010, Statistical methods in medical research.

[24]  M. J. van der Laan,et al.  Statistical Applications in Genetics and Molecular Biology Super Learner , 2010 .

[25]  Mark J. van der Laan,et al.  Cross-Validated Targeted Minimum-Loss-Based Estimation , 2011 .

[26]  P. Diggle,et al.  Latent Variable Modeling and Applications to Causality , 2017 .

[27]  H. Strasser.  Efficient and adaptive estimation for semiparametric models - P. J. Bickel; Ch. A. J. Klaassen; Ya 'acov Ritov; J. A. Wellner. , 1997 .

[28]  James M. Robins,et al.  Causal Inference from Complex Longitudinal Data , 1997 .

[29]  R Core Team,et al.  R: A language and environment for statistical computing. , 2014 .

[30]  Jon A. Wellner,et al.  Weak Convergence and Empirical Processes: With Applications to Statistics , 1996 .

[31]  K. Do,et al.  Efficient and Adaptive Estimation for Semiparametric Models. , 1994 .

[32]  Mark J. van der Laan,et al.  Asymptotic Theory for Cross-validated Targeted Maximum Likelihood Estimation , 2010 .

[33]  J. Robins,et al.  Doubly Robust Estimation in Missing Data and Causal Inference Models , 2005, Biometrics.

[34]  J. Robins,et al.  Comparison of dynamic treatment regimes via inverse probability weighting. , 2006, Basic & clinical pharmacology & toxicology.

[35]  Cha Zhang,et al.  Ensemble Machine Learning , 2012 .

[36]  J. Robins,et al.  Sensitivity Analysis for Selection bias and unmeasured Confounding in missing Data and Causal inference models , 2000 .

[37]  J. Pearl.  Causality: Models, Reasoning and Inference , 2000 .

[38]  J. Robins.  Addendum to “a new approach to causal inference in mortality studies with a sustained exposure period—application to control of the healthy worker survivor effect” , 1987 .

[39]  Philip W. Lavori,et al.  A design for testing clinical strategies: biased adaptive within‐subject randomization , 2000 .

[40]  Ree Dawson,et al.  Adaptive treatment strategies in chronic disease. , 2008, Annual review of medicine.

[41]  James M. Robins,et al.  Unified Methods for Censored Longitudinal Data and Causality , 2003 .

[42]  Maia Berkane.  Latent Variable Modeling and Applications to Causality , 1997 .

[43]  Terence P. Speed,et al.  Introductory Remarks on Neyman (1923) , 1990 .

[44]  Ree Dawson,et al.  Dynamic treatment regimes: practical design considerations , 2004, Clinical trials.

[45]  A. V. D. Vaart,et al.  Asymptotic Statistics: Frontmatter , 1998 .

[46]  H. Sung,et al.  Evaluating multiple treatment courses in clinical trials. , 2000, Statistics in medicine.

[47]  Eric B. Laber,et al.  Dynamic treatment regimes: Technical challenges and applications , 2014 .

[48]  Dylan S. Small,et al.  Discussion of “Dynamic treatment regimes: Technical challenges and applications” , 2014 .

[49]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[50]  Vassilis Giakoumakis,et al.  Addendum to: Maximum Weight Independent Sets in hole- and co-chair-free graphs , 2015, Inf. Process. Lett..

[51]  Inbal Nahum-Shani,et al.  Q-learning: a data analysis method for constructing adaptive interventions. , 2012, Psychological methods.

[52]  P. Holland.  Statistics and Causal Inference , 1985 .

[53]  D. Rubin.  Estimating causal effects of treatments in randomized and nonrandomized studies. , 1974 .

[54]  J. Robins.  A graphical approach to the identification and estimation of causal parameters in mortality studies with sustained exposure periods. , 1987, Journal of chronic diseases.

[55]  Angie Wade.  Matched Sampling for Causal Effects , 2008 .

[56]  H. Sung,et al.  Selecting Therapeutic Strategies Based on Efficacy and Death in Multicourse Clinical Trials , 2002 .

[57]  Mark J van der Laan,et al.  Super Learning: An Application to the Prediction of HIV-1 Drug Resistance , 2007, Statistical applications in genetics and molecular biology.

[58]  Susan M Shortreed,et al.  Estimating the optimal dynamic antipsychotic treatment regime: evidence from the sequential multiple‐assignment randomized Clinical Antipsychotic Trials of Intervention and Effectiveness schizophrenia study , 2012, Journal of the Royal Statistical Society. Series C, Applied statistics.

[59]  A Data Augmentation Method for Estimating the Causal Effect of Adherence to Treatment Regimens Targeting Control of an Intermediate Measure , 2011 .

[60]  M. J. van der Laan,et al.  Super-Learning of an Optimal Dynamic Treatment Rule , 2016, The international journal of biostatistics.

[61]  S. Murphy,et al.  Experimental design and primary data analysis methods for comparing adaptive interventions. , 2012, Psychological methods.

[62]  Mark J van der Laan,et al.  History-adjusted marginal structural models for estimating time-varying effect modification. , 2007, American journal of epidemiology.

[63]  Iván Díaz,et al.  Targeted Data Adaptive Estimation of the Causal Dose–Response Curve , 2013 .