Identification of predicted individual treatment effects in randomized clinical trials

In most medical research, treatment effectiveness is assessed using the average treatment effect or some version of subgroup analysis. The practice of individualized or precision medicine, however, requires new approaches that predict how an individual will respond to treatment, rather than relying on aggregate measures of effect. In this study, we present a conceptual framework for estimating individual treatment effects, referred to as predicted individual treatment effects. We first apply the predicted individual treatment effect approach to a randomized controlled trial designed to improve behavioral and physical symptoms. Despite trivial average effects of the intervention, we show substantial heterogeneity in predicted individual treatment response using the predicted individual treatment effect approach. The predicted individual treatment effects can be used to predict individuals for whom the intervention may be most effective (or harmful). Next, we conduct a Monte Carlo simulation study to evaluate the accuracy of predicted individual treatment effects. We compare the performance of two methods used to obtain predictions: multiple imputation and non-parametric random decision trees. Results showed that, on average, both predictive methods produced accurate estimates at the individual level; however, the random decision trees tended to underestimate the predicted individual treatment effect for people at the extreme and showed more variability in predictions across repetitions compared to the imputation approach. Limitations and future directions are discussed.

[1]  D. Rubin.  Estimating causal effects of treatments in randomized and nonrandomized studies. , 1974 .

[2]  L. Radloff.  The CES-D Scale , 1977 .

[3]  M. Gail,et al.  Testing for qualitative interactions between treatment effects and patient subsets. , 1985, Biometrics.

[4]  P. Holland.  Statistics and Causal Inference , 1985 .

[5]  Donald B. Rubin,et al.  Formal modes of statistical inference for causal effects , 1990 .

[6]  Terence P. Speed,et al.  Introductory Remarks on Neyman (1923) , 1990 .

[7]  R. Simon,et al.  Bayesian subset analysis. , 1991, Biometrics.

[8]  Z Lou,et al.  Tree-structured prediction for censored survival data and the Cox model. , 1995, Journal of clinical epidemiology.

[9]  S. Assmann,et al.  Subgroup analysis and other (mis)uses of baseline data in clinical trials , 2000, The Lancet.

[10]  David E. Booth,et al.  Analysis of Incomplete Multivariate Data , 2000, Technometrics.

[11]  H. Iyer,et al.  Unit–Treatment Interaction and Its Practical Consequences , 2000, Biometrics.

[12]  John Van Hoewyk,et al.  A multivariate technique for multiply imputing missing values using a sequence of regression models , 2001 .

[13]  T. Peters,et al.  Subgroup analyses in randomised controlled trials: quantifying the risks of false-positives and false-negatives. , 2001, Health technology assessment.

[14]  H. Iyer,et al.  EVALUATING SUBJECT-TREATMENT INTERACTION WHEN COMPARING TWO TREATMENTS , 2001, Journal of biopharmaceutical statistics.

[15]  R. Simon,et al.  Bayesian subset analysis: application to studying treatment‐by‐gender interactions , 2002, Statistics in medicine.

[16]  S. Pocock,et al.  Subgroup analysis, covariate adjustment and baseline comparisons in clinical trial reporting: current practiceand problems , 2002, Statistics in medicine.

[17]  Yi Tsong,et al.  ISSUES RELATED TO SUBGROUP ANALYSIS IN CLINICAL TRIALS , 2002, Journal of biopharmaceutical statistics.

[18]  Roderick J. A. Little,et al.  Statistical Analysis with Missing Data: Little/Statistical Analysis with Missing Data , 2002 .

[19]  Nicole A. Lazar,et al.  Statistical Analysis With Missing Data , 2003, Technometrics.

[20]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[21]  D. Ruppert.  The Elements of Statistical Learning: Data Mining, Inference, and Prediction , 2004 .

[22]  Russell V. Lenth,et al.  Statistical Analysis With Missing Data (2nd ed.) (Book) , 2004 .

[23]  Jeffrey M. Albert,et al.  Individual treatment effects in randomized trials with binary outcomes , 2004 .

[24]  D. Rubin.  Causal Inference Using Potential Outcomes , 2005 .

[25]  S. Lagakos.  The challenge of subgroup analyses--reporting without distorting. , 2006, The New England journal of medicine.

[26]  James J Heckman,et al.  Understanding Instrumental Variables in Models with Essential Heterogeneity , 2006, The Review of Economics and Statistics.

[27]  Julie A. Johnson,et al.  Influence of coagulation factor, vitamin K epoxide reductase complex subunit 1, and cytochrome P450 2C9 gene polymorphisms on warfarin dose requirements , 2006, Clinical pharmacology and therapeutics.

[28]  Stephen W Lagakos,et al.  Statistics in medicine--reporting of subgroup analyses in clinical trials. , 2007, The New England journal of medicine.

[29]  Paul R Rosenbaum,et al.  Confidence Intervals for Uncommon but Dramatic Responses to Treatment , 2007, Biometrics.

[30]  Xiaogang Su,et al.  Interaction Trees with Censored Survival Data , 2008, The international journal of biostatistics.

[31]  K. Hornik,et al.  Model-Based Recursive Partitioning , 2008 .

[32]  Hansheng Wang,et al.  Subgroup Analysis via Recursive Partitioning , 2009, J. Mach. Learn. Res..

[33]  Robert Tibshirani,et al.  The Elements of Statistical Learning: Data Mining, Inference, and Prediction, 2nd Edition , 2001, Springer Series in Statistics.

[34]  M. LeBlanc,et al.  Boosting predictions of treatment success , 2010, Proceedings of the National Academy of Sciences.

[35]  Claudio Conversano,et al.  Combining an Additive and Tree-Based Regression Model Simultaneously: STIMA , 2010 .

[36]  Lei Chen,et al.  The mean does not mean as much anymore: finding sub-groups for tailored therapeutics , 2010, Clinical trials.

[37]  V. Preedy,et al.  Center for Epidemiologic Studies-Depression Scale , 2010 .

[38]  Wesley O. Johnson,et al.  Interaction trees: exploring the differential effects of an intervention programme for breast cancer survivors , 2011 .

[39]  Building Strong Families (BSF) Project Data Collection, 2005-2008, United States , 2011 .

[40]  Sebastian Vollmer,et al.  Testing for heterogeneous treatment effects in experimental data: false discovery risks and correction procedures , 2014 .

[41]  K. Imai,et al.  Estimation of Heterogeneous Treatment Effects from Randomized Experiments, with Application to the Optimal Planning of the Get-Out-the-Vote Campaign , 2011, Political Analysis.

[42]  L. Tian,et al.  Analysis of randomized comparative clinical trial data for personalized treatment selections. , 2011, Biostatistics.

[43]  I. Lipkovich,et al.  Subgroup identification based on differential effect search—A recursive partitioning method for establishing response to treatment in patient subpopulations , 2011, Statistics in medicine.

[44]  A. Gelman,et al.  Multiple Imputation with Diagnostics (mi) in R: Opening Windows into the Black Box , 2011 .

[45]  J. M. Taylor,et al.  Subgroup identification from randomized clinical trial data , 2011, Statistics in medicine.

[46]  Xiaogang Su,et al.  Tree-structured analysis of treatment effects with large observational data , 2012 .

[47]  James Y Dai,et al.  Two-stage testing procedures with independent filtering for genome-wide gene-environment interaction. , 2012, Biometrika.

[48]  Understanding Treatment Effects Heterogeneities Using Multi-Site Regression Discontinuity Designs: Example from a "Double-Dose" Algebra Study in Chicago. , 2012 .

[49]  Holly Janes,et al.  Assessing Treatment‐Selection Markers using a Potential Outcomes Framework , 2012, Biometrics.

[50]  Xin Yan,et al.  Facilitating score and causal inference trees for large observational studies , 2012, J. Mach. Learn. Res..

[51]  D. Green,et al.  Modeling Heterogeneous Treatment Effects in Survey Experiments with Bayesian Additive Regression Trees , 2012 .

[52]  Boris Freidlin,et al.  Randomized phase II trial designs with biomarkers. , 2012, Journal of clinical oncology : official journal of the American Society of Clinical Oncology.

[53]  D. Allison,et al.  Treatment Heterogeneity and Individual Qualitative Interaction , 2012, The American statistician.

[54]  N. Simon,et al.  Adaptive enrichment designs for clinical trials. , 2013, Biostatistics.

[55]  V. Mor,et al.  Different analyses estimate different parameters of the effect of erythropoietin stimulating agents on survival in end stage renal disease: a comparison of payment policy analysis, instrumental variables, and multiple imputation of potential outcomes. , 2013, Journal of clinical epidemiology.

[56]  Zhiwei Zhang,et al.  Assessing the heterogeneity of treatment effects via potential outcomes of individual patients. , 2013, Journal of the Royal Statistical Society. Series C, Applied statistics.

[57]  Wendy J. Mack,et al.  Statistical Learning for Biomedical Data , 2013 .

[58]  Alex M. Andrew,et al.  Boosting: Foundations and Algorithms , 2012 .

[59]  C. Shen,et al.  Treatment Benefit and Treatment Harm Rate to Characterize Heterogeneity in Treatment Effect , 2013, Biometrics.

[60]  Marc Ratkovic,et al.  Estimating treatment effect heterogeneity in randomized program evaluation , 2013, 1305.5682.

[61]  C. Perou,et al.  Personalizing the treatment of women with early breast cancer: highlights of the St Gallen International Expert Consensus on the Primary Therapy of Early Breast Cancer 2013 , 2013, Annals of oncology : official journal of the European Society for Medical Oncology.

[62]  Ilya Lipkovich,et al.  Strategies for Identifying Predictive Biomarkers and Subgroups with Enhanced Treatment Effect in Clinical Trials Using SIDES , 2014, Journal of biopharmaceutical statistics.

[63]  R Core Team,et al.  R: A language and environment for statistical computing. , 2014 .

[64]  Iven Van Mechelen,et al.  A comparison of five recursive partitioning methods to find person subgroups involved in meaningful treatment–subgroup interactions , 2013, Advances in Data Analysis and Classification.

[65]  David Meyre,et al.  Jumping on the Train of Personalized Medicine: A Primer for Non- Geneticist Clinicians: Part 3. Clinical Applications in the Personalized Medicine Area , 2014, Current psychiatry reviews.

[66]  Estimating Person-Centered Treatment (Pet) Effects Using Instrumental Variables , 2012, Journal of applied econometrics.

[67]  Thomas A. Loughran,et al.  On the Importance of Treatment Effect Heterogeneity in Experimentally-Evaluated Criminal Justice Interventions , 2015 .

[68]  Lena Osterhagen,et al.  Multiple Imputation For Nonresponse In Surveys , 2016 .

[69]  Bo Zhang,et al.  Use of auxiliary covariates in estimating a biomarker-adjusted treatment effect model with clinical trial data , 2016, Statistical methods in medical research.

[70]  Jonah B. Gelbach,et al.  Can Variation in Subgroups' Average Treatment Effects Explain Treatment Effect Heterogeneity? Evidence from a Social Experiment , 2014, Review of Economics and Statistics.