Predictive approaches to heterogeneous treatment effects: a systematic review

Background: Recent evidence suggests that there is often substantial variation in the benefits and harms across a trial population. We aimed to identify regression modeling approaches that assess heterogeneity of treatment effect within a randomized clinical trial. Methods: We performed a literature review using a broad search strategy, complemented by suggestions of a technical expert panel. Results: The approaches are classified into 3 categories: 1) Risk-based methods (11 papers) use only prognostic factors to define patient subgroups, relying on the mathematical dependency of the absolute risk difference on baseline risk; 2) Treatment effect modeling methods (9 papers) use both prognostic factors and treatment effect modifiers to explore characteristics that interact with the effects of therapy on a relative scale. These methods couple data-driven subgroup identification with approaches to prevent overfitting, such as penalization or use of separate data sets for subgroup identification and effect estimation. 3) Optimal treatment regime methods (12 papers) focus primarily on treatment effect modifiers to classify the trial population into those who benefit from treatment and those who do not. Finally, we also identified papers which describe model evaluation methods (4 papers). Conclusion: Three classes of approaches were identified to assess heterogeneity of treatment effect. Methodological research, including both simulations and empirical evaluations, is required to compare the available methods in different settings and to derive well-informed guidance for their application in RCT analysis.

[1]  Helena Chmura Kraemer,et al.  Discovering, comparing, and combining moderators of treatment on outcome after randomized clinical trials: a parametric approach , 2013, Statistics in medicine.

[2]  Richard D Riley,et al.  Explicit inclusion of treatment in prognostic modeling was recommended in observational and randomized settings. , 2016, Journal of clinical epidemiology.

[3]  Eric B. Laber,et al.  Inference about the expected performance of a data-driven dynamic treatment regime , 2014, Clinical trials.

[4]  Max Welling,et al.  Causal Effect Inference with Deep Latent-Variable Models , 2017, NIPS 2017.

[5]  Yvonne Vergouwe,et al.  Estimates of absolute treatment benefit for individual patients required careful modeling of statistical interactions. , 2015, Journal of clinical epidemiology.

[6]  J. Berger,et al.  A Bayesian Approach to Subgroup Identification , 2014, Journal of biopharmaceutical statistics.

[7]  David M. Kent,et al.  Multistate Model to Predict Heart Failure Hospitalizations and All-Cause Mortality in Outpatients With Heart Failure With Reduced Ejection Fraction: Model Derivation and External Validation , 2016, Circulation. Heart failure.

[8]  B. Freidlin,et al.  Statistical and practical considerations for clinical evaluation of predictive biomarkers. , 2013, Journal of the National Cancer Institute.

[9]  Daniel J Sargent,et al.  The Fundamental Difficulty With Evaluating the Accuracy of Biomarkers for Guiding Treatment. , 2015, Journal of the National Cancer Institute.

[10]  Holly Janes,et al.  Assessing Treatment‐Selection Markers using a Potential Outcomes Framework , 2012, Biometrics.

[11]  Wenting Cheng,et al.  Reader reaction to “A robust method for estimating optimal treatment regimes” by Zhang et al. (2012) , 2015, Biometrics.

[12]  Ravi Varadhan,et al.  A framework for the analysis of heterogeneity of treatment effect in patient-centered outcomes research. , 2013, Journal of clinical epidemiology.

[13]  Michael J. Pencina,et al.  Use of Open Access Platforms for Clinical Trial Data. , 2016, JAMA.

[14]  Lacey Gunter,et al.  Variable Selection for Qualitative Interactions in Personalized Medicine While Controlling the Family-Wise Error Rate , 2011, Journal of biopharmaceutical statistics.

[15]  Ellen Frank,et al.  A novel approach for developing and interpreting treatment moderator profiles in randomized clinical trials. , 2013, JAMA psychiatry.

[16]  Harlan M Krumholz,et al.  Overview and experience of the YODA Project with clinical trial data sharing after 5 years , 2018, Scientific Data.

[17]  Mark J van der Laan,et al.  Evaluating the impact of treating the optimal subgroup , 2016, Statistical methods in medical research.

[18]  Matthew M. Chingos,et al.  The Review of Economics and Statistics , 2018 .

[19]  D A Follmann,et al.  A Multivariate Test of Interaction for Use in Clinical Trials , 1999, Biometrics.

[20]  Gordon H Guyatt,et al.  Can we individualize the 'number needed to treat'? An empirical study of summary effect measures in meta-analyses. , 2002, International journal of epidemiology.

[21]  Menggang Yu,et al.  Regularized outcome weighted subgroup identification for differential treatment effects , 2015, Biometrics.

[22]  Sanjay Basu,et al.  Benefit and harm of intensive blood pressure treatment: Derivation and validation of risk models using data from the SPRINT and ACCORD trials , 2017, PLoS medicine.

[23]  Michael E. Miller,et al.  Effects of intensive glucose lowering in type 2 diabetes. , 2008, The New England journal of medicine.

[24]  R. Varadhan,et al.  Bedside Optic Nerve Ultrasonography for Diagnosing Increased Intracranial Pressure , 2019, Annals of Internal Medicine.

[25]  Andrew J Vickers,et al.  Method for evaluating prediction models that apply the results of randomized trials to individual patients , 2007, Trials.

[26]  Ewout Steyerberg,et al.  Personalized evidence based medicine: predictive approaches to heterogeneous treatment effects , 2018, British Medical Journal.

[27]  Lee-Jen Wei,et al.  Treatment selections using risk-benefit profiles based on data from comparative randomized clinical trials with multiple endpoints. , 2015, Biostatistics.

[28]  David M Kent,et al.  Baseline characteristics predict risk of progression and response to combined medical therapy for benign prostatic hyperplasia (BPH) , 2015, BJU international.

[29]  Antonio Colombo,et al.  Percutaneous coronary intervention versus coronary-artery bypass grafting for severe coronary artery disease. , 2009, The New England journal of medicine.

[30]  Mark J van der Laan,et al.  Targeted Learning of the Mean Outcome under an Optimal Dynamic Treatment Rule , 2015, Journal of causal inference.

[31]  James F Burke,et al.  Implications of Heterogeneity of Treatment Effect for Reporting and Analysis of Randomized Trials in Critical Care. , 2015, American journal of respiratory and critical care medicine.

[32]  Robert Tibshirani,et al.  A comparison of methods for model selection when estimating individual treatment effects , 2018, 1804.05146.

[33]  M. J. van der Laan,et al.  STATISTICAL INFERENCE FOR THE MEAN OUTCOME UNDER A POSSIBLY NON-UNIQUE OPTIMAL TREATMENT STRATEGY. , 2016, Annals of statistics.

[34]  Ewout W Steyerberg,et al.  Models with interactions overestimated heterogeneity of treatment effects and were prone to treatment mistargeting. , 2019, Journal of clinical epidemiology.

[35]  F E Harrell,et al.  Selection of thrombolytic therapy for individual patients: development of a clinical model. GUSTO-I Investigators. , 1997, American heart journal.

[36]  Sören R. Künzel,et al.  Metalearners for estimating heterogeneous treatment effects using machine learning , 2017, Proceedings of the National Academy of Sciences.

[37]  Federico Rotolo,et al.  Identification of biomarker‐by‐treatment interactions in randomized clinical trials with survival outcomes and high‐dimensional spaces , 2016, Biometrical journal. Biometrische Zeitschrift.

[38]  P. Rothwell,et al.  Can overall results of clinical trials be applied to all patients? , 1995, The Lancet.

[39]  L. Caplan,et al.  Evidence based medicine: concerns of a clinical neurologist , 2001, Journal of neurology, neurosurgery, and psychiatry.

[40]  Niko Kaciroti,et al.  Simple subgroup approximations to optimal treatment regimes from randomized clinical trial data. , 2015, Biostatistics.

[41]  Issa J Dahabreh,et al.  Risk and treatment effect heterogeneity: re-analysis of individual participant data from 32 large clinical trials. , 2016, International journal of epidemiology.

[42]  Eva Petkova,et al.  Generated effect modifiers (GEM's) in randomized clinical trials. , 2017, Biostatistics.

[43]  Ewout W Steyerberg,et al.  Estimating treatment effects for individual patients based on the results of randomised clinical trials , 2011, BMJ : British Medical Journal.

[44]  H. Weisberg,et al.  Post hoc subgroups in clinical trials: Anathema or analytics? , 2015, Clinical trials.

[45]  David M. Kent,et al.  Using Internally Developed Risk Models to Assess Heterogeneity in Treatment Effects in Clinical Trials , 2014, Circulation. Cardiovascular quality and outcomes.

[46]  David M Kent,et al.  Limitations of applying summary results of clinical trials to individual patients: the need for risk stratification. , 2007, JAMA.

[47]  Discussion of “Dynamic treatment regimes: Technical challenges and applications” , 2014 .

[48]  Eric B. Laber,et al.  A Robust Method for Estimating Optimal Treatment Regimes , 2012, Biometrics.

[49]  Lu Tian,et al.  A Simple Method for Detecting Interactions between a Treatment and a Large Number of Covariates , 2012, 1212.2995.

[50]  Joseph S. Ross,et al.  Clinical research data sharing: what an open science world means for researchers involved in evidence synthesis , 2016, Systematic Reviews.

[51]  Ewout W Steyerberg,et al.  The proposed 'concordance-statistic for benefit' provided a useful metric when modeling heterogeneous treatment effects. , 2018, Journal of clinical epidemiology.

[52]  S. Murphy,et al.  PERFORMANCE GUARANTEES FOR INDIVIDUALIZED TREATMENT RULES. , 2011, Annals of statistics.

[53]  E. Steyerberg Clinical Prediction Models , 2008, Statistics for Biology and Health.

[54]  Lu Tian,et al.  Effectively Selecting a Target Population for a Future Comparative Study , 2013, Journal of the American Statistical Association.

[55]  James A Hanley,et al.  Profile-specific survival estimates: Making reports of clinical trials more patient-relevant , 2008, Clinical trials.

[56]  L. Tian,et al.  Analysis of randomized comparative clinical trial data for personalized treatment selections. , 2011, Biostatistics.

[57]  Debashis Ghosh,et al.  Bayesian Variable Selection with Joint Modeling of Categorical and Survival Outcomes: An Application to Individualizing Chemotherapy Treatment in Advanced Colorectal Cancer , 2009, Biometrics.

[58]  Jennifer L Krull,et al.  Advancing Personalized Medicine: Application of a Novel Statistical Method to Identify Treatment Moderators in the Coordinated Anxiety Learning and Management Study. , 2017, Behavior therapy.

[59]  Min Zhang,et al.  Estimating optimal treatment regimes from a classification perspective , 2012, Stat.

[60]  S. Athey,et al.  Generalized random forests , 2016, The Annals of Statistics.

[61]  Holly Janes,et al.  Measuring the Performance of Markers for Guiding Treatment Decisions , 2011, Annals of Internal Medicine.

[62]  David M Kent,et al.  Multivariable risk prediction can greatly enhance the statistical power of clinical trial subgroup analysis , 2006, BMC medical research methodology.

[63]  Trevor Hastie,et al.  Some methods for heterogeneous treatment effect estimation in high dimensions , 2017, Statistics in medicine.

[64]  Donglin Zeng,et al.  Estimating Individualized Treatment Rules Using Outcome Weighted Learning , 2012, Journal of the American Statistical Association.

[65]  Jeremy B Sussman,et al.  Improving diabetes prevention with benefit based tailored treatment: risk based reanalysis of Diabetes Prevention Program , 2015, BMJ : British Medical Journal.

[66]  Stephanie Kovalchik,et al.  Assessing heterogeneity of treatment effect in a clinical trial with the proportional interactions model , 2013, Statistics in medicine.

[67]  David M Kent,et al.  Assessing and reporting heterogeneity in treatment effects in clinical trials: a proposal , 2010, Trials.

[68]  Eric B. Laber,et al.  Doubly Robust Learning for Estimating Individualized Treatment with Censored Data. , 2015, Biometrika.