Predictive approaches to heterogeneous treatment effects: a scoping review

Background Recent evidence suggests that there is often substantial variation in the benefits and harms across a trial population. We aimed to identify regression modeling approaches that assess heterogeneity of treatment effect within a randomized clinical trial. Methods We performed a literature review using a broad search strategy, complemented by suggestions of a technical expert panel. Results The approaches are classified into 3 categories: 1) Risk-based methods (11 papers) use only prognostic factors to define patient subgroups, relying on the mathematical dependency of the absolute risk difference on baseline risk; 2) Treatment effect modeling methods (9 papers) use both prognostic factors and treatment effect modifiers to explore characteristics that interact with the effects of therapy on a relative scale. These methods couple data-driven subgroup identification with approaches to prevent overfitting, such as penalization or use of separate data sets for subgroup identification and effect estimation. 3) Optimal treatment regime methods (12 papers) focus primarily on treatment effect modifiers to classify the trial population into those who benefit from treatment and those who do not. Finally, we also identified papers which describe model evaluation methods (4 papers). Conclusions Three classes of approaches were identified to assess heterogeneity of treatment effect. Methodological research, including both simulations and empirical evaluations, is required to compare the available methods in different settings and to derive well-informed guidance for their application in RCT analysis.

[1]  Eric B. Laber,et al.  A Robust Method for Estimating Optimal Treatment Regimes , 2012, Biometrics.

[2]  Holly Janes,et al.  Measuring the Performance of Markers for Guiding Treatment Decisions , 2011, Annals of Internal Medicine.

[3]  Donglin Zeng,et al.  Estimating Individualized Treatment Rules Using Outcome Weighted Learning , 2012, Journal of the American Statistical Association.

[4]  Jeremy B Sussman,et al.  Improving diabetes prevention with benefit based tailored treatment: risk based reanalysis of Diabetes Prevention Program , 2015, BMJ : British Medical Journal.

[5]  D A Follmann,et al.  A Multivariate Test of Interaction for Use in Clinical Trials , 1999, Biometrics.

[6]  Issa J Dahabreh,et al.  Risk and treatment effect heterogeneity: re-analysis of individual participant data from 32 large clinical trials. , 2016, International journal of epidemiology.

[7]  Ellen Frank,et al.  A novel approach for developing and interpreting treatment moderator profiles in randomized clinical trials. , 2013, JAMA psychiatry.

[8]  Lu Tian,et al.  A Simple Method for Detecting Interactions between a Treatment and a Large Number of Covariates , 2012, 1212.2995.

[9]  J. Berger,et al.  A Bayesian Approach to Subgroup Identification , 2014, Journal of biopharmaceutical statistics.

[10]  B. Freidlin,et al.  Statistical and practical considerations for clinical evaluation of predictive biomarkers. , 2013, Journal of the National Cancer Institute.

[11]  Michael E. Miller,et al.  ACTION TO CONTROL CARDIOVASCULAR RISK IN DIABETES STUDY GROUP. EFFECTS OF INTENSIVE GLUCOSE LOWERING IN TYPE 2 DIABETES , 2010 .

[12]  David M. Kent,et al.  Multistate Model to Predict Heart Failure Hospitalizations and All-Cause Mortality in Outpatients With Heart Failure With Reduced Ejection Fraction: Model Derivation and External Validation , 2016, Circulation. Heart failure.

[13]  Ewout Steyerberg,et al.  Personalized evidence based medicine: predictive approaches to heterogeneous treatment effects , 2018, British Medical Journal.

[14]  Eric B. Laber,et al.  Inference about the expected performance of a data-driven dynamic treatment regime , 2014, Clinical trials.

[15]  Sander Greenland,et al.  Modern Epidemiology 3rd edition , 1986 .

[16]  Ellen Fineout-Overholt,et al.  Users' Guides to the Medical Literature , 2002 .

[17]  Ewout W Steyerberg,et al.  Estimating treatment effects for individual patients based on the results of randomised clinical trials , 2011, BMJ : British Medical Journal.

[18]  H. Weisberg,et al.  Post hoc subgroups in clinical trials: Anathema or analytics? , 2015, Clinical trials.

[19]  Joseph S. Ross,et al.  Clinical research data sharing: what an open science world means for researchers involved in evidence synthesis , 2016, Systematic Reviews.

[20]  David M Kent,et al.  Multivariable risk prediction can greatly enhance the statistical power of clinical trial subgroup analysis , 2006, BMC medical research methodology.

[21]  Stephanie Kovalchik,et al.  Assessing heterogeneity of treatment effect in a clinical trial with the proportional interactions model , 2013, Statistics in medicine.

[22]  Eric B. Laber,et al.  Dynamic treatment regimes: Technical challenges and applications , 2014 .

[23]  Dylan S. Small,et al.  Discussion of “Dynamic treatment regimes: Technical challenges and applications” , 2014 .

[24]  William White,et al.  A Proposal , 2008, Moon, Sun, and Witches.

[25]  Lee-Jen Wei,et al.  Treatment selections using risk-benefit profiles based on data from comparative randomized clinical trials with multiple endpoints. , 2015, Biostatistics.

[26]  David M Kent,et al.  Baseline characteristics predict risk of progression and response to combined medical therapy for benign prostatic hyperplasia (BPH) , 2015, BJU international.

[27]  Antonio Colombo,et al.  Percutaneous coronary intervention versus coronary-artery bypass grafting for severe coronary artery disease. , 2009, The New England journal of medicine.

[28]  Mark J van der Laan,et al.  Targeted Learning of the Mean Outcome under an Optimal Dynamic Treatment Rule , 2015, Journal of causal inference.

[29]  James F Burke,et al.  Implications of Heterogeneity of Treatment Effect for Reporting and Analysis of Randomized Trials in Critical Care. , 2015, American journal of respiratory and critical care medicine.

[30]  Gordon H. Guyatt,et al.  Users' guides to the medical literature. II. How to use an article about therapy or prevention. A. Are the results of the study valid? Evidence-Based Medicine Working Group. , 1993, JAMA.

[31]  Helena Chmura Kraemer,et al.  Discovering, comparing, and combining moderators of treatment on outcome after randomized clinical trials: a parametric approach , 2013, Statistics in medicine.

[32]  David M Kent,et al.  Against pragmatism: on efficacy, effectiveness and the real world , 2009, Trials.

[33]  Sally Morton,et al.  The Predictive Approaches to Treatment effect Heterogeneity (PATH) Statement: Explanation and Elaboration , 2019, Annals of Internal Medicine.

[34]  Menggang Yu,et al.  Regularized outcome weighted subgroup identification for differential treatment effects , 2015, Biometrics.

[35]  Richard D Riley,et al.  Explicit inclusion of treatment in prognostic modeling was recommended in observational and randomized settings. , 2016, Journal of clinical epidemiology.

[36]  Sanjay Basu,et al.  Benefit and harm of intensive blood pressure treatment: Derivation and validation of risk models using data from the SPRINT and ACCORD trials , 2017, PLoS medicine.

[37]  Wenting Cheng,et al.  Reader reaction to “A robust method for estimating optimal treatment regimes” by Zhang et al. (2012) , 2015, Biometrics.

[38]  Laura Lovato,et al.  Effects of High Density Lipoprotein Raising Therapies on Cardiovascular Outcomes in Patients with Type 2 Diabetes Mellitus, with or without Renal Impairment: The Action to Control Cardiovascular Risk in Diabetes Study , 2016, American Journal of Nephrology.

[39]  Max Welling,et al.  Causal Effect Inference with Deep Latent-Variable Models , 2017, NIPS 2017.

[40]  Ravi Varadhan,et al.  A framework for the analysis of heterogeneity of treatment effect in patient-centered outcomes research. , 2013, Journal of clinical epidemiology.

[41]  Yvonne Vergouwe,et al.  Estimates of absolute treatment benefit for individual patients required careful modeling of statistical interactions. , 2015, Journal of clinical epidemiology.

[42]  Michael J. Pencina,et al.  Use of Open Access Platforms for Clinical Trial Data. , 2016, JAMA.

[43]  Robert Tibshirani,et al.  A comparison of methods for model selection when estimating individual treatment effects , 2018, 1804.05146.

[44]  J. Norrie,et al.  Pragmatic Trials. , 2016, The New England journal of medicine.

[45]  David M. Kent,et al.  Using Internally Developed Risk Models to Assess Heterogeneity in Treatment Effects in Clinical Trials , 2014, Circulation. Cardiovascular quality and outcomes.

[46]  David M Kent,et al.  Assessing and reporting heterogeneity in treatment effects in clinical trials: a proposal , 2010, Trials.

[47]  Eric B. Laber,et al.  Doubly Robust Learning for Estimating Individualized Treatment with Censored Data. , 2015, Biometrika.

[48]  David M Kent,et al.  Limitations of applying summary results of clinical trials to individual patients: the need for risk stratification. , 2007, JAMA.

[49]  Daniel J Sargent,et al.  The Fundamental Difficulty With Evaluating the Accuracy of Biomarkers for Guiding Treatment. , 2015, Journal of the National Cancer Institute.

[50]  Federico Rotolo,et al.  Identification of biomarker‐by‐treatment interactions in randomized clinical trials with survival outcomes and high‐dimensional spaces , 2016, Biometrical journal. Biometrische Zeitschrift.

[51]  Ewout W Steyerberg,et al.  Models with interactions overestimated heterogeneity of treatment effects and were prone to treatment mistargeting. , 2019, Journal of clinical epidemiology.

[52]  Ewout W Steyerberg,et al.  The proposed 'concordance-statistic for benefit' provided a useful metric when modeling heterogeneous treatment effects. , 2018, Journal of clinical epidemiology.

[53]  S. Murphy,et al.  PERFORMANCE GUARANTEES FOR INDIVIDUALIZED TREATMENT RULES. , 2011, Annals of statistics.

[54]  Lu Tian,et al.  Effectively Selecting a Target Population for a Future Comparative Study , 2013, Journal of the American Statistical Association.

[55]  Issa J Dahabreh,et al.  Using group data to treat individuals: understanding heterogeneous treatment effects in the age of precision medicine and patient-centred evidence. , 2016, International journal of epidemiology.

[56]  Trevor Hastie,et al.  Some methods for heterogeneous treatment effect estimation in high dimensions , 2017, Statistics in medicine.

[57]  Sally Morton,et al.  The Predictive Approaches to Treatment effect Heterogeneity (PATH) Statement , 2019, Annals of Internal Medicine.

[58]  Holly Janes,et al.  Assessing Treatment‐Selection Markers using a Potential Outcomes Framework , 2012, Biometrics.

[59]  Matthew M. Chingos,et al.  The Review of Economics and Statistics , 2018 .

[60]  S. Schinner,et al.  Effects of Intensive Glucose Lowering in Type 2 Diabetes , 2009 .

[61]  F E Harrell,et al.  Selection of thrombolytic therapy for individual patients: development of a clinical model. GUSTO-I Investigators. , 1997, American heart journal.

[62]  Sören R. Künzel,et al.  Metalearners for estimating heterogeneous treatment effects using machine learning , 2017, Proceedings of the National Academy of Sciences.

[63]  Eva Petkova,et al.  Generated effect modifiers (GEM's) in randomized clinical trials. , 2017, Biostatistics.

[64]  Niko Kaciroti,et al.  Simple subgroup approximations to optimal treatment regimes from randomized clinical trial data. , 2015, Biostatistics.

[65]  Debashis Ghosh,et al.  Bayesian Variable Selection with Joint Modeling of Categorical and Survival Outcomes: An Application to Individualizing Chemotherapy Treatment in Advanced Colorectal Cancer , 2009, Biometrics.

[66]  Jennifer L Krull,et al.  Advancing Personalized Medicine: Application of a Novel Statistical Method to Identify Treatment Moderators in the Coordinated Anxiety Learning and Management Study. , 2017, Behavior therapy.

[67]  Min Zhang,et al.  Estimating optimal treatment regimes from a classification perspective , 2012, Stat.

[68]  Hilde van der Togt,et al.  Publisher's Note , 2003, J. Netw. Comput. Appl..

[69]  H. Daudt,et al.  Enhancing the scoping study methodology: a large, inter-professional team’s experience with Arksey and O’Malley’s framework , 2013, BMC Medical Research Methodology.

[70]  Harlan M Krumholz,et al.  Overview and experience of the YODA Project with clinical trial data sharing after 5 years , 2018, Scientific Data.

[71]  Mark J van der Laan,et al.  Evaluating the impact of treating the optimal subgroup , 2016, Statistical methods in medical research.

[72]  M. J. van der Laan,et al.  STATISTICAL INFERENCE FOR THE MEAN OUTCOME UNDER A POSSIBLY NON-UNIQUE OPTIMAL TREATMENT STRATEGY. , 2016, Annals of statistics.

[73]  S. Athey,et al.  Generalized random forests , 2016, The Annals of Statistics.

[74]  Lacey Gunter,et al.  Variable Selection for Qualitative Interactions in Personalized Medicine While Controlling the Family-Wise Error Rate , 2011, Journal of biopharmaceutical statistics.

[75]  J. Lellouch,et al.  Explanatory and pragmatic attitudes in therapeutical trials. , 1967, Journal of chronic diseases.

[76]  Peter Fayers,et al.  Can overall results of clinical trials be applied to all patients? , 1995, The Lancet.

[77]  L. Caplan,et al.  Evidence based medicine: concerns of a clinical neurologist , 2001, Journal of neurology, neurosurgery, and psychiatry.

[78]  C.J.H. Mann,et al.  Clinical Prediction Models: A Practical Approach to Development, Validation and Updating , 2009 .

[79]  Andrew J Vickers,et al.  Method for evaluating prediction models that apply the results of randomized trials to individual patients , 2007, Trials.

[80]  James A Hanley,et al.  Profile-specific survival estimates: Making reports of clinical trials more patient-relevant , 2008, Clinical trials.

[81]  L. Tian,et al.  Analysis of randomized comparative clinical trial data for personalized treatment selections. , 2011, Biostatistics.