An outcome model approach to transporting a randomized controlled trial results to a target population

Objective Participants enrolled into randomized controlled trials (RCTs) often do not reflect real‐world populations. Previous research in how best to transport RCT results to target populations has focused on weighting RCT data to look like the target data. Simulation work, however, has suggested that an outcome model approach may be preferable. Here, we describe such an approach using source data from the 2 × 2 factorial NAVIGATOR (Nateglinide And Valsartan in Impaired Glucose Tolerance Outcomes Research) trial, which evaluated the impact of valsartan and nateglinide on cardiovascular outcomes and new‐onset diabetes in a prediabetic population. Materials and Methods Our target data consisted of people with prediabetes serviced at the Duke University Health System. We used random survival forests to develop separate outcome models for each of the 4 treatments, estimating the 5‐year risk difference for progression to diabetes, and estimated the treatment effect in our local patient populations, as well as subpopulations, and compared the results with the traditional weighting approach. Results Our models suggested that the treatment effect for valsartan in our patient population was the same as in the trial, whereas for nateglinide treatment effect was stronger than observed in the original trial. Our effect estimates were more efficient than the weighting approach and we effectively estimated subgroup differences. Conclusions The described method represents a straightforward approach to efficiently transporting an RCT result to any target population.

[1]  Yangxin Huang,et al.  Generalizability analysis for clinical trials: a simulation study , 2017, Statistics in medicine.

[2]  S. Cole,et al.  Correction: Sensitivity analysis for an unobserved moderator in RCT-to-target-population generalization of treatment effects , 2016, The Annals of Applied Statistics.

[3]  Shuang Wang,et al.  Assessing the Collective Population Representativeness of Related Type 2 Diabetes Trials by Combining Public Data from ClinicalTrials.gov and NHANES , 2015, MedInfo.

[4]  M. Flather,et al.  Generalizing results of randomized trials to clinical practice: reliability and cautions , 2006, Clinical trials.

[5]  M. Kivimäki,et al.  Risk of Cardiovascular Disease and Death in Individuals With Prediabetes Defined by Different Criteria: The Whitehall II Study , 2018, Diabetes Care.

[6]  J. Johnston,et al.  A literature review on the representativeness of randomized controlled trial samples and implications for the external validity of trial results , 2015, Trials.

[7]  Sarah E. Robertson,et al.  Generalizing causal inferences from individuals in randomized trials to all trial‐eligible individuals , 2017, Biometrics.

[8]  S. Cole,et al.  Generalizing Randomized Clinical Trial Results: Implementation and Challenges Related to Missing Data in the Target Population , 2018, American journal of epidemiology.

[9]  S. Cole,et al.  Generalizing evidence from randomized clinical trials to target populations: The ACTG 320 trial. , 2010, American journal of epidemiology.

[10]  Jennifer Hill,et al.  Assessing Methods for Generalizing Experimental Impact Estimates to Target Populations , 2016, Journal of research on educational effectiveness.

[11]  Mark McClellan,et al.  Implementing MACRA: Implications for Physicians and for Physician Leadership. , 2016, JAMA.

[12]  G Hripcsak,et al.  A Distribution-based Method for Assessing The Differences between Clinical Trial Target Populations and Patient Populations in Electronic Health Records , 2014, Applied Clinical Informatics.

[13]  Angela M Wood,et al.  The use of repeated blood pressure measures for cardiovascular risk prediction: a comparison of statistical models in the ARIC study , 2016, Statistics in medicine.

[14]  M. Laakso,et al.  Effect of nateglinide on the incidence of diabetes and cardiovascular events. , 2010, The New England journal of medicine.

[15]  Udaya B. Kogalur,et al.  Random Survival Forests for R , 2007 .

[16]  R. Califf,et al.  Prevention of diabetes and cardiovascular disease in patients with impaired glucose tolerance: rationale and design of the Nateglinide And Valsartan in Impaired Glucose Tolerance Outcomes Research (NAVIGATOR) Trial. , 2008, American heart journal.

[17]  W. Hohenberger,et al.  Differences Between Clinical Trial Participants and Patients in a Population-Based Registry: the German Rectal Cancer Study vs. the Rostock Cancer Registry , 2009, Diseases of the colon and rectum.

[18]  M. Laakso,et al.  Effect of valsartan on the incidence of diabetes and cardiovascular events. , 2010, The New England journal of medicine.

[19]  Elizabeth A Stuart,et al.  Generalizability of Randomized Trial Results to Target Populations , 2018, Research on social work practice.

[20]  Eloise E Kaizar,et al.  The use of propensity scores and observational data to estimate randomized controlled trial generalizability bias , 2013, Statistics in medicine.

[21]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[22]  Chunhua Weng,et al.  Sick Patients Have More Data: The Non-Random Completeness of Electronic Health Records , 2013, AMIA.

[23]  Benjamin A. Goldstein,et al.  Illustrating Informed Presence Bias in Electronic Health Records Data: How Patient Interactions with a Health System Can Impact Inference , 2017, EGEMS.

[24]  Marie Lynn Miranda,et al.  Geographic health information systems: a platform to support the 'triple aim'. , 2013, Health affairs.

[25]  Michael G Hudgens,et al.  Generalizing evidence from randomized trials using inverse probability of sampling weights , 2018, Journal of the Royal Statistical Society. Series A,.

[26]  Stefan Wager,et al.  Estimation and Inference of Heterogeneous Treatment Effects using Random Forests , 2015, Journal of the American Statistical Association.

[27]  Hemant Ishwaran,et al.  Random Survival Forests , 2008, Wiley StatsRef: Statistics Reference Online.

[28]  Hans-Ulrich Prokosch,et al.  Evaluation of data completeness in the electronic health record for the purpose of patient recruitment into clinical trials: a retrospective analysis of element presence , 2013, BMC Medical Informatics and Decision Making.

[29]  Hemant Ishwaran,et al.  Estimating Individual Treatment Effect in Observational Data Using Random Forest Methods , 2017, Journal of computational and graphical statistics : a joint publication of American Statistical Association, Institute of Mathematical Statistics, Interface Foundation of North America.

[30]  Michael J Pencina,et al.  A comparison of risk prediction methods using repeated observations: an application to electronic health records for hemodialysis. , 2017, Statistics in medicine.

[31]  Trevor Hastie,et al.  Regularization Paths for Cox's Proportional Hazards Model via Coordinate Descent. , 2011, Journal of statistical software.

[32]  Zab Mosenifar,et al.  Population issues in clinical trials. , 2007, Proceedings of the American Thoracic Society.

[33]  Stephen R Cole,et al.  Transportability of Trial Results Using Inverse Odds of Sampling Weights. , 2017, American journal of epidemiology.

[34]  Catherine P. Bradshaw,et al.  The use of propensity scores to assess the generalizability of results from randomized trials , 2011, Journal of the Royal Statistical Society. Series A,.

[35]  D. Rubin,et al.  Constructing a Control Group Using Multivariate Matched Sampling Methods That Incorporate the Propensity Score , 1985 .