Comparison of Methods to Generalize Randomized Clinical Trial Results Without Individual-Level Data for the Target Population

Our study explored the application of methods to generalize randomized controlled trial results to a target population without individual-level data. We compared 4 methods using aggregate data for the target population to generalize results from the international trial, Justification for the Use of Statins in Prevention: an Intervention Trial Evaluating Rosuvastatin (JUPITER), to a target population of trial-eligible patients in the UK Clinical Practice Research Datalink (CPRD). The gold-standard method used individual data from both the trial and CPRD to predict probabilities of being sampled in the trial and to reweight trial participants to reflect CPRD patient characteristics. Methods 1 and 2 used weighting methods based on simulated individual data or the method of moments, respectively. Method 3 weighted the trial's subgroup-specific treatment effects to match the distribution of an effect modifier in CPRD. Method 4 calculated the expected absolute benefits in CPRD assuming homogeneous relative treatment effect. Methods based on aggregate data for the target population generally yielded results between the trial and gold-standard estimates. Methods 1 and 2 yielded estimates closest to the gold-standard estimates when continuous effect modifiers were represented as categorical variables. Although individual data or data on joint distributions remains the best approach to generalize trial results, these methods using aggregate data might be useful tools for timely assessment of randomized trial generalizability.

[1]  Jeffrey A. Smith,et al.  Does Matching Overcome Lalonde's Critique of Nonexperimental Estimators? , 2000 .

[2]  Patrick Royston,et al.  The cost of dichotomising continuous variables , 2006, BMJ : British Medical Journal.

[3]  Stephen H Bell,et al.  External Validity in Policy Evaluations that Choose Sites Purposively. , 2013, Journal of policy analysis and management : [the journal of the Association for Public Policy Analysis and Management].

[4]  Ingeborg Waernbaum,et al.  Correlation and efficiency of propensity score-based estimators for average causal effects , 2015, Commun. Stat. Simul. Comput..

[5]  S. Cole,et al.  Generalizing Randomized Clinical Trial Results: Implementation and Challenges Related to Missing Data in the Target Population , 2018, American journal of epidemiology.

[6]  J. Caro,et al.  Generalizing the results of clinical trials to actual practice: the example of clopidogrel therapy for the prevention of vascular events. CAPRA (CAPRIE Actual Practice Rates Analysis) Study Group. Clopidogrel versus Aspirin in Patients at Risk of Ischaemic Events. , 1999, The American journal of medicine.

[7]  Mei Lu,et al.  Matching-adjusted indirect comparisons: a new tool for timely comparative effectiveness research. , 2012, Value in health : the journal of the International Society for Pharmacoeconomics and Outcomes Research.

[8]  S. Cole,et al.  Generalizing evidence from randomized clinical trials to target populations: The ACTG 320 trial. , 2010, American journal of epidemiology.

[9]  S. Cole,et al.  Practice of Epidemiology Estimation of the Standardized Risk Difference and Ratio in a Competing Risks Framework : Application to Injection Drug Use and Progression to AIDS After Initiation of Antiretroviral Therapy , 2015 .

[10]  J. Dalton,et al.  A unified approach to measuring the effect size between two groups using SAS , 2012 .

[11]  Paul M. Ridker,et al.  Rosuvastatin to prevent vascular events in men and women with elevated C-reactive protein , 2009 .

[12]  Peter Sandercock,et al.  Interpretation of the evidence for the efficacy and safety of statin therapy , 2016, The Lancet.

[13]  Stephen R Cole,et al.  Transportability of Trial Results Using Inverse Odds of Sampling Weights. , 2017, American journal of epidemiology.

[14]  Catherine P. Bradshaw,et al.  The use of propensity scores to assess the generalizability of results from randomized trials , 2011, Journal of the Royal Statistical Society. Series A,.

[15]  M. Stuart,et al.  Rosuvastatin to Prevent Vascular Events in Men and Women with Elevated C-Reactive Protein (CRP) , 2009 .

[16]  Robert J. Glynn,et al.  Rosuvastatin for Primary Prevention in Older Persons With Elevated C-Reactive Protein and Low to Average Low-Density Lipoprotein Cholesterol Levels: Exploratory Analysis of a Randomized Trial , 2010, Annals of Internal Medicine.

[17]  Sofia Dias,et al.  Methods for Population-Adjusted Indirect Comparisons in Health Technology Appraisal , 2017, Medical decision making : an international journal of the Society for Medical Decision Making.

[18]  Daniel F. McCaffrey,et al.  Comment: Demystifying Double Robustness: A Comparison of Alternative Strategies for Estimating a Population Mean from Incomplete Data , 2008, 0804.2962.

[19]  T. VanderWeele,et al.  Evaluating Public Health Interventions: 7. Let the Subject Matter Choose the Effect Measure: Ratio, Difference, or Something Else Entirely , 2018, American Journal of Public Health.

[20]  Eric Q. Wu,et al.  Comparative Effectiveness Without Head-to-Head Trials , 2012, PharmacoEconomics.

[21]  C. Drake Effects of misspecification of the propensity score on estimators of treatment effect , 1993 .

[22]  P. Ridker,et al.  Efficacy of rosuvastatin among men and women with moderate chronic kidney disease and elevated high-sensitivity C-reactive protein: a secondary analysis from the JUPITER (Justification for the Use of Statins in Prevention-an Intervention Trial Evaluating Rosuvastatin) trial. , 2010, Journal of the American College of Cardiology.

[23]  Jennifer Hill,et al.  Assessing Methods for Generalizing Experimental Impact Estimates to Target Populations , 2016, Journal of research on educational effectiveness.