Assessing the Validity of a a priori Patient-Trial Generalizability Score using Real-world Data from a Large Clinical Data Research Network: A Colorectal Cancer Clinical Trial Case Study

Existing trials had not taken enough consideration of their population representativeness, which can lower the effectiveness when the treatment is applied in real-world clinical practice. We analyzed the eligibility criteria of Bevacizumab colorectal cancer treatment trials, assessed their a priori generalizability, and examined how it affects patient outcomes when applied in real-world clinical settings. To do so, we extracted patient-level data from a large collection of electronic health records (EHRs) from the OneFlorida consortium. We built a zero-inflated negative binomial model using a composite patient-trial generalizability (cPTG) score to predict patients' clinical outcomes (i.e., number of serious adverse events, [SAEs]). Our study results provide a body of evidence that 1) the cPTG scores can predict patient outcomes; and 2) patients who are more similar to the study population in the trials that were used to develop the treatment will have a significantly lower possibility to experience serious adverse events.

[1]  S. Mathoulin-Pélissier,et al.  Barriers to inclusion of older adults in randomised controlled clinical trials on Non-Hodgkin's lymphoma: a systematic review. , 2013, Cancer treatment reviews.

[2]  E. Beers,et al.  Participation of Older People in Preauthorization Trials of Recently Approved Medicines , 2014, Journal of the American Geriatrics Society.

[3]  Patrick B. Ryan,et al.  Simulation-based Evaluation of the Generalizability Index for Study Traits , 2015, AMIA.

[4]  A Oakley,et al.  Assessment of generalisability in trials of health interventions: suggested framework and systematic review , 2006, BMJ : British Medical Journal.

[5]  Edward S. Kim,et al.  Broadening Eligibility Criteria to Make Clinical Trials More Representative: American Society of Clinical Oncology and Friends of Cancer Research Joint Research Statement. , 2017, Journal of clinical oncology : official journal of the American Society of Clinical Oncology.

[6]  Shuang Wang,et al.  Multivariate analysis of the population representativeness of related clinical studies , 2016, J. Biomed. Informatics.

[7]  C. Gotay Increasing trial generalizability. , 2006, Journal of clinical oncology : official journal of the American Society of Clinical Oncology.

[8]  P. Sedgwick External and internal validity in clinical trials , 2012, BMJ : British Medical Journal.

[9]  E. Shenkman,et al.  OneFlorida Clinical Research Consortium: Linking a Clinical and Translational Science Institute With a Community-Based Distributive Medical Education Model , 2017, Academic medicine : journal of the Association of American Medical Colleges.

[10]  Mark Zimmerman,et al.  Exclusion Criteria Used in Antidepressant Efficacy Trials: Consistency Across Studies and Representativeness of Samples Included , 2004, The Journal of nervous and mental disease.

[11]  H. White,et al.  The older the better: are elderly study participants more non-representative? A cross-sectional analysis of clinical trial and observational study samples , 2012, BMJ Open.

[12]  G Hripcsak,et al.  A Distribution-based Method for Assessing The Differences between Clinical Trial Target Populations and Patient Populations in Electronic Health Records , 2014, Applied Clinical Informatics.

[13]  Shuang Wang,et al.  GIST 2.0: A scalable multi-trait metric for quantifying population representativeness of individual clinical studies , 2016, J. Biomed. Informatics.

[14]  Jonathan H Seltzer,et al.  Underrepresentation of women, elderly patients, and racial minorities in the randomized trials used for cardiovascular guidelines. , 2014, JAMA internal medicine.

[15]  M. Extermann,et al.  Assessment of the External Validity of the National Comprehensive Cancer Network and European Society for Medical Oncology Guidelines for Non-Small-Cell Lung Cancer in a Population of Patients Aged 80 Years and Older. , 2017, Clinical lung cancer.

[16]  W. A. Gool,et al.  The age gap between patients in clinical studies and in the general population: a pitfall for dementia research , 2004, The Lancet Neurology.

[17]  Chunhua Weng,et al.  Correlating eligibility criteria generalizability and adverse events using Big Data for patients and clinical trials , 2017, Annals of the New York Academy of Sciences.

[18]  P. Rothwell,et al.  External validity of randomised controlled trials: “To whom do the results of this trial apply?” , 2005, The Lancet.

[19]  J. T. Wulu,et al.  Regression analysis of count data , 2002 .

[20]  R. Califf,et al.  Real-World Evidence - What Is It and What Can It Tell Us? , 2016, The New England journal of medicine.

[21]  Joy H. Lewis,et al.  Participation of patients 65 years of age or older in cancer clinical trials. , 2003, Journal of clinical oncology : official journal of the American Society of Clinical Oncology.

[22]  D. Wysowski,et al.  Adverse drug event surveillance and drug withdrawals in the United States, 1969-2002: the importance of reporting suspected reactions. , 2005, Archives of internal medicine.

[23]  P. Keegan,et al.  FDA drug approval summary: bevacizumab plus FOLFOX4 as second-line treatment of colorectal cancer. , 2007, The oncologist.

[24]  Yi Guo,et al.  Computable Eligibility Criteria through Ontology-driven Data Access: A Case Study of Hepatitis C Virus Trials , 2018, AMIA.

[25]  K. Jenpanich,et al.  [Drug administration]. , 1976, Thai journal of nursing.

[26]  Yi Guo,et al.  Implementing a Hash-based Privacy-Preserving Entity Resolution Tool in the OneFlorida Clinical Data Research Network , 2017, AMIA.