Correlating eligibility criteria generalizability and adverse events using Big Data for patients and clinical trials

Randomized controlled trials can benefit from proactive assessment of how well their participant selection strategies during the design of eligibility criteria can influence the study generalizability. In this paper, we present a quantitative metric called generalizability index for study traits 2.0 (GIST 2.0) to assess the a priori generalizability (based on population representativeness) of a clinical trial by accounting for the dependencies among multiple eligibility criteria. The metric was evaluated on 16 sepsis trials identified from ClinicalTrials.gov, with their adverse event reports extracted from the trial results sections. The correlation between GIST scores and adverse events was analyzed. We found that the GIST 2.0 score was significantly correlated with total adverse events and serious adverse events (weighted correlation coefficients of 0.825 and 0.709, respectively, with P < 0.01). This study exemplifies the promising use of Big Data in electronic health records and ClinicalTrials.gov for optimizing eligibility criteria design for clinical studies.

[1]  S. Tu,et al.  Analysis of Eligibility Criteria Complexity in Clinical Trials , 2010, Summit on translational bioinformatics.

[2]  Chunhua Weng,et al.  Identification of Questionable Exclusion Criteria in Mental Disorder Clinical Trials Using a Medical Encyclopedia , 2016, PSB.

[3]  D E Grobbee,et al.  External validation is necessary in prediction research: a clinical example. , 2003, Journal of clinical epidemiology.

[4]  K. Sikaris The Correlation of Hemoglobin A1c to Blood Glucose , 2009, Journal of diabetes science and technology.

[5]  Chunhua Weng,et al.  Optimizing Clinical Research Participant Selection with Informatics. , 2015, Trends in pharmacological sciences.

[6]  Irene Katzan,et al.  The Knowledge Program: an innovative, comprehensive electronic data capture system and warehouse. , 2011, AMIA ... Annual Symposium proceedings. AMIA Symposium.

[7]  Adrian F Hernandez,et al.  Representativeness of RELAX-AHF Clinical Trial Population in Acute Heart Failure , 2014, Circulation. Cardiovascular quality and outcomes.

[8]  D W Bates,et al.  Screening for adverse events. , 1999, Journal of evaluation in clinical practice.

[9]  Harlan M Krumholz,et al.  Most hospitalized older persons do not meet the enrollment criteria for clinical trials in heart failure. , 2003, American heart journal.

[10]  George Hripcsak,et al.  Defining and measuring completeness of electronic health records for secondary use , 2013, J. Biomed. Informatics.

[11]  L. Bäckman,et al.  Effects of multifactorial memory training in old age: generalizability across tasks and individuals. , 1995, The journals of gerontology. Series B, Psychological sciences and social sciences.

[12]  A. Tsertsvadze,et al.  Assessment of generalizability, applicability and predictability (GAP) for evaluating external validity in studies of universal family-based prevention of alcohol misuse in young people: systematic methodological review of randomized controlled trials. , 2012, Addiction.

[13]  Eloise E Kaizar,et al.  The use of propensity scores and observational data to estimate randomized controlled trial generalizability bias , 2013, Statistics in medicine.

[14]  C Wagner,et al.  Exploring the causes of adverse events in hospitals and potential prevention strategies , 2010, Quality and Safety in Health Care.

[15]  B Nebiyou Bekele,et al.  Generalizability of cancer clinical trial results , 2006, Cancer.

[16]  G Hripcsak,et al.  A Distribution-based Method for Assessing The Differences between Clinical Trial Target Populations and Patient Populations in Electronic Health Records , 2014, Applied Clinical Informatics.

[17]  A Oakley,et al.  Assessment of generalisability in trials of health interventions: suggested framework and systematic review , 2006, BMJ : British Medical Journal.

[18]  M K Slack,et al.  Establishing the internal and external validity of experimental studies. , 2001, American journal of health-system pharmacy : AJHP : official journal of the American Society of Health-System Pharmacists.

[19]  A. Hart,et al.  Effects of patient selection on the applicability of results from a randomised clinical trial (EORTC 10853) investigating breast-conserving therapy for DCIS , 2002, British Journal of Cancer.

[20]  Michael G Hudgens,et al.  Generalizing evidence from randomized trials using inverse probability of sampling weights , 2018, Journal of the Royal Statistical Society. Series A,.

[21]  J. Pearl Generalizing Experimental Findings , 2015 .

[22]  W. A. Gool,et al.  The age gap between patients in clinical studies and in the general population: a pitfall for dementia research , 2004, The Lancet Neurology.

[23]  T. J. Moore,et al.  Serious adverse drug events reported to the Food and Drug Administration, 1998-2005. , 2007, Archives of internal medicine.

[24]  Tianyong Hao,et al.  Clustering clinical trials with similar eligibility criteria features , 2014, J. Biomed. Informatics.

[25]  Patricia J. Jordan,et al.  Screening, safety, and adverse events in physical activity interventions: Collaborative experiences from the behavior change consortium , 2005, Annals of behavioral medicine : a publication of the Society of Behavioral Medicine.

[26]  Daniel L. Rubin,et al.  Knowledge representation and tool support for critiquing clinical trial protocols , 2000, AMIA.

[27]  P. Marik,et al.  The definition of septic shock: implications for treatment. , 2007, Critical care and resuscitation : journal of the Australasian Academy of Critical Care Medicine.

[28]  R. Pretorius,et al.  Reducing the risk of adverse drug events in older adults. , 2013, American family physician.

[29]  Shuang Wang,et al.  Multivariate analysis of the population representativeness of related clinical studies , 2016, J. Biomed. Informatics.

[30]  Elizabeth A Stuart,et al.  Assessing sample representativeness in randomized controlled trials: application to the National Institute of Drug Abuse Clinical Trials Network. , 2016, Addiction.

[31]  Lawrence M. Fagan,et al.  Knowledge engineering for a clinical trial advice system: uncovering errors in protocol specification. , 1987, Bulletin du cancer.

[32]  J. Bostwick,et al.  Antidepressant treatment concerns and recommendations for avoiding adverse events , 2014 .

[33]  William Gardner,et al.  Generalizing from clinical trial data: A case study. The risk of suicidality among pediatric antidepressant users , 2008, Statistics in medicine.

[34]  Chunhua Weng,et al.  Sick Patients Have More Data: The Non-Random Completeness of Electronic Health Records , 2013, AMIA.

[35]  W. Kukull,et al.  Generalizability , 2012, Neurology.