Data Consult Service: Can we use observational data to address immediate clinical needs?

OBJECTIVE A number of clinical decision support tools aim to use observational data to address immediate clinical needs, but few of them address challenges and biases inherent in such data. The goal of this article is to describe the experience of running a data consult service that generates clinical evidence in real time and characterize the challenges related to its use of observational data. MATERIALS AND METHODS In 2019, we launched the Data Consult Service pilot with clinicians affiliated with Columbia University Irving Medical Center. We created and implemented a pipeline (question gathering, data exploration, iterative patient phenotyping, study execution, and assessing validity of results) for generating new evidence in real time. We collected user feedback and assessed issues related to producing reliable evidence. RESULTS We collected 29 questions from 22 clinicians through clinical rounds, emails, and in-person communication. We used validated practices to ensure reliability of evidence and answered 24 of them. Questions differed depending on the collection method, with clinical rounds supporting proactive team involvement and gathering more patient characterization questions and questions related to a current patient. The main challenges we encountered included missing and incomplete data, underreported conditions, and nonspecific coding and accurate identification of drug regimens. CONCLUSIONS While the Data Consult Service has the potential to generate evidence and facilitate decision making, only a portion of questions can be answered in real time. Recognizing challenges in patient phenotyping and designing studies along with using validated practices for observational research are mandatory to produce reliable evidence.

[1]  Jason W Busse,et al.  What is your research question? An introduction to the PICOT format for clinicians. , 2012, The Journal of the Canadian Chiropractic Association.

[2]  George Hripcsak,et al.  Principles of Large-scale Evidence Generation and Evaluation across a Network of Databases (LEGEND) , 2020, J. Am. Medical Informatics Assoc..

[3]  D. Rubin,et al.  Reducing Bias in Observational Studies Using Subclassification on the Propensity Score , 1984 .

[4]  Ross D. Williams,et al.  Development and validation of a prognostic model predicting symptomatic hemorrhagic transformation in acute ischemic stroke at scale in the OHDSI network , 2020, PloS one.

[5]  M. Schuemie,et al.  Channeling in the Use of Nonprescription Paracetamol and Ibuprofen in an Electronic Medical Records Database: Evidence and Implications , 2017, Drug Safety.

[6]  William DuMouchel,et al.  Interpreting observational studies: why empirical calibration is needed to correct p-values , 2013, Statistics in medicine.

[7]  Charles E. Leonard,et al.  Diagnostic Algorithms for Cardiovascular Death in Administrative Claims Databases: A Systematic Review , 2018, Drug Safety.

[8]  George Hripcsak,et al.  Risk of angioedema associated with levetiracetam compared with phenytoin: Findings of the observational health data sciences and informatics research network , 2017, Epilepsia.

[9]  George Hripcsak,et al.  Risk of hydroxychloroquine alone and in combination with azithromycin in the treatment of rheumatoid arthritis: a multinational, retrospective study , 2020, The Lancet Rheumatology.

[10]  L. Logan,et al.  Ceftriaxone-Associated Biliary and Cardiopulmonary Adverse Events in Neonates: A Systematic Review of the Literature , 2017, Pediatric Drugs.

[11]  S. Schneeweiss,et al.  Considerations for the analysis of longitudinal electronic health records linked to claims data to study the effectiveness and safety of drugs , 2016, Clinical pharmacology and therapeutics.

[12]  Vijay Sivaraman,et al.  Bringing cohort studies to the bedside: framework for a 'green button' to support clinical decision-making. , 2015, Journal of comparative effectiveness research.

[13]  George Hripcsak,et al.  Adapting electronic health records-derived phenotypes to claims data: Lessons learned in using limited clinical data for phenotyping , 2019, J. Biomed. Informatics.

[14]  P. Gorman,et al.  Clinical questions raised by clinicians at the point of care: a systematic review. , 2014, JAMA internal medicine.

[15]  George Hripcsak,et al.  Association of Hemoglobin A1c Levels With Use of Sulfonylureas, Dipeptidyl Peptidase 4 Inhibitors, and Thiazolidinediones in Patients With Type 2 Diabetes Treated With Metformin , 2018, JAMA network open.

[16]  G. Caughey,et al.  Documentation of penicillin adverse drug reactions in electronic health records: inconsistent use of allergy and intolerance labels , 2017, Internal medicine journal.

[17]  M. Ebell,et al.  Analysis of questions asked by family doctors regarding patient care , 1999, BMJ.

[18]  G. Hripcsak,et al.  Characterizing physicians’ information needs related to a gap in knowledge unmet by current evidence , 2020, JAMIA open.

[19]  George Hripcsak,et al.  PheValuator: Development and evaluation of a phenotype algorithm evaluator , 2019, J. Biomed. Informatics.

[20]  Paul A. Harris,et al.  PheKB: a catalog and workflow for creating electronic phenotype algorithms for transportability , 2016, J. Am. Medical Informatics Assoc..

[21]  J. Karnon,et al.  Variation in Clinical Practice: A Priority Setting Approach to the Staged Funding of Quality Improvement , 2016, Applied Health Economics and Health Policy.

[22]  Rod J. Rohrich,et al.  The Levels of Evidence and Their Role in Evidence-Based Medicine , 2011, Plastic and reconstructive surgery.

[23]  G. Hripcsak,et al.  Metformin Is Associated With a Lower Risk of Atrial Fibrillation and Ventricular Arrhythmias Compared With Sulfonylureas , 2021, Circulation. Arrhythmia and electrophysiology.

[24]  George Hripcsak,et al.  Comprehensive comparative effectiveness and safety of first-line antihypertensive drug classes: a systematic, multinational, large-scale analysis , 2019, The Lancet.

[25]  Amit X Garg,et al.  Detecting chronic kidney disease in population-based administrative databases using an algorithm of hospital encounter and physician claim codes , 2013, BMC Nephrology.

[26]  S. Cuschieri.  The STROBE guidelines , 2019, Saudi journal of anaesthesia.

[27]  David A Cook,et al.  Barriers and decisions when answering clinical questions at the point of care: a grounded theory study. , 2013, JAMA internal medicine.

[28]  Himanshu Sharma,et al.  Developing a portable natural language processing based phenotyping system , 2018, BMC Medical Informatics and Decision Making.

[29]  Anna Ostropolets,et al.  A scoping review of clinical decision support tools that generate new knowledge to support decision making in real time , 2020, J. Am. Medical Informatics Assoc..

[30]  Spiros C. Denaxas,et al.  Big data from electronic health records for early and late translational cardiovascular research: challenges and potential , 2017, European heart journal.

[31]  James M. Walker,et al.  Bridging the inferential gap: the electronic health record and clinical evidence. , 2007, Health affairs.

[32]  Catherine P. Bradshaw,et al.  Assessing the Generalizability of Randomized Trial Results to Target Populations , 2015, Prevention Science.

[33]  J. Busse,et al.  Evidence-based medicine and precision medicine: Complementary approaches to clinical decision-making , 2018, Precision clinical medicine.

[34]  M. Graber,et al.  Diagnosis: Interpreting the Shadows , 2017 .

[35]  N B Giuse,et al.  Clinical medical librarianship: the Vanderbilt experience. , 1998, Bulletin of the Medical Library Association.

[36]  N. Shah,et al.  A 'green button' for using aggregate patient data at the point of care. , 2014, Health affairs.

[37]  Banafsheh Sadeghi,et al.  Evaluation of the predictive value of ICD-9-CM coded administrative data for venous thromboembolism in the United States. , 2010, Thrombosis research.

[38]  Martijn J Schuemie,et al.  Evaluating large-scale propensity score performance through real-world and synthetic data experiments. , 2018, International journal of epidemiology.

[39]  Lucy Savitz,et al.  Challenges in Using Electronic Health Record Data for CER: Experience of 4 Learning Organizations and Solutions Applied , 2013, Medical care.

[40]  D. Madigan,et al.  A Systematic Statistical Approach to Evaluating Evidence from Observational Studies , 2014 .

[41]  Wendy Novicoff,et al.  Information exchange in the NICU: what sources of patient data do physicians prefer to use? , 2004, Int. J. Medical Informatics.

[42]  Alison Callahan,et al.  It is time to learn from patients like mine , 2019, npj Digital Medicine.

[43]  P. Falzer.  Evidence-based medicine's curious path: From clinical epidemiology to patient-centered care through decision analysis. , 2020, Journal of evaluation in clinical practice.

[44]  Yu-Chuan Li,et al.  Observational Health Data Sciences and Informatics (OHDSI): Opportunities for Observational Researchers , 2015, MedInfo.

[45]  K. Bollaerts,et al.  Bias due to differential and non-differential disease- and exposure misclassification in studies of vaccine effectiveness , 2018, PloS one.

[46]  Patrick B. Ryan,et al.  Multisite Evaluation of a Data Quality Tool for Patient-Level Clinical Data Sets , 2016, EGEMS.

[47]  G. Guyatt,et al.  Drug treatments for covid-19: living systematic review and network meta-analysis , 2020, BMJ.

[48]  George Hripcsak,et al.  How Confident Are We about Observational Findings in Healthcare: A Benchmark Study. , 2020, Harvard data science review.

[49]  Rae Woong Park,et al.  Characterizing treatment pathways at scale using the OHDSI network , 2016, Proceedings of the National Academy of Sciences.

[50]  Andrew E. Williams,et al.  An international characterisation of patients hospitalised with COVID-19 and a comparison with those previously hospitalised with influenza , 2020, medRxiv.

[51]  The spectre of Berkson's paradox: Collider bias in Covid‐19 research , 2020 .

[52]  George Hripcsak,et al.  Large-scale evidence generation and evaluation across a network of databases (LEGEND): assessing validity using hypertension as a case study , 2020, J. Am. Medical Informatics Assoc..

[53]  R. Kelishadi,et al.  Personal, technical and organisational factors affect whether physicians seek answers to clinical questions during patient care: a literature review. , 2020, Health information and libraries journal.

[54]  Richard Smith.  What clinical information do doctors need? , 1996, BMJ.

[55]  J. Johnston,et al.  A literature review on the representativeness of randomized controlled trial samples and implications for the external validity of trial results , 2015, Trials.

[56]  Melissa A. Basford,et al.  Validation of electronic medical record-based phenotyping algorithms: results and lessons learned from the eMERGE network. , 2013, Journal of the American Medical Informatics Association : JAMIA.

[57]  George Hripcsak,et al.  Next-generation phenotyping of electronic health records , 2012, J. Am. Medical Informatics Assoc..