Dynamically Extracting Outcome-Specific Problem Lists from Clinical Notes with Guided Multi-Headed Attention

Problem lists are intended to provide clinicians with a relevant summary of patient medical issues and are embedded in many electronic health record systems. Despite their importance, problem lists are often cluttered with resolved or currently irrelevant conditions. In this work, we develop a novel end-to-end framework that first extracts diagnosis and procedure information from clinical notes and subsequently uses the extracted medical problems to predict patient outcomes. This framework is both more performant and more interpretable than existing models used within the domain, achieving an AU-ROC of 0.710 for bounceback readmission and 0.869 for in-hospital mortality occurring after ICU discharge. We identify risk factors for both readmission and mortality outcomes and demonstrate that our framework can be used to develop dynamic problem lists that present clinical problems along with their quantitative importance. We conduct a qualitative user study with medical experts and demonstrate that they view the lists produced by our framework favorably and find them to be a more effective clinical decision support tool than a strong baseline.

[1]  P. Duncan,et al.  Inaccuracy of the International Classification of Diseases (ICD-9-CM) in identifying the diagnosis of ischemic cerebrovascular disease , 1997, Neurology.

[2]  Anna Rumshisky,et al.  Unfolding physiological state: mortality modelling in intensive care units , 2014, KDD.

[3]  Casey Holmes The problem list beyond meaningful use. Part I: The problems with problem lists. , 2011, Journal of AHIMA.

[4]  Yuval Pinter,et al.  Attention is not not Explanation , 2019, EMNLP.

[5]  S. Normand,et al.  An Administrative Claims Model Suitable for Profiling Hospital Performance Based on 30-Day Mortality Rates Among Patients With an Acute Myocardial Infarction , 2006, Circulation.

[6]  N. Cox,et al.  Evaluating phecodes, clinical classification software, and ICD-9-CM codes for phenome-wide association studies in the electronic health record , 2017, PloS one.

[7]  AndreasTerént,et al.  Identification of Incident Stroke in Norway , 1999 .

[8]  Adam Wright,et al.  Healthcare provider attitudes towards the problem list in an electronic health record: a mixed-methods qualitative study , 2012, BMC Medical Informatics and Decision Making.

[9]  Oscar Perez-Concha,et al.  A Deep Representation of Longitudinal EMR Data Used for Predicting Readmission to the ICU and Describing Patients-at-Risk , 2019, ArXiv.

[10]  Byron C. Wallace,et al.  An Analysis of Attention over Clinical Notes for Predictive Tasks , 2019, Proceedings of the 2nd Clinical Natural Language Processing Workshop.

[11]  E. Fisher,et al.  The accuracy of Medicare's hospital claims data: progress has been made, but problems remain. , 1992, American journal of public health.

[12]  Zachariah Zhang,et al.  Deep EHR: Chronic Disease Prediction Using Medical Notes , 2018, MLHC.

[13]  Mary G. George,et al.  Accuracy of ICD‐9‐CM Codes by Hospital Characteristics and Stroke Severity: Paul Coverdell National Acute Stroke Program , 2016, Journal of the American Heart Association.

[14]  Wei-Hung Weng,et al.  Publicly Available Clinical BERT Embeddings , 2019, Proceedings of the 2nd Clinical Natural Language Processing Workshop.

[15]  Xia Hu,et al.  Techniques for interpretable machine learning , 2018, Commun. ACM.

[16]  Yoon Kim,et al.  Convolutional Neural Networks for Sentence Classification , 2014, EMNLP.

[17]  B. Gage,et al.  Accuracy of ICD-9-CM Codes for Identifying Cardiovascular and Stroke Risk Factors , 2005, Medical care.

[18]  W. Rosamond,et al.  Validity of Hospital Discharge Diagnosis Codes for Stroke: The Atherosclerosis Risk in Communities Study , 2014, Stroke.

[19]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[20]  D. Sontag,et al.  Comparison of Approaches for Heart Failure Case Identification From Electronic Health Record Data. , 2016, JAMA cardiology.

[21]  Harlan M Krumholz,et al.  Relationship between hospital readmission and mortality rates for patients hospitalized with acute myocardial infarction, heart failure, or pneumonia. , 2013, JAMA.

[22]  Jimeng Sun,et al.  Explainable Prediction of Medical Codes from Clinical Text , 2018, NAACL.

[23]  Susan Hutfless,et al.  Mining high-dimensional administrative claims data to predict early hospital readmissions , 2014, J. Am. Medical Informatics Assoc..

[24]  Anna Goldenberg,et al.  What Clinicians Want: Contextualizing Explainable Machine Learning for Clinical End Use , 2019, MLHC.

[25]  C. Kooperberg,et al.  Comparison of self-report, hospital discharge codes, and adjudication of cardiovascular events in the Women's Health Initiative. , 2004, American journal of epidemiology.

[26]  B. Kissela,et al.  Validity of Claims-Based Stroke Algorithms in Contemporary Medicare Data: Reasons for Geographic and Racial Differences in Stroke (REGARDS) Study Linked With Medicare Claims , 2014, Circulation. Cardiovascular quality and outcomes.

[27]  Jeffrey Dean,et al.  Distributed Representations of Words and Phrases and their Compositionality , 2013, NIPS.

[28]  Jaideep Srivastava,et al.  Using Clinical Notes with Time Series Data for ICU Management , 2019, EMNLP/IJCNLP.

[29]  Marylyn D. Ritchie,et al.  PheWAS: demonstrating the feasibility of a phenome-wide scan to discover gene–disease associations , 2010, Bioinform..

[30]  Ming-Wei Chang,et al.  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding , 2019, NAACL.

[31]  Jimeng Sun,et al.  Pre-training of Graph Augmented Transformers for Medication Recommendation , 2019, IJCAI.

[32]  P. Kristensen,et al.  ...or in Norway , 1994, BMJ.

[33]  David Suendermann-Oeft,et al.  Medical code prediction with multi-view convolution and description-regularized label-dependent attention , 2018, ArXiv.

[34]  Bobak Mortazavi,et al.  Prediction of ICU Readmissions Using Data at Patient Discharge , 2018, 2018 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[35]  Peter Szolovits,et al.  MIMIC-III, a freely accessible critical care database , 2016, Scientific Data.

[36]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[37]  W. Winkelmayer,et al.  Comparison of Medicare Claims Versus Physician Adjudication for Identifying Stroke Outcomes in the Women’s Health Initiative , 2014, Stroke.

[38]  Crystal Kallem,et al.  Problem list guidance in the EHR. , 2011, Journal of AHIMA.

[39]  Adler J. Perotte,et al.  Phenotype inference with Semi-Supervised Mixed Membership Models , 2019, MLHC.

[40]  Yoshua Bengio,et al.  Show, Attend and Tell: Neural Image Caption Generation with Visual Attention , 2015, ICML.

[41]  Byron C. Wallace,et al.  Attention is not Explanation , 2019, NAACL.

[42]  Pengtao Xie,et al.  Multimodal Machine Learning for Automated ICD Coding , 2018, MLHC.