Prediction of emergency department patient disposition based on natural language processing of triage notes

BACKGROUND Nursing triage documentation is the first free-form text data created at the start of an emergency department (ED) visit. These 1-3 unstructured sentences reflect the clinical impression of an experienced nurse and are key in gauging a patient's illness. We aimed to predict final ED disposition using three commonly-employed natural language processing (NLP) techniques of nursing triage notes in isolation from other data. METHODS We constructed a retrospective cohort of all 260,842 consecutive ED encounters in 2015-16, from three clinically heterogeneous academically-affiliated EDs. After exclusion of 3964 encounters based on completeness of triage, and disposition data, we included 256,878 encounters. We defined the outcome as: 1) admission, transfer, or in-ED death [68,092 encounters] vs. 2) discharge, "left without being seen," and "left against medical advice" [188,786 encounters]. The dataset was divided into training and testing subsets. Neural network regression models were trained using bag-of-words, paragraph vectors, and topic distributions to predict disposition and were evaluated using the testing dataset. RESULTS Area under the curve for disposition using triage notes as bag-of-words, paragraph vectors, and topic distributions were 0.737 (95% CI: 0.734 - 0.740), 0.785 (95% CI: 0.782 - 0.788), and 0.687 (95% CI: 0.684 - 0.690), respectively. CONCLUSIONS Nursing triage notes can be used to predict final ED patient disposition, even when used separately from other clinical information. These findings have substantial implications for future studies, suggesting that free text from medical records may be considered as a critical predictor in research of patient outcomes.

[1]  G. Mckay,et al.  Predicting admission at triage: are nurses better than a simple objective score? , 2016, Emergency Medicine Journal.

[2]  S. Mason,et al.  Comparison of Glasgow Admission Prediction Score and Amb Score in predicting need for inpatient care , 2018, Emergency Medicine Journal.

[3]  Ellen Weber,et al.  Solutions to emergency department 'boarding' and crowding are underused and may need to be legislated. , 2012, Health Affairs.

[4]  E. DeLong,et al.  Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. , 1988, Biometrics.

[5]  A. D. de Craen,et al.  Early prediction of hospital admission for emergency department patients: a comparison between patients younger or older than 70 years , 2017, Emergency Medicine Journal.

[6]  Y. Singer,et al.  Ultraconservative online algorithms for multiclass problems , 2003 .

[7]  Tadahiro Goto,et al.  Emergency department triage prediction of clinical outcomes using machine learning models , 2019, Critical Care.

[8]  Yan Sun,et al.  Predicting hospital admissions at emergency department triage using routine administrative data. , 2011, Academic emergency medicine : official journal of the Society for Academic Emergency Medicine.

[9]  Donald E. Ward.  Classification of Groups , 2006 .

[10]  D. Aronsky,et al.  Predicting hospital admission at triage in an emergency department. , 2007, AMIA ... Annual Symposium proceedings. AMIA Symposium.

[11]  Scott Levin,et al.  Machine‐Learning‐Based Electronic Triage More Accurately Differentiates Patients With Respect to Clinical Outcomes Compared With the Emergency Severity Index , 2017, Annals of emergency medicine.

[12]  Ben Y. Reis,et al.  Progressive prediction of hospitalisation in the emergency department: uncovering hidden patterns to improve patient flow , 2017, Emergency Medicine Journal.

[13]  Arvind Venkat,et al.  Evaluation of a hospital admission prediction model adding coded chief complaint data using neural network methodology , 2015, European journal of emergency medicine : official journal of the European Society for Emergency Medicine.

[14]  Jordan S Peck,et al.  Predicting emergency department inpatient admissions to improve same-day patient flow. , 2012, Academic emergency medicine : official journal of the Society for Academic Emergency Medicine.

[15]  Dominik Aronsky,et al.  Predicting Hospital Admission for Emergency Department Patients using a Bayesian Network , 2005, AMIA.

[16]  Davood Golmohammadi,et al.  Predicting hospital admissions to reduce emergency department boarding , 2016 .

[17]  Xingyu Zhang,et al.  Prediction of Emergency Department Hospital Admission Based on Natural Language Processing and Neural Networks , 2017, Methods of Information in Medicine.

[18]  Karl Pearson F.R.S..  LIII. On lines and planes of closest fit to systems of points in space , 1901 .

[19]  Ken Hillman,et al.  Access block and emergency department overcrowding , 2011, Critical care.

[20]  Jing Zhou,et al.  Hate Speech Detection with Comment Embeddings , 2015, WWW.

[21]  Donald B. Chalfin,et al.  Impact of delayed transfer of critically ill patients from the emergency department to the intensive care unit* , 2007, Critical care medicine.

[22]  R. Ivers,et al.  The Sydney Triage to Admission Risk Tool (START) to predict Emergency Department Disposition: A derivation and internal validation study using retrospective state-wide data from New South Wales, Australia , 2016, BMC Emergency Medicine.

[23]  Tadahiro Goto,et al.  Machine Learning–Based Prediction of Clinical Outcomes for Children During Emergency Department Triage , 2019, JAMA network open.

[24]  M. Dinh,et al.  The Sydney Triage to Admission Risk Tool (START): A prospective validation study , 2018, Emergency medicine Australasia : EMA.

[25]  Michel J. Anzanello,et al.  Text mining approach to predict hospital admissions using early medical records from the emergency department , 2017, Int. J. Medical Informatics.

[26]  Quoc V. Le,et al.  Distributed Representations of Sentences and Documents , 2014, ICML.

[27]  Ben Y. Reis,et al.  Early Prediction Model of Patient Hospitalization From the Pediatric Emergency Department , 2017, Pediatrics.