Research and applications: N-gram support vector machines for scalable procedure and diagnosis classification, with applications to clinical free text data from the intensive care unit

BACKGROUND Existing risk adjustment models for intensive care unit (ICU) outcomes rely on manual abstraction of patient-level predictors from medical charts. Developing an automated method for abstracting these data from free text might reduce cost and data collection times. OBJECTIVE To develop a support vector machine (SVM) classifier capable of identifying a range of procedures and diagnoses in ICU clinical notes for use in risk adjustment. MATERIALS AND METHODS We selected notes from 2001-2008 for 4191 neonatal ICU (NICU) and 2198 adult ICU patients from the MIMIC-II database from the Beth Israel Deaconess Medical Center. Using these notes, we developed an implementation of the SVM classifier to identify procedures (mechanical ventilation and phototherapy in NICU notes) and diagnoses (jaundice in NICU and intracranial hemorrhage (ICH) in adult ICU). On the jaundice classification task, we also compared classifier performance using n-gram features to unigrams with application of a negation algorithm (NegEx). RESULTS Our classifier accurately identified mechanical ventilation (accuracy=0.982, F1=0.954) and phototherapy use (accuracy=0.940, F1=0.912), as well as jaundice (accuracy=0.898, F1=0.884) and ICH diagnoses (accuracy=0.938, F1=0.943). Including bigram features improved performance on the jaundice (accuracy=0.898 vs 0.865) and ICH (0.938 vs 0.927) tasks, and outperformed NegEx-derived unigram features (accuracy=0.898 vs 0.863) on the jaundice task. DISCUSSION Overall, a classifier using n-gram support vectors displayed excellent performance characteristics. The classifier generalizes to diverse patient populations, diagnoses, and procedures. CONCLUSIONS SVM-based classifiers can accurately identify procedure status and diagnoses among ICU patients, and including n-gram features improves performance, compared to existing methods.

[1]  S. Mayer,et al.  Clinical review: Critical care management of spontaneous intracerebral hemorrhage , 2008, Critical care.

[2]  Hinrich Schütze,et al.  Book Reviews: Foundations of Statistical Natural Language Processing , 1999, CL.

[3]  Eduard E Vasilevskis,et al.  Predictors of early postdischarge mortality in critically ill patients: a retrospective cohort study from the California Intensive Care Outcomes project. , 2011, Journal of critical care.

[4]  M. J. Maisels,et al.  What's in a Name? Physiologic and Pathologic Jaundice: The Conundrum of Defining Normal Bilirubin Levels in the Newborn , 2006, Pediatrics.

[5]  Eduard E Vasilevskis,et al.  Mortality probability model III and simplified acute physiology score II: assessing their value in predicting length of stay and comparison to APACHE IV. , 2009, Chest.

[6]  Gavin C. Cawley,et al.  On Over-fitting in Model Selection and Subsequent Selection Bias in Performance Evaluation , 2010, J. Mach. Learn. Res..

[7]  Cynthia Brandt,et al.  Semi-supervised clinical text classification with Laplacian SVMs: An application to cancer case management , 2013, J. Biomed. Informatics.

[8]  Mitzi L. Dean,et al.  Variation in ICU risk-adjusted mortality: impact of methods of assessment and potential confounders. , 2008, Chest.

[9]  Robert L. Mercer,et al.  Class-Based n-gram Models of Natural Language , 1992, CL.

[10]  Jeffrey M. Hausdorff,et al.  Physionet: Components of a New Research Resource for Complex Physiologic Signals". Circu-lation Vol , 2000 .

[11]  D. Hess,et al.  Approaches to Conventional Mechanical Ventilation of the Patient With Acute Respiratory Distress Syndrome , 2011, Respiratory Care.

[12]  William Rose,et al.  Practical implementation of an existing smoking detection pipeline and reduced support vector machine training corpus requirements , 2014, J. Am. Medical Informatics Assoc..

[13]  Gaël Varoquaux,et al.  Scikit-learn: Machine Learning in Python , 2011, J. Mach. Learn. Res..

[14]  T. H. Kyaw,et al.  Multiparameter Intelligent Monitoring in Intensive Care II: A public-access intensive care unit database* , 2011, Critical care medicine.

[15]  Peter Norvig,et al.  The Unreasonable Effectiveness of Data , 2009, IEEE Intelligent Systems.

[16]  Richard Simon,et al.  Bias in error estimation when using cross-validation for model selection , 2006, BMC Bioinformatics.

[17]  Michael McGill,et al.  Introduction to Modern Information Retrieval , 1983 .

[18]  N. Halpern,et al.  Critical care medicine in the United States 1985–2000: An analysis of bed numbers, use, and costs* , 2004, Critical care medicine.

[19]  Bruce F. Folck,et al.  Rehospitalization in the First Two Weeks After Discharge From the Neonatal Intensive Care Unit , 1999, Pediatrics.

[20]  Wendy W. Chapman,et al.  Document-level classification of CT pulmonary angiography reports based on an extension of the ConText algorithm , 2011, J. Biomed. Informatics.

[21]  Prakash M. Nadkarni,et al.  Overcoming barriers to NLP for clinical text: the role of shared tasks and the need for additional creative solutions , 2011, J. Am. Medical Informatics Assoc..

[22]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[23]  Adam Wright,et al.  Use of a support vector machine for categorizing free-text notes: assessment of accuracy across two institutions , 2013, J. Am. Medical Informatics Assoc..

[24]  Brian Wilson,et al.  Case Report: Identifying Smokers with a Medical Extraction System , 2008, J. Am. Medical Informatics Assoc..

[25]  Hanqing Cao,et al.  Heuristics to determine ventilation times of ICU patients from the MIMIC-II database , 2010, 2010 Annual International Conference of the IEEE Engineering in Medicine and Biology.

[26]  M. F. Porter,et al.  An algorithm for suffix stripping , 1997 .

[27]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[28]  Mitzi L. Dean,et al.  The effect of race and ethnicity on outcomes among patients in the intensive care unit: A comprehensive study involving socioeconomic status and resuscitation preferences* , 2011, Critical care medicine.

[29]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[30]  Wendy W. Chapman,et al.  A Simple Algorithm for Identifying Negated Findings and Diseases in Discharge Summaries , 2001, J. Biomed. Informatics.