Using Active Learning to Identify Health Information Technology Related Patient Safety Events

The widespread adoption of health information technology (HIT) has led to new patient safety hazards that are often difficult to identify. Patient safety event reports, which are self-reported descriptions of safety hazards, provide one view of potential HIT-related safety events. However, identifying HIT-related reports can be challenging as they are often categorized under other more predominate clinical categories. This challenge of identifying HIT-related reports is exacerbated by the increasing number and complexity of reports which pose challenges to human annotators that must manually review reports. In this paper, we apply active learning techniques to support classification of patient safety event reports as HIT-related. We evaluated different strategies and demonstrated a 30% increase in average precision of a confirmatory sampling strategy over a baseline no active learning approach after 10 learning iterations.

[1]  Henrik Boström,et al.  De-identifying health records by means of active learning , 2012, ICML 2012.

[2]  Dean F Sittig,et al.  Measuring and improving patient safety through health information technology: The Health IT Safety Framework , 2015, BMJ Quality & Safety.

[3]  Mark Craven,et al.  Active Learning with Real Annotation Costs , 2008 .

[4]  Ameet Talwalkar,et al.  MLlib: Machine Learning in Apache Spark , 2015, J. Mach. Learn. Res..

[5]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[6]  David D. Lewis,et al.  Heterogeneous Uncertainty Sampling for Supervised Learning , 1994, ICML.

[7]  Jaime G. Carbonell,et al.  Proactive learning: cost-sensitive active learning with multiple imperfect oracles , 2008, CIKM '08.

[8]  Farah Magrabi,et al.  Using statistical text classification to identify health information technology incidents , 2013, J. Am. Medical Informatics Assoc..

[9]  Robert L. Wears,et al.  Health information technology: fallacies and sober realities , 2010, J. Am. Medical Informatics Assoc..

[10]  Robyn Kozierok,et al.  Active Learning with a Human in The Loop , 2012 .

[11]  Anthony N. Nguyen,et al.  Active learning: a step towards automating medical concept extraction , 2015, J. Am. Medical Informatics Assoc..

[12]  R Ratwani,et al.  An Evaluation of Patient Safety Event Report Categories Using Unsupervised Topic Modeling. , 2015, Methods of information in medicine.

[13]  Hinrich Schütze,et al.  Introduction to information retrieval , 2008 .

[14]  Tim Kraska,et al.  MLbase: A Distributed Machine-learning System , 2013, CIDR.

[15]  Farah Magrabi,et al.  Automated categorisation of clinical incident reports using statistical text classification , 2010, Quality and Safety in Health Care.

[16]  Mary G. Amato,et al.  Computerized prescriber order entry–related patient safety reports: analysis of 2522 medication errors , 2017, J. Am. Medical Informatics Assoc..

[17]  Xiaoyan Zhu,et al.  A Unified Active Learning Framework for Biomedical Relation Extraction , 2012, Journal of Computer Science and Technology.

[18]  Farah Magrabi,et al.  Using FDA reports to inform a classification for health information technology safety problems , 2012, J. Am. Medical Informatics Assoc..

[19]  Burr Settles,et al.  Active Learning Literature Survey , 2009 .

[20]  A. Kushniruk,et al.  Towards an Integrative Cognitive-Socio-Technical Approach in Health Informatics: Analyzing Technology-Induced Error Involving Health Information Systems to Improve Patient Safety , 2010, The open medical informatics journal.

[21]  Burr Settles,et al.  Active Learning , 2012, Synthesis Lectures on Artificial Intelligence and Machine Learning.

[22]  Dean F Sittig,et al.  Exploring the sociotechnical intersection of patient safety and electronic health record implementation. , 2014, Journal of the American Medical Informatics Association : JAMIA.