Creating a database for health IT events via a hybrid deep learning model

OBJECTIVE The use of poorly designed and improperly implemented health information technology (HIT) may compound risks because it can disrupt established work patterns and encourage workarounds. Analyzing and learning from HIT events could reduce the risks and improve safety but are limited by accessible HIT event reports. In this study, we propose a hybrid deep learning model to identify HIT event reports from the FDA resource and thus establish the first publicly accessible database for HIT event reports. MATERIALS AND METHODS 6,994 samples (3,521 HIT and 3,473 non-HIT events) extracted from the FDA MAUDE database were employed to assess nine individual and 120 hybrid models on the task of HIT identification. The optimal model was evaluated on an independent dataset prior to its application for establishing the HIT event database. RESULTS The hybrid model consisting of logistic regression, CNN, and Hierarchical RNN (ACC=0.903, AUC=0.954, F1 score=0.876) is superior to all the other models. The causes of errors include lack of root cause (72.3%), short descriptions (19.7%), and model undertrained (8.0%). The accuracy of the hybrid model on an independent dataset is reported as 0.862. We applied the optimal model to the entire MAUDE database (1991-2018) and generated an HIT event database with 48,997 reports. CONCLUSION The first HIT event database contains 48,997 reports with an annual growth rate of 10% (∼5,000 reports). The strategy of HIT event identification and establishment of the database could help healthcare professionals describe, understand, integrate the events and propose solutions in the context of a fuller spectrum of HIT events.