Deep multitask ensemble classification of emergency medical call incidents combining multimodal data improves emergency medical dispatch

Objective: To develop a predictive model to aid non-clinical dispatchers to classify emergency medical call incidents by their life-threatening level (yes/no), admissible response delay (undelayable, minutes, hours, days) and emergency system jurisdiction (emergency system/primary care) in real time. Materials: A total of 1 244 624 independent retrospective incidents from the Valencian emergency medical dispatch service in Spain from 2009 to 2012, comprising clinical features, demographics, circumstantial factors and free text dispatcher observations. Methods: A deep multitask ensemble model integrating four subnetworks, composed in turn by multi-layer perceptron modules, bidirectional long short-term memory units and a bidirectional encoding representations from transformers module. Results: The model showed a F1 score of 0.771 in life-threatening classification, 0.592 in response delay and 0.801 in jurisdiction, obtaining a performance increase of 13.2%, 16.4% and 4.5%, respectively, with regard to the current in-house triage protocol of the Valencian emergency medical dispatch service. Discussion: The model captures information present in emergency medical calls not considered by the existing in-house triage protocol, but relevant to carry out incident classification. Besides, the results suggest that most of this information is present in the free text dispatcher observations. Conclusion: To our knowledge, this study presents the development of the first deep learning model undertaking emergency medical call incidents classification. Its adoption in medical dispatch centers would potentially improve emergency dispatch processes, resulting in a positive impact in patient wellbeing and health services sustainability.

[1]  A P Pearce,et al.  Emergency medical services at the crossroads , 2009, Emergency Medicine Journal.

[2]  Maria E. Mayorga,et al.  A model for optimally dispatching ambulances to emergency calls with classification errors in patient priorities , 2013 .

[3]  Kevin Mackway-Jones,et al.  Emergency triage. , 2013, Emergency nurse : the journal of the RCN Accident and Emergency Nursing Association.

[4]  Other Contributors Are Indicated Where They Contribute Python Software Foundation , 2017 .

[5]  Yoshua Bengio,et al.  Algorithms for Hyper-Parameter Optimization , 2011, NIPS.

[6]  Burr Settles,et al.  Active Learning Literature Survey , 2009 .

[7]  Juan Miguel García-Gómez,et al.  Kinematics of Big Biomedical Data to characterize temporal variability and seasonality of data repositories: Functional Data Analysis of data temporal evolution over non-parametric statistical manifolds , 2018, Int. J. Medical Informatics.

[8]  A. Ng Feature selection, L1 vs. L2 regularization, and rotational invariance , 2004, Twenty-first international conference on Machine learning - ICML '04.

[9]  Wes McKinney,et al.  Data Structures for Statistical Computing in Python , 2010, SciPy.

[10]  Paul J. Werbos,et al.  Backpropagation Through Time: What It Does and How to Do It , 1990, Proc. IEEE.

[11]  Pavel Pudil,et al.  Introduction to Statistical Pattern Recognition , 2006 .

[12]  Gregorio Ismael Sainz Palmero,et al.  Interpretable knowledge extraction from emergency call data based on fuzzy unsupervised decision tree , 2012, Knowl. Based Syst..

[13]  R. Cydulka,et al.  Performance of a system to determine EMS dispatch priorities. , 1996, The American journal of emergency medicine.

[14]  Chris Eliasmith,et al.  Hyperopt: a Python library for model selection and hyperparameter optimization , 2015 .

[15]  Juan Miguel García-Gómez,et al.  Probabilistic change detection and visualization methods for the assessment of temporal stability in biomedical data quality , 2015, Data Mining and Knowledge Discovery.

[16]  Michael J. Fischer,et al.  The String-to-String Correction Problem , 1974, JACM.

[17]  Albert Y. Chen,et al.  A GIS-based Demand Forecast using Machine Learning for Emergency Medical Services , 2014 .

[18]  Sumithra Velupillai,et al.  Enhancing predictions of patient conveyance using emergency call handler free text notes for unconscious and fainting incidents reported to the London Ambulance Service , 2020, Int. J. Medical Informatics.

[19]  Sebastian Ruder,et al.  An Overview of Multi-Task Learning in Deep Neural Networks , 2017, ArXiv.

[20]  Plamen Angelov Handbook in computational intelligence , 2016 .

[21]  Yiming Yang,et al.  A re-examination of text categorization methods , 1999, SIGIR '99.

[22]  Fredrik Folke,et al.  Machine learning as a supportive tool to recognize cardiac arrest in emergency calls. , 2019, Resuscitation.

[23]  Anders Krogh,et al.  A Simple Weight Decay Can Improve Generalization , 1991, NIPS.

[24]  Jeff Clawson,et al.  Principles of Emergency Medical Dispatch , 1987 .

[25]  Norman Crolee Dalkey,et al.  An experimental study of group opinion , 1969 .

[26]  Grigorios Tsoumakas,et al.  Mining Multi-label Data , 2010, Data Mining and Knowledge Discovery Handbook.

[27]  Carlos Sáez,et al.  Applying probabilistic temporal and multisite data quality control methods to a public health mortality registry in Spain: a systematic approach to quality control of repositories , 2016, J. Am. Medical Informatics Assoc..

[28]  Michael S. Bernstein,et al.  ImageNet Large Scale Visual Recognition Challenge , 2014, International Journal of Computer Vision.

[29]  E. Steyerberg,et al.  Undertriage in the Manchester triage system: an assessment of severity and options for improvement , 2011, Archives of Disease in Childhood.

[30]  Shiliang Sun,et al.  A Survey of Optimization Methods From a Machine Learning Perspective , 2019, IEEE Transactions on Cybernetics.

[31]  Yoshua Bengio,et al.  Understanding the difficulty of training deep feedforward neural networks , 2010, AISTATS.

[32]  Ron Kohavi,et al.  A Study of Cross-Validation and Bootstrap for Accuracy Estimation and Model Selection , 1995, IJCAI.

[33]  Luca Antiga,et al.  Automatic differentiation in PyTorch , 2017 .

[34]  Rich Caruana,et al.  Multitask Learning , 1998, Encyclopedia of Machine Learning and Data Mining.

[35]  Nitish Srivastava,et al.  Improving neural networks by preventing co-adaptation of feature detectors , 2012, ArXiv.

[36]  R. Koster,et al.  Telephone triage of cardiac emergency calls by dispatchers: a prospective study of 1386 emergency calls. , 1994, British heart journal.

[37]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[38]  Gerard FitzGerald,et al.  Emergency department triage revisited , 2009, Emergency Medicine Journal.

[39]  Henriëtte A. Moll,et al.  Validity of the Manchester Triage System in emergency care: A prospective observational study , 2017, PloS one.

[40]  Matthew S. Maxwell,et al.  Ambulance redeployment: An approximate dynamic programming approach , 2009, Proceedings of the 2009 Winter Simulation Conference (WSC).

[41]  J. Herlitz,et al.  Why are people without medical needs transported by ambulance? A study of indications for pre-hospital care , 2007, European journal of emergency medicine : official journal of the European Society for Emergency Medicine.

[42]  D. Nutzinger,et al.  International classification of diseases (ICD), 9th revision , 1992 .

[43]  C. Saez,et al.  EHRtemporalVariability: delineating temporal data-set shifts in electronic health records , 2020, medRxiv.

[44]  S. Stratton Triage By Emergency Medical Dispatchers , 1992, Prehospital and Disaster Medicine.

[45]  Gaël Varoquaux,et al.  The NumPy Array: A Structure for Efficient Numerical Computation , 2011, Computing in Science & Engineering.

[46]  Robert Hecht-Nielsen,et al.  Theory of the backpropagation neural network , 1989, International 1989 Joint Conference on Neural Networks.

[47]  Václav Snásel,et al.  Using SOM in the performance monitoring of the emergency call-taking system , 2011, Simul. Model. Pract. Theory.

[48]  J Leprohon,et al.  Decision-making Strategies for Telephone Triage in Emergency Medical Services , 1995, Medical decision making : an international journal of the Society for Medical Decision Making.

[49]  Dimitri P. Bertsekas,et al.  Incremental Least Squares Methods and the Extended Kalman Filter , 1996, SIAM J. Optim..

[50]  Andrés Montoyo,et al.  Advances on natural language processing , 2007, Data Knowl. Eng..

[51]  A. Kihlgren,et al.  Operators' experiences of emergency calls , 2004, Journal of telemedicine and telecare.

[52]  L. Weibel,et al.  Work-related stress in an emergency medical dispatch center. , 2003, Annals of emergency medicine.

[53]  Carlos Sáez,et al.  Guest editorial: Special issue in biomedical data quality assessment methods , 2019, Comput. Methods Programs Biomed..

[54]  Juan Miguel García-Gómez,et al.  Randomized pilot study and qualitative evaluation of a clinical decision support system for brain tumour diagnosis based on SV 1H MRS: Evaluation as an additional information procedure for novice radiologists , 2014, Comput. Biol. Medicine.

[55]  Armann Ingolfsson,et al.  The application of forecasting techniques to modeling emergency medical system calls in Calgary, Alberta , 2007, Health care management science.

[56]  M. Svedlund,et al.  Reliability of a Swedish pre-hospital dispatch system in prioritizing patients. , 2013, International emergency nursing.

[57]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[58]  Dirk T Ubbink,et al.  Comparison of an informally structured triage system, the emergency severity index, and the manchester triage system to distinguish patient priority in the emergency department. , 2011, Academic emergency medicine : official journal of the Society for Academic Emergency Medicine.

[59]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[60]  Jian Sun,et al.  Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[61]  Ming-Wei Chang,et al.  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding , 2019, NAACL.

[62]  Yoshua Bengio,et al.  A Neural Probabilistic Language Model , 2003, J. Mach. Learn. Res..

[63]  Gaël Varoquaux,et al.  Scikit-learn: Machine Learning in Python , 2011, J. Mach. Learn. Res..

[64]  Carlos Sáez,et al.  Stability metrics for multi-source biomedical data based on simplicial projections from probability distribution distances , 2017, Statistical methods in medical research.

[65]  Kuldip K. Paliwal,et al.  Bidirectional recurrent neural networks , 1997, IEEE Trans. Signal Process..

[66]  Michael Christ,et al.  Modern triage in the emergency department. , 2010, Deutsches Arzteblatt international.

[67]  Jorge Nocedal,et al.  On the limited memory BFGS method for large scale optimization , 1989, Math. Program..

[68]  Demis Hassabis,et al.  Mastering the game of Go with deep neural networks and tree search , 2016, Nature.

[69]  P. McCullagh,et al.  Generalized Linear Models , 1972, Predictive Analytics.

[70]  Frank Rosenblatt,et al.  PRINCIPLES OF NEURODYNAMICS. PERCEPTRONS AND THE THEORY OF BRAIN MECHANISMS , 1963 .

[71]  Tara N. Sainath,et al.  Deep Neural Networks for Acoustic Modeling in Speech Recognition: The Shared Views of Four Research Groups , 2012, IEEE Signal Processing Magazine.

[72]  Sebastian Ruder,et al.  An overview of gradient descent optimization algorithms , 2016, Vestnik komp'iuternykh i informatsionnykh tekhnologii.

[73]  Wojciech Czarnecki,et al.  On Loss Functions for Deep Neural Networks in Classification , 2017, ArXiv.

[74]  Derek C Angus,et al.  Randomized Clinical Trials of Artificial Intelligence. , 2020, JAMA.

[75]  Léon J. M. Rothkrantz,et al.  Automatic stress detection in emergency (telephone) calls , 2011, Int. J. Intell. Def. Support Syst..

[76]  Robert Hecht-Nielsen III.3 – Theory of the Backpropagation Neural Network* , 1992 .

[77]  R'emi Louf,et al.  HuggingFace's Transformers: State-of-the-art Natural Language Processing , 2019, ArXiv.

[78]  M. Bullard,et al.  Revisions to the Canadian Emergency Department Triage and Acuity Scale (CTAS) Guidelines 2016. , 2008, CJEM.

[79]  Andrew L. Maas Rectifier Nonlinearities Improve Neural Network Acoustic Models , 2013 .

[80]  Neil D. Lawrence,et al.  Dataset Shift in Machine Learning , 2009 .

[81]  V. Novák,et al.  Mathematical Principles of Fuzzy Logic , 1999 .

[82]  Ann Blandford,et al.  Situation awareness in emergency medical dispatch , 2004, Int. J. Hum. Comput. Stud..

[83]  Kilian Stoffel,et al.  Theoretical Comparison between the Gini Index and Information Gain Criteria , 2004, Annals of Mathematics and Artificial Intelligence.

[84]  M. Bullard,et al.  Revisions to the Canadian Emergency Department Triage and Acuity Scale (CTAS) Guidelines. , 2014, CJEM.

[85]  Ke Chen,et al.  Deep and Modular Neural Networks , 2015, Handbook of Computational Intelligence.

[86]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[87]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[88]  J Leprohon,et al.  The role of protocols and professional judgement in emergency medical dispatching. , 1995, European journal of emergency medicine : official journal of the European Society for Emergency Medicine.

[89]  Carlos Sáez,et al.  EHRtemporalVariability: delineating temporal data-set shifts in electronic health records , 2020, GigaScience.

[90]  Donald R. Jones,et al.  A Taxonomy of Global Optimization Methods Based on Response Surfaces , 2001, J. Glob. Optim..