Jurisdictional Usage of the New ESSENCE Word Alert Feature

Objective The objective of this presentation is to describe the new word alert capability in ESSENCE and how it has been used by the Florida Department of Health (FDOH). Specifically, this presentation will describe how the word alert feature works to find individual chief complaint terms that are occurring at an abnormal rate. It will then provide usage statistics and first-person accounts of how the alerts have impacted public health practice for the users. Finally, the presentation will offer future enhancement possibilities and a summary of the benefits and shortcomings of this new feature. Introduction Syndromic surveillance systems have historically focused on aggregating data into syndromes for analysis and visualization. These syndromes provide users a way to quickly filter large amounts of data into a manageable number of streams to analyze. Additionally, ESSENCE users have the ability to build their own case definitions to look for records matching particular sets of criteria. Those user- defined queries can be stored and analyzed automatically, along with the pre-defined syndromes. Aside from these predefined and user- defined syndromic categories, ESSENCE did not previously provide alerts based on individual words in the chief complaint text that had not been specified a priori. Thus, an interesting cluster of records linked only by non-syndromic keywords would likely not be brought to a user’s attention. Methods In the FDOH ESSENCE system a new detection feature was developed to trigger alerts based on anomalous occurrence of terms in chief complaints. 1 This feature used Fisher’s Exact Test to test frequencies of individual chief complaint terms relative to all terms in a 1-month baseline. The feature used a 7-day guard-band, and automatically switched to an efficient chi-square test for sufficiently large term counts. A term triggered an alert if its p-value ≤ 10E-4. This algorithm was then run on chief complaint sets both by hospital and by region, with region assignment according to patient zip code. Results were then displayed in new visualizations showing alerts in word cloud and line listing form. Additionally, users were given the option to ignore stop words, syndromic terms, and a user-created list of ignorable words in order to focus on words of greater interest. Results The result of using the tool since June 2016 has seen three major benefits. First, the original intent for the system to notify users of abnormal word clusters has proven useful. Users have been able to see terms such as Disaster, Shelter and Fireworks which were not part of any prior syndromes and use these notifications to investigate possible issues. The second benefit found by users was the ability to find new misspellings or abbreviations commonly used by hospitals. The terms Zyka and GLF (Ground Level Fall) are examples of these. Finally, the system has helped discover new trends in hospital processes. For example, the tool has helped discover first person and non-English phrases in the chief complaint. This observation led to the discovery that some hospitals are using kiosks or mobile phone apps to allow patients to enter their own chief complaints. Conclusions The word alert feature has provided value to the users of FDOH ESSENCE. While accomplishing its initial goal of triggering abnormal non-syndromic term usage, the additional ability to find new misspellings and abbreviations may have even larger impact by keeping syndrome and subsyndrome definitions up-to-date over time for traditional syndromic alerting. Beyond these current benefits, additional visualization enhancements are under consideration. Additionally, the resources required to perform the detection are substantial, and implementation improvements are under development to improve the performance and enable more advanced free-text anomaly detection.