AutoTriage - An Open Source Edge Computing Raspberry Pi-based Clinical Screening System

With the recent COVID-19 pandemic, healthcare systems all over the world are struggling to manage the massive increase in emergency department (ED) visits. This has put an enormous demand on medical professionals. Increased wait times in the ED increases the risk of infection transmission. In this work we present an open-source, low cost, off-body system to assist in the automatic triage of patients in the ED based on widely available hardware. The system initially focuses on two symptoms of the infection - fever and cyanosis. The use of visible and far-infrared cameras allows for rapid assessment at a 1m distance, thus reducing the load on medical staff and lowering the risk of spreading the infection within hospitals. Its utility can be extended to a general clinical setting in non-emergency times as well to reduce wait time, channel the time and effort of healthcare professionals to more critical tasks and also prioritize severe cases. Our system consists of a Raspberry Pi 4, a Google Coral USB accelerator, a Raspberry Pi Camera v2 and a FLIR Lepton 3.5 Radiometry Long-Wave Infrared Camera with an associated IO module. Algorithms running in real-time detect the presence and body parts of individual(s) in view, and segments out the forehead and lip regions using PoseNet. The temperature of the forehead-eye area is estimated from the infrared camera image and cyanosis is assessed from the image of the lips in the visible spectrum. In our preliminary experiments, an accuracy of 97% was achieved for detecting fever and 77% for the detection of cyanosis, with a sensitivity of 91% and area under the receiver operating characteristic curve of 0.91. Although preliminary results are promising, we note that the entire system needs to be optimized before use and assessed for efficacy. The use of low-cost instrumentation will not produce temperature readings and identification of cyanosis that is acceptable in many situations. For this reason, we are releasing the full code stack and system design to allow others to rapidly iterate and improve the system. This may be of particular benefit in low-resource settings, and low-to-middle income countries in particular, which are just beginning to be affected by COVID-19.

[1]  N. van der Linden,et al.  Emergency department crowding affects triage processes. , 2016, International emergency nursing.

[2]  D. Treloar,et al.  Comparison of rectal, axillary, and tympanic membrane temperatures in infants and young children. , 1991, Annals of emergency medicine.

[3]  A. Derse,et al.  Another Look at the Persistent Moral Problem of Emergency Department Crowding. , 2019, Annals of emergency medicine.

[4]  Yuan-Hsiang Lin,et al.  A Thermal Camera Based Continuous Body Temperature Measurement System , 2019, 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW).

[5]  Gladimir V. G. Baranoski,et al.  On the detection of peripheral cyanosis in individuals with distinct levels of cutaneous pigmentation , 2017, 2017 39th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[6]  Hiromitsu Kobayashi,et al.  Effect of measurement duration on accuracy of pulse-counting , 2013, Ergonomics.

[7]  Stefanos Zafeiriou,et al.  Robust Discriminative Response Map Fitting with Constrained Local Models , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[8]  Dirk Abel,et al.  Simulation physiologischer Regelkreise mit der objektorientierten Modellbibliothek “HumanLib” , 2011, Autom..

[9]  James B. Mercer,et al.  Fever screening and infrared thermal imaging: concerns and guidelines , 2009 .

[10]  Lindsey Woodworth.  Swamped: Emergency Department Crowding and Patient Mortality. , 2019, Journal of health economics.

[11]  Lalit K. Mestha,et al.  Estimation of Respiratory Pattern From Video Using Selective Ensemble Aggregation , 2016, IEEE Transactions on Signal Processing.

[12]  Scott Levin,et al.  Accuracy and Reliability of Emergency Department Triage Using the Emergency Severity Index: An International Multicenter Assessment , 2018, Annals of emergency medicine.

[13]  P. Welch.  The use of fast Fourier transform for the estimation of power spectra: A method based on time averaging over short, modified periodograms , 1967 .

[14]  Timnit Gebru,et al.  Gender Shades: Intersectional Accuracy Disparities in Commercial Gender Classification , 2018, FAT.

[15]  Jonathan Tompson,et al.  PersonLab: Person Pose Estimation and Instance Segmentation with a Bottom-Up, Part-Based, Geometric Embedding Model , 2018, ECCV.

[16]  R. Ward.  The Merck Manuals , 2011 .

[17]  David Fofi,et al.  Heart rate estimation using facial video: A review , 2017, Biomed. Signal Process. Control..

[18]  Chunming Li,et al.  Distance Regularized Level Set Evolution and Its Application to Image Segmentation , 2010, IEEE Transactions on Image Processing.

[19]  Dominik Aronsky,et al.  An Integrated Computerized Triage System in the Emergency Department , 2008, AMIA.

[20]  Ali Farhadi,et al.  You Only Look Once: Unified, Real-Time Object Detection , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[21]  L. O. Svaasand,et al.  Remote plethysmographic imaging using ambient light. , 2008, Optics express.

[22]  S. Granick,et al.  Infrared assessment of human facial temperature in the presence and absence of common cosmetics , 2020, medRxiv.

[23]  Ciara O'Brien,et al.  A comparison of algorithms for estimation of a respiratory signal from the surface electrocardiogram , 2007, Comput. Biol. Medicine.

[24]  Rosalind W. Picard,et al.  Non-contact, automated cardiac pulse measurements using video imaging and blind source separation , 2022 .

[25]  Frédo Durand,et al.  Detecting Pulse from Head Motions in Video , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[26]  V. L. Clark,et al.  Clinical Methods: The History, Physical, and Laboratory Examinations , 1990 .

[27]  J. Karjalainen,et al.  Fever and cardiac rhythm. , 1986, Archives of internal medicine.

[28]  Y. Guan,et al.  Coronavirus as a possible cause of severe acute respiratory syndrome , 2003, The Lancet.

[29]  Most patients are triaged using the emergency severity index , 2017, Journal of Nuclear Cardiology.

[30]  Thomas B. Moeslund,et al.  Heartbeat Rate Measurement from Facial Video , 2016, IEEE Intelligent Systems.

[31]  Raimon Jané,et al.  Evidence towards Improved Estimation of Respiratory Muscle Effort from Diaphragm Mechanomyographic Signals with Cardiac Vibration Interference Using Sample Entropy with Fixed Tolerance Values , 2014, PloS one.

[32]  Guy A. Dumont,et al.  Estimation of Respiratory Rate From Photoplethysmographic Imaging Videos Compared to Pulse Oximetry , 2015, IEEE Journal of Biomedical and Health Informatics.

[33]  D. Rapoport,et al.  Non-Invasive detection of respiratory effort-related arousals (REras) by a nasal cannula/pressure transducer system. , 2000, Sleep.

[34]  Matti Pietikäinen,et al.  Remote Heart Rate Measurement from Face Videos under Realistic Situations , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[35]  Roman Goldenberg,et al.  Computer-aided simple triage (CAST) for coronary CT angiography (CCTA) , 2012, International Journal of Computer Assisted Radiology and Surgery.

[36]  Loe M. G. Feijs,et al.  Color Correction of Baby Images for Cyanosis Detection , 2018, MIUA.

[37]  Víctor González-Castro,et al.  Multidimensional Assessments of Abdominal Aortic Aneurysms by Magnetic Resonance Against Ultrasound Diameter Measurements , 2017, MIUA.

[38]  L. Kinsman,et al.  Emergency department crowding: A systematic review of causes, consequences and solutions , 2018, PloS one.

[39]  Tia Gao,et al.  A next generation electronic triage to aid mass casualty emergency medical response , 2006, 2006 International Conference of the IEEE Engineering in Medicine and Biology Society.

[40]  Olga Boric-Lubecke,et al.  Respiratory effort energy estimation using Doppler radar , 2012, 2012 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[41]  D. K. Arvind,et al.  Respiratory Rate and Flow Waveform Estimation from Tri-axial Accelerometer Data , 2010, 2010 International Conference on Body Sensor Networks.

[42]  Steffen Leonhardt,et al.  A Broader Look: Camera-Based Vital Sign Estimation across the Spectrum , 2019, Yearbook of Medical Informatics.

[43]  Jesse M Pines,et al.  International perspectives on emergency department crowding. , 2011, Academic emergency medicine : official journal of the Society for Academic Emergency Medicine.

[44]  Sander Stuijk,et al.  Robust respiration detection from remote photoplethysmography. , 2016, Biomedical optics express.

[45]  Usama Ijaz Bajwa,et al.  Detecting Fever in Polish Children by Infrared Thermography , 2008 .

[46]  Eddie Y K Ng,et al.  Analysis of IR thermal imager for mass blind fever screening. , 2004, Microvascular research.

[47]  Sarun Sumriddetchkajorn,et al.  Mobile-platform for automatic fever screening system based on infrared forehead temperature , 2017, 2017 Opto-Electronics and Communications Conference (OECC) and Photonics Global Conference (PGC).