Contactless Measurement of Vital Signs Using Thermal and RGB Cameras: A Study of COVID 19-Related Health Monitoring

In this study, a contactless vital signs monitoring system was proposed, which can measure body temperature (BT), heart rate (HR) and respiration rate (RR) for people with and without face masks using a thermal and an RGB camera. The convolution neural network (CNN) based face detector was applied and three regions of interest (ROIs) were located based on facial landmarks for vital sign estimation. Ten healthy subjects from a variety of ethnic backgrounds with skin colors from pale white to darker brown participated in several different experiments. The absolute error (AE) between the estimated HR using the proposed method and the reference HR from all experiments is 2.70±2.28 beats/min (mean ± std), and the AE between the estimated RR and the reference RR from all experiments is 1.47±1.33 breaths/min (mean ± std) at a distance of 0.6–1.2 m.

[1]  Guangtao Zhai,et al.  Detection of Respiratory Infections Using RGB-Infrared Sensors on Portable Device , 2020, IEEE Sensors Journal.

[2]  Irene Kotsia,et al.  RetinaFace: Single-Shot Multi-Level Face Localisation in the Wild , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[3]  Walter Karlen,et al.  Multispectral Video Fusion for Non-Contact Monitoring of Respiratory Rate and Apnea , 2020, IEEE Transactions on Biomedical Engineering.

[4]  K. Kim,et al.  What Is COVID-19? , 2020, Frontiers for Young Minds.

[5]  R. Saatchi,et al.  P016 Using non-invasive thermal imaging for apnoea detection , 2019, Poster Presentations.

[6]  Philipp V. Rouast,et al.  Remote heart rate measurement using low-cost RGB face video: a technical literature review , 2018, Frontiers of Computer Science.

[7]  K. Mutlu,et al.  IR thermography-based monitoring of respiration phase without image segmentation , 2018, Journal of Neuroscience Methods.

[8]  Xu Tang,et al.  PyramidBox: A Context-assisted Single Shot Face Detector , 2018, ECCV.

[9]  Shifeng Zhang,et al.  S^3FD: Single Shot Scale-Invariant Face Detector , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[10]  H. van Goor,et al.  Continuous Monitoring of Vital Signs Using Wearable Devices on the General Ward: Pilot Study , 2017, JMIR mHealth and uHealth.

[11]  Nicolai Marquardt,et al.  Robust tracking of respiratory rate in high-dynamic range scenes using mobile thermal imaging. , 2017, Biomedical optics express.

[12]  Ki H. Chon,et al.  Tidal Volume and Instantaneous Respiration Rate Estimation using a Volumetric Surrogate Signal Acquired via a Smartphone Camera , 2017, IEEE Journal of Biomedical and Health Informatics.

[13]  Yu Qiao,et al.  Joint Face Detection and Alignment Using Multitask Cascaded Convolutional Networks , 2016, IEEE Signal Processing Letters.

[14]  Shuo Yang,et al.  WIDER FACE: A Face Detection Benchmark , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[15]  Lai-Man Po,et al.  Motion artifacts suppression for remote imaging photoplethysmography , 2014, 2014 19th International Conference on Digital Signal Processing.

[16]  Josephine Sullivan,et al.  One millisecond face alignment with an ensemble of regression trees , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[17]  Matti Pietikäinen,et al.  Remote Heart Rate Measurement from Face Videos under Realistic Situations , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[18]  Yuting Yang,et al.  Noncontact Monitoring Breathing Pattern, Exhalation Flow Rate and Pulse Transit Time , 2014, IEEE Transactions on Biomedical Engineering.

[19]  Winston H. Hsu,et al.  Learning-based heart rate detection from remote photoplethysmography features , 2014, 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[20]  Peter J Lesniewski,et al.  Signal recovery in imaging photoplethysmography , 2013, Physiological measurement.

[21]  J. Tsien,et al.  Remote Measurements of Heart and Respiration Rates for Telemedicine , 2013, PloS one.

[22]  Stefanos Zafeiriou,et al.  Robust Discriminative Response Map Fitting with Constrained Local Models , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[23]  Kwang Suk Park,et al.  Validation of heart rate extraction using video imaging on a built-in camera system of a smartphone , 2012, 2012 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[24]  Tomasz Kocejko,et al.  Proceedings of the Federated Conference on Computer Science and Information Systems pp. 405–410 ISBN 978-83-60810-22-4 Measuring Pulse Rate with a Webcam – a Non-contact Method for Evaluating Cardiac Activity , 2022 .

[25]  Rosalind W. Picard,et al.  Non-contact, automated cardiac pulse measurements using video imaging and blind source separation , 2022 .

[26]  L. O. Svaasand,et al.  Remote plethysmographic imaging using ambient light. , 2008, Optics express.

[27]  Ali M. Reza,et al.  Realization of the Contrast Limited Adaptive Histogram Equalization (CLAHE) for Real-Time Image Enhancement , 2004, J. VLSI Signal Process..

[28]  Paul A. Viola,et al.  Rapid object detection using a boosted cascade of simple features , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[29]  C. E..  WHO Coronavirus Disease (COVID-19) Dashboard , 2020 .

[30]  Jean-Franois Cardoso.  High-Order Contrasts for Independent Component Analysis , 1999, Neural Computation.