Optimal color channel combination across skin tones for remote heart rate measurement in camera-based photoplethysmography

Abstract Objective The heart rate is an essential vital sign that can be measured remotely with camera-based photoplethysmography (cbPPG). Systems for cbPPG typically use cameras that deliver red, green, and blue (RGB) channels. The combination of these channels has been proven to increase signal-to-noise ratio (SNR) and heart rate measurement accuracy (ACC). However, many combinations remain untested, the comparison of proposed combinations on large datasets is insufficiently investigated, and the interplay with skin tone is rarely addressed. Methods Eight regions of interest and eight color spaces with a total of 25 color channels were compared in terms of ACC and SNR based on the Binghamton-Pittsburgh-RPI Multimodal Spontaneous Emotion Database (BP4D+). Additionally, two systematic grid searches were performed to evaluate ACC in the space of linear combinations of the RGB channels. Results Glabella and forehead regions of interest provided highest ACC (up to 74.1 %) and SNR (> -3 dB) with the hue channel H from HSV color space and the chrominance channel Q from NTSC color space. The grid searches revealed a global optimum of linear RGB combinations (ACC: 79.2 %). This optimum occurred for all skin tones, although ACC dropped for darker skin tones. Conclusion Through systematic grid searches we were able to identify the skin tone independent optimal linear RGB color combination for measuring heart rate with cbPPG. Our results proved on a large dataset that the identified optimum outperformed conventionally used color channels. Significance The presented findings provide useful evidence for future considerations of algorithmic approaches for cbPPG.

[1]  Iztok Fister,et al.  Computational Intelligence in Sports , 2018, Adaptation, Learning, and Optimization.

[2]  L. O. Svaasand,et al.  Remote plethysmographic imaging using ambient light. , 2008, Optics express.

[3]  Gerard de Haan,et al.  Robust Pulse Rate From Chrominance-Based rPPG , 2013, IEEE Transactions on Biomedical Engineering.

[4]  Steffen Leonhardt,et al.  A Broader Look: Camera-Based Vital Sign Estimation across the Spectrum , 2019, Yearbook of Medical Informatics.

[5]  Sander Stuijk,et al.  Color-Distortion Filtering for Remote Photoplethysmography , 2017, 2017 12th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2017).

[6]  H Malberg,et al.  Camera-based photoplethysmography in critical care patients. , 2016, Clinical hemorheology and microcirculation.

[7]  A. N. Bashkatov,et al.  Optical properties of human skin, subcutaneous and mucous tissues in the wavelength range from 400 to 2000 nm , 2005 .

[8]  K. Tamura,et al.  The Face – A Vascular Perspective , 2018 .

[9]  T. Fitzpatrick.  The validity and practicality of sun-reactive skin types I through VI. , 1988, Archives of dermatology.

[10]  Hagen Malberg,et al.  Cardiovascular assessment by imaging photoplethysmography – a review , 2018, Biomedizinische Technik. Biomedical engineering.

[11]  Zheng Li,et al.  On the benefits of alternative color spaces for noncontact heart rate measurements using standard red-green-blue cameras , 2015, Journal of biomedical optics.

[12]  Toshiaki Kondo,et al.  A comparative study of video signals for non-contact heart rate measurement , 2015, 2015 12th International Conference on Electrical Engineering/Electronics, Computer, Telecommunications and Information Technology (ECTI-CON).

[13]  Hagen Malberg,et al.  Skin Detection and Tracking for Camera-Based Photoplethysmography Using a Bayesian Classifier and Level Set Segmentation , 2017, Bildverarbeitung für die Medizin.

[14]  H. Malberg,et al.  More Reliable Remote Heart Rate Measurement by Signal Quality Indexes , 2020, 2020 Computing in Cardiology.

[15]  D. Haan,et al.  Improved motion robustness of remote-PPG by using the blood volume pulse signature , 2014 .

[16]  Yuting Yang,et al.  Motion robust remote photoplethysmography in CIELab color space , 2016, Journal of biomedical optics.

[17]  Frédéric Bousefsaf,et al.  Automatic Selection of Webcam Photoplethysmographic Pixels Based on Lightness Criteria , 2017 .

[18]  Matjaz Perc,et al.  Computational intelligence in sports: Challenges and opportunities within a new research domain , 2015, Appl. Math. Comput..

[19]  H. Malberg,et al.  Color Spaces and Regions of Interest in Camera Based Heart Rate Estimation , 2020, 2020 11th Conference of the European Study Group on Cardiovascular Oscillations (ESGCO).

[20]  John Baker,et al.  Telemedicine Technology: a Review of Services, Equipment, and Other Aspects , 2018, Current Allergy and Asthma Reports.

[21]  Sander Stuijk,et al.  New insights into the origin of remote PPG signals in visible light and infrared , 2018, Scientific Reports.

[22]  Ayoub Al-Hamadi,et al.  How the Region of Interest Impacts Contact Free Heart Rate Estimation Algorithms , 2018, 2018 25th IEEE International Conference on Image Processing (ICIP).

[23]  Sander Stuijk,et al.  Algorithmic Principles of Remote PPG , 2017, IEEE Transactions on Biomedical Engineering.

[24]  Shaun J. Canavan,et al.  Multimodal Spontaneous Emotion Corpus for Human Behavior Analysis , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[25]  Steffen Leonhardt,et al.  Ambient and Unobtrusive Cardiorespiratory Monitoring Techniques , 2015, IEEE Reviews in Biomedical Engineering.

[26]  M. Tarvainen,et al.  Heart rate variability and occupational stress—systematic review , 2018, Industrial health.

[27]  Yu Sun,et al.  Photoplethysmography Revisited: From Contact to Noncontact, From Point to Imaging , 2016, IEEE Transactions on Biomedical Engineering.