Delft University of Technology Efficient Real-Time Camera Based Estimation of Heart Rate and Its Variability

Remote photo-plethysmography (rPPG) uses a remotely placed camera to estimating a person’s heart rate (HR). Similar to how heart rate can provide useful information about a person’s vital signs, insights about the underlying physio/psychological conditions can be obtained from heart rate variability (HRV). HRV is a measure of the fine fluctuations in the intervals between heart beats. However, this measure requires temporally locating heart beats with a high degree of precision. We introduce a refined and efficient real-time rPPG pipeline with novel filtering and motion suppression that not only estimates heart rate more accurately, but also extracts the pulse waveform to time heart beats and measure heart rate variability. This method requires no rPPG specific training and is able to operate in real-time. We validate our method on a self-recorded dataset under an idealized lab setting, and show state-of-the-art results on two public dataset with realistic conditions (VicarPPG and PURE).

[1]  Yoshua Bengio,et al.  Convolutional networks for images, speech, and time series , 1998 .

[2]  Xiaoou Tang,et al.  Facial Landmark Detection by Deep Multi-task Learning , 2014, ECCV.

[3]  Matti Pietikäinen,et al.  Remote Heart Rate Measurement from Face Videos under Realistic Situations , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[4]  Abdulmotaleb El Saddik,et al.  Heart Rate Variability Extraction From Videos Signals: ICA vs. EVM Comparison , 2017, IEEE Access.

[5]  Frédo Durand,et al.  Detecting Pulse from Head Motions in Video , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[6]  F. Shaffer,et al.  An Overview of Heart Rate Variability Metrics and Norms , 2017, Front. Public Health.

[7]  Raymond Chiong,et al.  Remote heart rate measurement using low-cost RGB face video: a technical literature review , 2018, Frontiers of Computer Science.

[8]  Jiri Matas,et al.  Visual Heart Rate Estimation with Convolutional Neural Network , 2018, BMVC.

[9]  Daniel McDuff,et al.  Remote Detection of Photoplethysmographic Systolic and Diastolic Peaks Using a Digital Camera , 2014, IEEE Transactions on Biomedical Engineering.

[10]  Marco Wiering,et al.  A Model Based Method for Automatic Facial Expression Recognition , 2005, ECML.

[11]  Thomas B. Moeslund,et al.  Heartbeat Rate Measurement from Facial Video , 2016, IEEE Intelligent Systems.

[12]  Mohammad Soleymani,et al.  A Multimodal Database for Affect Recognition and Implicit Tagging , 2012, IEEE Transactions on Affective Computing.

[13]  David Fofi,et al.  Heart rate estimation using facial video: A review , 2017, Biomed. Signal Process. Control..

[14]  L. O. Svaasand,et al.  Remote plethysmographic imaging using ambient light. , 2008, Optics express.

[15]  Daniel McDuff,et al.  DeepPhys: Video-Based Physiological Measurement Using Convolutional Attention Networks , 2018, ECCV.

[16]  Paul A. Viola,et al.  Rapid object detection using a boosted cascade of simple features , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[17]  Rosalind W. Picard,et al.  Non-contact, automated cardiac pulse measurements using video imaging and blind source separation , 2022 .

[18]  Daniel McDuff,et al.  Advancements in Noncontact, Multiparameter Physiological Measurements Using a Webcam , 2011, IEEE Transactions on Biomedical Engineering.

[19]  J. Ramos-Castro,et al.  Video pulse rate variability analysis in stationary and motion conditions , 2018, BioMedical Engineering OnLine.

[20]  Horst-Michael Groß,et al.  Non-contact video-based pulse rate measurement on a mobile service robot , 2014, The 23rd IEEE International Symposium on Robot and Human Interactive Communication.

[21]  H. Emrah Tasli,et al.  Remote PPG based vital sign measurement using adaptive facial regions , 2014, 2014 IEEE International Conference on Image Processing (ICIP).

[22]  Roy Kalawsky,et al.  Noncontact imaging photoplethysmography to effectively access pulse rate variability , 2012, Journal of biomedical optics.

[23]  Cigdem Eroglu Erdem,et al.  Remote Photoplethysmography Using Nonlinear Mode Decomposition , 2018, 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[24]  D. Haan,et al.  Improved motion robustness of remote-PPG by using the blood volume pulse signature , 2014 .

[25]  Daniel McDuff,et al.  Improvements in Remote Cardiopulmonary Measurement Using a Five Band Digital Camera , 2014, IEEE Transactions on Biomedical Engineering.

[26]  I. Starr,et al.  STUDIES ON THE ESTIMATION OF CARDIAC OUPTUT IN MAN, AND OF ABNORMALITIES IN CARDIAC FUNCTION, FROM THE HEART'S RECOIL AND THE BLOOD'S IMPACTS; THE BALLISTOCARDIOGRAM , 1939 .

[27]  Nicu Sebe,et al.  Self-Adaptive Matrix Completion for Heart Rate Estimation from Face Videos under Realistic Conditions , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[28]  Sander Stuijk,et al.  Algorithmic Principles of Remote PPG , 2017, IEEE Transactions on Biomedical Engineering.

[29]  Silvia L. Pintea,et al.  Video Acceleration Magnification , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[30]  Yu Sun,et al.  Photoplethysmography Revisited: From Contact to Noncontact, From Point to Imaging , 2016, IEEE Transactions on Biomedical Engineering.

[31]  Westgate Road,et al.  Photoplethysmography and its application in clinical physiological measurement , 2007 .

[32]  Frédo Durand,et al.  Eulerian video magnification for revealing subtle changes in the world , 2012, ACM Trans. Graph..

[33]  Mika P. Tarvainen,et al.  An advanced detrending method with application to HRV analysis , 2002, IEEE Transactions on Biomedical Engineering.

[34]  Lan-Rong Dung,et al.  Measurement of heart rate variability using off-the-shelf smart phones , 2016, BioMedical Engineering OnLine.

[35]  Gerard de Haan,et al.  Robust Pulse Rate From Chrominance-Based rPPG , 2013, IEEE Transactions on Biomedical Engineering.