Efficient Real-Time Camera Based Estimation of Heart Rate and Its Variability

Remote photo-plethysmography (rPPG) uses a remotely placed camera to estimating a person's heart rate (HR). Similar to how heart rate can provide useful information about a person's vital signs, insights about the underlying physio/psychological conditions can be obtained from heart rate variability (HRV). HRV is a measure of the fine fluctuations in the intervals between heart beats. However, this measure requires temporally locating heart beats with a high degree of precision. We introduce a refined and efficient real-time rPPG pipeline with novel filtering and motion suppression that not only estimates heart rate more accurately, but also extracts the pulse waveform to time heart beats and measure heart rate variability. This method requires no rPPG specific training and is able to operate in real-time. We validate our method on a self-recorded dataset under an idealized lab setting, and show state-of-the-art results on two public dataset with realistic conditions (VicarPPG and PURE).

[1]  Westgate Road,et al.  Photoplethysmography and its application in clinical physiological measurement , 2007 .

[2]  Mohammad Soleymani,et al.  A Multimodal Database for Affect Recognition and Implicit Tagging , 2012, IEEE Transactions on Affective Computing.

[3]  Thomas B. Moeslund,et al.  Heartbeat Rate Measurement from Facial Video , 2016, IEEE Intelligent Systems.

[4]  Daniel McDuff,et al.  Improvements in Remote Cardiopulmonary Measurement Using a Five Band Digital Camera , 2014, IEEE Transactions on Biomedical Engineering.

[5]  Samy Bengio,et al.  The Handbook of Brain Theory and Neural Networks , 2002 .

[6]  Frédo Durand,et al.  Eulerian video magnification for revealing subtle changes in the world , 2012, ACM Trans. Graph..

[7]  Xiaoou Tang,et al.  Facial Landmark Detection by Deep Multi-task Learning , 2014, ECCV.

[8]  Abdulmotaleb El Saddik,et al.  Heart Rate Variability Extraction From Videos Signals: ICA vs. EVM Comparison , 2017, IEEE Access.

[9]  Marco Wiering,et al.  A Model Based Method for Automatic Facial Expression Recognition , 2005, ECML.

[10]  Roy Kalawsky,et al.  Noncontact imaging photoplethysmography to effectively access pulse rate variability , 2012, Journal of biomedical optics.

[11]  Rosalind W. Picard,et al.  Non-contact, automated cardiac pulse measurements using video imaging and blind source separation , 2022 .

[12]  Jiri Matas,et al.  Visual Heart Rate Estimation with Convolutional Neural Network , 2018, BMVC.

[13]  Frédo Durand,et al.  Detecting Pulse from Head Motions in Video , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[14]  Yu Sun,et al.  Photoplethysmography Revisited: From Contact to Noncontact, From Point to Imaging , 2016, IEEE Transactions on Biomedical Engineering.

[15]  F. Shaffer,et al.  An Overview of Heart Rate Variability Metrics and Norms , 2017, Front. Public Health.

[16]  Yoshua Bengio,et al.  Convolutional networks for images, speech, and time series , 1998 .

[17]  L. O. Svaasand,et al.  Remote plethysmographic imaging using ambient light. , 2008, Optics express.

[18]  Raymond Chiong,et al.  Remote heart rate measurement using low-cost RGB face video: a technical literature review , 2018, Frontiers of Computer Science.

[19]  Lan-Rong Dung,et al.  Measurement of heart rate variability using off-the-shelf smart phones , 2016, BioMedical Engineering OnLine.

[20]  Mika P. Tarvainen,et al.  An advanced detrending method with application to HRV analysis , 2002, IEEE Transactions on Biomedical Engineering.

[21]  D. Haan,et al.  Improved motion robustness of remote-PPG by using the blood volume pulse signature , 2014 .

[22]  Nicu Sebe,et al.  Self-Adaptive Matrix Completion for Heart Rate Estimation from Face Videos under Realistic Conditions , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[23]  Matti Pietikäinen,et al.  Remote Heart Rate Measurement from Face Videos under Realistic Situations , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[24]  H. Emrah Tasli,et al.  Remote PPG based vital sign measurement using adaptive facial regions , 2014, 2014 IEEE International Conference on Image Processing (ICIP).

[25]  Horst-Michael Groß,et al.  Non-contact video-based pulse rate measurement on a mobile service robot , 2014, The 23rd IEEE International Symposium on Robot and Human Interactive Communication.

[26]  I. Starr,et al.  STUDIES ON THE ESTIMATION OF CARDIAC OUPTUT IN MAN, AND OF ABNORMALITIES IN CARDIAC FUNCTION, FROM THE HEART'S RECOIL AND THE BLOOD'S IMPACTS; THE BALLISTOCARDIOGRAM , 1939 .

[27]  Gerard de Haan,et al.  Robust Pulse Rate From Chrominance-Based rPPG , 2013, IEEE Transactions on Biomedical Engineering.

[28]  J. Ramos-Castro,et al.  Video pulse rate variability analysis in stationary and motion conditions , 2018, BioMedical Engineering OnLine.

[29]  Cigdem Eroglu Erdem,et al.  Remote Photoplethysmography Using Nonlinear Mode Decomposition , 2018, 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[30]  David Fofi,et al.  Heart rate estimation using facial video: A review , 2017, Biomed. Signal Process. Control..

[31]  Daniel McDuff,et al.  Advancements in Noncontact, Multiparameter Physiological Measurements Using a Webcam , 2011, IEEE Transactions on Biomedical Engineering.

[32]  A. Malliani,et al.  Heart rate variability. Standards of measurement, physiological interpretation, and clinical use , 1996 .

[33]  Silvia L. Pintea,et al.  Video Acceleration Magnification , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[34]  Daniel McDuff,et al.  Remote Detection of Photoplethysmographic Systolic and Diastolic Peaks Using a Digital Camera , 2014, IEEE Transactions on Biomedical Engineering.

[35]  Daniel McDuff,et al.  DeepPhys: Video-Based Physiological Measurement Using Convolutional Attention Networks , 2018, ECCV.

[36]  Sander Stuijk,et al.  Algorithmic Principles of Remote PPG , 2017, IEEE Transactions on Biomedical Engineering.

[37]  Paul A. Viola,et al.  Rapid object detection using a boosted cascade of simple features , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.