An Accurate LSTM Based Video Heart Rate Estimation Method

Pulse signal is an effective indicator to reflect the physiological and physical state of the human body. There are many heart rate estimation methods in videos and most of them manually design algorithm to modeling noise signal, which is not enough to represent the actual distribution of noise. In this paper, we propose to train a two-layer LSTM to estimate pulse signals because long short-term memory (LSTM) can preserve useful signals by filtering out noise signals upon data-driven. In order to overcome the problem of insufficient heart rate public database, we propose to use quantities of synthetic signals which are generated by the algorithm we designed to pre-train the model and pure periodic signals are filtered from LSTM to calculate the heart rate. Experiential results on the public-domain database show the effectiveness of our proposed method that can be a reference for the heart rate estimation.

[1]  Gerard de Haan,et al.  Robust Pulse Rate From Chrominance-Based rPPG , 2013, IEEE Transactions on Biomedical Engineering.

[2]  Rosalind W. Picard,et al.  Non-contact, automated cardiac pulse measurements using video imaging and blind source separation , 2022 .

[3]  Frédo Durand,et al.  Detecting Pulse from Head Motions in Video , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[4]  Tomasz Kocejko,et al.  Proceedings of the Federated Conference on Computer Science and Information Systems pp. 405–410 ISBN 978-83-60810-22-4 Measuring Pulse Rate with a Webcam – a Non-contact Method for Evaluating Cardiac Activity , 2022 .

[5]  Matti Pietikäinen,et al.  Generalized face anti-spoofing by detecting pulse from face videos , 2016, 2016 23rd International Conference on Pattern Recognition (ICPR).

[6]  Weihong Deng,et al.  Learning temporal features using LSTM-CNN architecture for face anti-spoofing , 2015, 2015 3rd IAPR Asian Conference on Pattern Recognition (ACPR).

[7]  Davis E. King,et al.  Dlib-ml: A Machine Learning Toolkit , 2009, J. Mach. Learn. Res..

[8]  L. O. Svaasand,et al.  Remote plethysmographic imaging using ambient light. , 2008, Optics express.

[9]  Shaun J. Canavan,et al.  Multimodal Spontaneous Emotion Corpus for Human Behavior Analysis , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[10]  Miroslav Goljan,et al.  Digital camera identification from sensor pattern noise , 2006, IEEE Transactions on Information Forensics and Security.

[11]  Anil K. Jain,et al.  Biometric Template Security , 2008, EURASIP J. Adv. Signal Process..

[12]  John Allen Photoplethysmography and its application in clinical physiological measurement , 2007, Physiological measurement.

[13]  Daniel McDuff,et al.  DeepPhys: Video-Based Physiological Measurement Using Convolutional Attention Networks , 2018, ECCV.

[14]  Winston H. Hsu,et al.  Learning-based heart rate detection from remote photoplethysmography features , 2014, 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[15]  Daniel McDuff,et al.  iPhys: An Open Non-Contact Imaging-Based Physiological Measurement Toolbox , 2019, 2019 41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[16]  G.B. Moody,et al.  PhysioNet: a Web-based resource for the study of physiologic signals , 2001, IEEE Engineering in Medicine and Biology Magazine.

[17]  Hossein Mobahi,et al.  Deep learning from temporal coherence in video , 2009, ICML '09.

[18]  Gee-Sern Hsu,et al.  Deep learning with time-frequency representation for pulse estimation from facial videos , 2017, 2017 IEEE International Joint Conference on Biometrics (IJCB).

[19]  Zachary Chase Lipton A Critical Review of Recurrent Neural Networks for Sequence Learning , 2015, ArXiv.

[20]  Wei Wu,et al.  End-to-End Flow Correlation Tracking with Spatial-Temporal Attention , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[21]  D. Hampson,et al.  Pharmacological profiles of the metabotropic glutamate receptor ligands [3H]L-AP4 and [3H]CPPG , 2001, Neuropharmacology.

[22]  Sander Stuijk,et al.  Algorithmic Principles of Remote PPG , 2017, IEEE Transactions on Biomedical Engineering.