Remote heart rate measurement using low-cost RGB face video: a technical literature review

Remote photoplethysmography (rPPG) allows remote measurement of the heart rate using low-cost RGB imaging equipment. In this study, we review the development of the field of rPPG since its emergence in 2008. We also classify existing rPPG approaches and derive a framework that provides an overview of modular steps. Based on this framework, practitioners can use our classification to design algorithms for an rPPG approach that suits their specific needs. Researchers can use the reviewed and classified algorithms as a starting point to improve particular features of an rPPG algorithm.

[1]  Lei Zhang,et al.  Real-Time Compressive Tracking , 2012, ECCV.

[2]  Gerard de Haan,et al.  Robust Pulse Rate From Chrominance-Based rPPG , 2013, IEEE Transactions on Biomedical Engineering.

[3]  Thomas B. Moeslund,et al.  Improved pulse detection from head motions using DCT , 2014, 2014 International Conference on Computer Vision Theory and Applications (VISAPP).

[4]  Takeo Kanade,et al.  An Iterative Image Registration Technique with an Application to Stereo Vision , 1981, IJCAI.

[5]  Hao-Yu Wu.  Eulerian Video Processing and Medical Applications , 2013 .

[6]  Matti Pietikäinen,et al.  Remote Heart Rate Measurement from Face Videos under Realistic Situations , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[7]  Rui Caseiro,et al.  Exploiting the Circulant Structure of Tracking-by-Detection with Kernels , 2012, ECCV.

[8]  Tomasz Kocejko,et al.  Proceedings of the Federated Conference on Computer Science and Information Systems pp. 405–410 ISBN 978-83-60810-22-4 Measuring Pulse Rate with a Webcam – a Non-contact Method for Evaluating Cardiac Activity , 2022 .

[9]  David A. Clausi,et al.  Remote Heart Rate Measurement through Broadband Video via Stochastic Bayesian Estimation , 2015 .

[10]  Raveendran Paramesran,et al.  Video-based heart rate measurement using short-time Fourier transform , 2013, 2013 International Symposium on Intelligent Signal Processing and Communication Systems.

[11]  Michael Felsberg,et al.  Accurate Scale Estimation for Robust Visual Tracking , 2014, BMVC.

[12]  H. Emrah Tasli,et al.  Remote PPG based vital sign measurement using adaptive facial regions , 2014, 2014 IEEE International Conference on Image Processing (ICIP).

[13]  Marius B. Müller,et al.  Auction Fever! How Time Pressure and Social Competition Affect Bidders’ Arousal and Bids in Retail Auctions , 2015 .

[14]  M. M. A. Hashem,et al.  Development of a device for remote monitoring of heart rate and body temperature , 2012, 2012 15th International Conference on Computer and Information Technology (ICCIT).

[15]  Lai-Man Po,et al.  Motion artifacts suppression for remote imaging photoplethysmography , 2014, 2014 19th International Conference on Digital Signal Processing.

[16]  Janko Drnovsek,et al.  Non-contact heart rate and heart rate variability measurements: A review , 2014, Biomed. Signal Process. Control..

[17]  Toni M. Somers,et al.  The Role of ERP Implementation in Enabling Digital Options: A Theoretical and Empirical Analysis , 2009, Int. J. Electron. Commer..

[18]  Paul A. Viola,et al.  Rapid object detection using a boosted cascade of simple features , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[19]  Rafael A. Calvo,et al.  Using Remote Heart Rate Measurement for Affect Detection , 2014, FLAIRS.

[20]  Changick Kim,et al.  A robust real time system for remote heart rate measurement via camera , 2015, 2015 IEEE International Conference on Multimedia and Expo (ICME).

[21]  Touradj Ebrahimi,et al.  Automatic Webcam-Based Human Heart Rate Measurements Using Laplacian Eigenmap , 2012, ACCV.

[22]  Peter J Lesniewski,et al.  Signal recovery in imaging photoplethysmography , 2013, Physiological measurement.

[23]  Carlo Tomasi,et al.  Good features to track , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[24]  Christopher Hunt,et al.  Notes on the OpenSURF Library , 2009 .

[25]  Sander Stuijk,et al.  Exploiting Spatial Redundancy of Image Sensor for Motion Robust rPPG , 2015, IEEE Transactions on Biomedical Engineering.

[26]  Christof Weinhardt,et al.  Integrating Biosignals into Information Systems: A NeuroIS Tool for Improving Emotion Regulation , 2013, J. Manag. Inf. Syst..

[27]  Yuan-Hsiang Lin,et al.  A real-time non-contact pulse rate detector based on smartphone , 2015, 2015 International Symposium on Next-Generation Electronics (ISNE).

[28]  Daniel McDuff,et al.  Remote Detection of Photoplethysmographic Systolic and Diastolic Peaks Using a Digital Camera , 2014, IEEE Transactions on Biomedical Engineering.

[29]  Derek Nowrouzezahrai,et al.  Learning hatching for pen-and-ink illustration of surfaces , 2012, TOGS.

[30]  Daniel McDuff,et al.  Improvements in Remote Cardiopulmonary Measurement Using a Five Band Digital Camera , 2014, IEEE Transactions on Biomedical Engineering.

[31]  Gunnar Farnebäck,et al.  Two-Frame Motion Estimation Based on Polynomial Expansion , 2003, SCIA.

[32]  Kual-Zheng Lee,et al.  Contact-Free Heart Rate Measurement Using a Camera , 2012, 2012 Ninth Conference on Computer and Robot Vision.

[33]  Zhilin Zhang,et al.  TROIKA: A General Framework for Heart Rate Monitoring Using Wrist-Type Photoplethysmographic Signals During Intensive Physical Exercise , 2014, IEEE Transactions on Biomedical Engineering.

[34]  J. Tsien,et al.  Remote Measurements of Heart and Respiration Rates for Telemedicine , 2013, PloS one.

[35]  Dorin Comaniciu,et al.  Kernel-Based Object Tracking , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[36]  Yadong Wang,et al.  A review of non-contact, low-cost physiological information measurement based on photoplethysmographic imaging , 2012, 2012 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[37]  G. Haan,et al.  Improved motion robustness of remote-PPG by using the blood volume pulse signature , 2014, Physiological measurement.

[38]  K. Hoffmann.  Biosignale erfassen und verarbeiten , 2011 .

[39]  Maja Pantic,et al.  Local Evidence Aggregation for Regression-Based Facial Point Detection , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[40]  Mika P. Tarvainen,et al.  An advanced detrending method with application to HRV analysis , 2002, IEEE Transactions on Biomedical Engineering.

[41]  P. Ake Oberg,et al.  Optical properties of blood in motion , 1993 .

[42]  Lei Wang,et al.  Exploration of the optimal skin-camera distance for facial photoplethysmographic imaging measurement using cameras of different types , 2015, EAI Endorsed Trans. Ambient Syst..

[43]  Ashok Veeraraghavan,et al.  DistancePPG: Robust non-contact vital signs monitoring using a camera , 2015, Biomedical optics express.

[44]  P. Ake Oberg,et al.  Optical properties of blood in motion , 1992, Photonics West - Lasers and Applications in Science and Engineering.

[45]  Frédo Durand,et al.  Eulerian video magnification for revealing subtle changes in the world , 2012, ACM Trans. Graph..

[46]  Daniel McDuff,et al.  Remote measurement of cognitive stress via heart rate variability , 2014, 2014 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[47]  Simon Lucey,et al.  Deformable Model Fitting by Regularized Landmark Mean-Shift , 2010, International Journal of Computer Vision.

[48]  Winston H. Hsu,et al.  Learning-based heart rate detection from remote photoplethysmography features , 2014, 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[49]  Lai-Man Po,et al.  Motion-Resistant Remote Imaging Photoplethysmography Based on the Optical Properties of Skin , 2015, IEEE Transactions on Circuits and Systems for Video Technology.

[50]  Timothy F. Cootes,et al.  Active Appearance Models , 1998, ECCV.

[51]  Christof Weinhardt,et al.  Excitement Up! Price Down! Measuring Emotions in Dutch Auctions , 2012, Int. J. Electron. Commer..

[52]  Kwang Suk Park,et al.  Validation of heart rate extraction using video imaging on a built-in camera system of a smartphone , 2012, 2012 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[53]  Rubén Usamentiaga,et al.  Unobtrusive health monitoring system using video-based physiological information and activity measurements , 2015, 2015 International Conference on Computer, Information and Telecommunication Systems (CITS).

[54]  Stefanos Zafeiriou,et al.  Robust Discriminative Response Map Fitting with Constrained Local Models , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[55]  Léon J. M. Rothkrantz,et al.  Noncontact automatic heart rate analysis in visible spectrum by specific face regions , 2013, CompSysTech '13.

[56]  Daniel McDuff,et al.  A survey of remote optical photoplethysmographic imaging methods , 2015, 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[57]  Minghui Yu,et al.  Video-based heart rate measurement using head motion tracking and ICA , 2013, 2013 6th International Congress on Image and Signal Processing (CISP).

[58]  Hagen Malberg,et al.  ROI Selection for Remote Photoplethysmography , 2013, Bildverarbeitung für die Medizin.

[59]  Daniel McDuff,et al.  Advancements in Noncontact, Multiparameter Physiological Measurements Using a Webcam , 2011, IEEE Transactions on Biomedical Engineering.

[60]  David Cornforth,et al.  Using Contactless Heart Rate Measurements for Real-Time Assessment of Affective States , 2017 .

[61]  Hong Yan,et al.  A Machine Learning Approach to Improve Contactless Heart Rate Monitoring Using a Webcam , 2014, IEEE Journal of Biomedical and Health Informatics.

[62]  Rosalind W. Picard,et al.  Non-contact, automated cardiac pulse measurements using video imaging and blind source separation , 2022 .

[63]  Frédo Durand,et al.  Detecting Pulse from Head Motions in Video , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[64]  René Riedl,et al.  On the biology of technostress: literature review and research agenda , 2012, DATB.

[65]  Lai-Man Po,et al.  Dynamic ROI based on K-means for remote photoplethysmography , 2015, 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[66]  I. Starr,et al.  STUDIES ON THE ESTIMATION OF CARDIAC OUPTUT IN MAN, AND OF ABNORMALITIES IN CARDIAC FUNCTION, FROM THE HEART'S RECOIL AND THE BLOOD'S IMPACTS; THE BALLISTOCARDIOGRAM , 1939 .

[67]  L. O. Svaasand,et al.  Remote plethysmographic imaging using ambient light. , 2008, Optics express.

[68]  John Allen.  Photoplethysmography and its application in clinical physiological measurement , 2007, Physiological measurement.

[69]  Shuchang Xu,et al.  Robust efficient estimation of heart rate pulse from video. , 2014, Biomedical optics express.