Vision-Based Heart and Breath Rate Monitoring Using Skin Color and Motion

Visual sensory system has attracted much interest in medical research due to the non-contact characteristic. Heart and breath rate are usually used as the primary indicator of the subject health status. Conventional heart and breath rate monitoring that is attached to the body is practically difficult to apply under certain conditions such as active babies, patients with burns or skin healing, and when free movement is needed. Therefore, in this paper, visual-based heart and breath rate monitoring system is developed by using a standard 30-fps camera as the primary sensor. Skin color and motion are magnified, followed by capturing region-of-interest, normalization, and filtering to get the respective heart and breath signals. Skin color is used due to blood flow activity during the cardiac cycle. Color and motion magnification is employed to strengthen changes in the information of video because signals from non-contact measurements have a low amplitude. Results from 28 subjects provided good acceptance level of 95% compared to the contact-based sensor, which is used as a validation tool.

[1]  L. O. Svaasand,et al.  Remote plethysmographic imaging using ambient light. , 2008, Optics express.

[2]  Rafik A. Goubran,et al.  Adaptive eulerian video magnification methods to extract heart rate from thermal video , 2016, 2016 IEEE International Symposium on Medical Measurements and Applications (MeMeA).

[3]  Anton M. Unakafov,et al.  Pulse rate estimation using imaging photoplethysmography: generic framework and comparison of methods on a publicly available dataset , 2017, 1710.08369.

[4]  L. Tarassenko,et al.  An assessment of algorithms to estimate respiratory rate from the electrocardiogram and photoplethysmogram , 2016, Physiological measurement.

[5]  John Allen.  Photoplethysmography and its application in clinical physiological measurement , 2007, Physiological measurement.

[6]  James R. Green,et al.  Eulerian Magnification of Multi-Modal RGB-D Video for Heart Rate Estimation , 2018, 2018 IEEE International Symposium on Medical Measurements and Applications (MeMeA).

[7]  Wenjin Wang.  Robust and automatic remote photoplethysmography , 2017 .

[8]  Philippe Renevey,et al.  Wrist-located pulse detection using IR signals, activity and nonlinear artifact cancellation , 2001, 2001 Conference Proceedings of the 23rd Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[9]  K. Hillman,et al.  Introduction of the medical emergency team (MET) system: a cluster-randomised controlled trial , 2005, The Lancet.

[10]  Michael Rubinstein,et al.  Analysis and visualization of temporal variations in video , 2014 .

[11]  Yu Sun,et al.  Photoplethysmography Revisited: From Contact to Noncontact, From Point to Imaging , 2016, IEEE Transactions on Biomedical Engineering.

[12]  C. Subbe,et al.  Effect of introducing the Modified Early Warning score on clinical outcomes, cardio‐pulmonary arrests and intensive care utilisation in acute medical admissions * , 2003, Anaesthesia.

[13]  Gerard de Haan,et al.  Robust Pulse Rate From Chrominance-Based rPPG , 2013, IEEE Transactions on Biomedical Engineering.

[14]  Y. Mendelson,et al.  A Wearable Reflectance Pulse Oximeter for Remote Physiological Monitoring , 2006, 2006 International Conference of the IEEE Engineering in Medicine and Biology Society.

[15]  Abdulmotaleb El Saddik,et al.  Heart Rate Variability Extraction From Videos Signals: ICA vs. EVM Comparison , 2017, IEEE Access.

[16]  Frédo Durand,et al.  Eulerian video magnification for revealing subtle changes in the world , 2012, ACM Trans. Graph..

[17]  K. Hillman,et al.  The objective medical emergency team activation criteria: a case-control study. , 2007, Resuscitation.

[18]  Daniel McDuff,et al.  Advancements in Noncontact, Multiparameter Physiological Measurements Using a Webcam , 2011, IEEE Transactions on Biomedical Engineering.

[19]  D. Wakefield,et al.  Respiratory rate predicts cardiopulmonary arrest for internal medicine inpatients , 1993, Journal of General Internal Medicine.

[20]  Paul A. Viola,et al.  Rapid object detection using a boosted cascade of simple features , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[21]  H. Harry Asada,et al.  A twenty-four hour tele-nursing system using a ring sensor , 1998, Proceedings. 1998 IEEE International Conference on Robotics and Automation (Cat. No.98CH36146).

[22]  F. Flack,et al.  Comparative Review of Techniques for Recording Respiratory Events at Rest and during Deglutition , 1997, Dysphagia.