Non-Contact Respiration Monitoring and Body Movements Detection for Sleep Using Thermal Imaging

Monitoring of respiration and body movements during sleep is a part of screening sleep disorders related to health status. Nowadays, thermal-based methods are presented to monitor the sleeping person without any sensors attached to the body to protect privacy. A non-contact respiration monitoring based on thermal videos requires visible facial landmarks like nostril and mouth. The limitation of these techniques is the failure of face detection while sleeping with a fixed camera position. This study presents the non-contact respiration monitoring approach that does not require facial landmark visibility under the natural sleep environment, which implies an uncontrolled sleep posture, darkness, and subjects covered with a blanket. The automatic region of interest (ROI) extraction by temperature detection and breathing motion detection is based on image processing integrated to obtain the respiration signals. A signal processing technique was used to estimate respiration and body movements information from a sequence of thermal video. The proposed approach has been tested on 16 volunteers, for which video recordings were carried out by themselves. The participants were also asked to wear the Go Direct respiratory belt for capturing reference data. The result revealed that our proposed measuring respiratory rate obtains root mean square error (RMSE) of 1.82±0.75 bpm. The advantage of this approach lies in its simplicity and accessibility to serve users who require monitoring the respiration during sleep without direct contact by themselves.

[1]  Joel Nothman,et al.  SciPy 1.0-Fundamental Algorithms for Scientific Computing in Python , 2019, ArXiv.

[2]  K R Casey,et al.  Hypopnea in sleep-disordered breathing in adults. , 2001, Sleep.

[3]  Ioannis T. Pavlidis,et al.  Analysis of Breathing Air Flow Patterns in Thermal Imaging , 2006, 2006 International Conference of the IEEE Engineering in Medicine and Biology Society.

[4]  Sergio Calixto-Carrera,et al.  Novel approach to assess the emissivity of the human skin. , 2009, Journal of biomedical optics.

[5]  Adelaide M. Arruda-Olson,et al.  Sleep Apnea and Cardiovascular Disease , 2003, Herz.

[6]  Ali Al-Naji,et al.  Remote respiratory monitoring system based on developing motion magnification technique , 2016, Biomed. Signal Process. Control..

[7]  Mariusz Kaczmarek,et al.  Evaluating Accuracy of Respiratory Rate Estimation from Super Resolved Thermal Imagery , 2019, 2019 41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[8]  Reza Saatchi,et al.  Facial Tracking in Thermal Images for Real-Time Noncontact Respiration Rate Monitoring , 2013, 2013 European Modelling Symposium.

[9]  Wenhan Zhu,et al.  Combination of near-infrared and thermal imaging techniques for the remote and simultaneous measurements of breathing and heart rates under sleep situation , 2018, PloS one.

[10]  Reza Saatchi,et al.  Thermal image processing for real-time non-contact respiration rate monitoring , 2017, IET Circuits Devices Syst..

[11]  Z. Yaniv,et al.  Monitoring Patient Respiration using a Single Optical Camera , 2007, 2007 29th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[12]  Douglas Tapper,et al.  Sleep apnea. , 2018, Otolaryngologic clinics of North America.

[13]  Steffen Leonhardt,et al.  Estimation of breathing rate in thermal imaging videos: a pilot study on healthy human subjects , 2017, Journal of Clinical Monitoring and Computing.

[14]  J. Amat,et al.  VISION BASED RESPIRATORY MONITORING SYSTEM , 2022 .

[15]  Toshiyo Tamura,et al.  A monitor for posture changes and respiration in bed using real time image sequence analysis , 2000, Proceedings of the 22nd Annual International Conference of the IEEE Engineering in Medicine and Biology Society (Cat. No.00CH37143).

[16]  K. Hillman,et al.  The objective medical emergency team activation criteria: a case-control study. , 2007, Resuscitation.

[17]  Steffen Leonhardt,et al.  Remote monitoring of breathing dynamics using infrared thermography. , 2015, Biomedical optics express.

[18]  Reza Saatchi,et al.  Non-invasive respiration monitoring by thermal imaging to detect sleep apnoea , 2019 .

[19]  Abbas K. Abbas,et al.  Neonatal non-contact respiratory monitoring based on real-time infrared thermography , 2011, Biomedical engineering online.

[20]  Sabine Van Huffel,et al.  Automated Respiration Detection from Neonatal Video Data , 2015, ICPRAM.

[21]  R. Schafer,et al.  What Is a Savitzky-Golay Filter? , 2022 .

[22]  Gregory F. Lewis,et al.  A novel method for extracting respiration rate and relative tidal volume from infrared thermography. , 2011, Psychophysiology.

[23]  I. Pavlidis,et al.  Noncontact measurement of breathing function , 2006, IEEE Engineering in Medicine and Biology Magazine.

[24]  Reza Saatchi,et al.  Respiratory rate measurement in children using a thermal camera , 2015 .

[25]  Ioannis T. Pavlidis,et al.  Thermistor at a Distance: Unobtrusive Measurement of Breathing , 2010, IEEE Transactions on Biomedical Engineering.

[26]  I. Pavlidis,et al.  Touchless monitoring of breathing function , 2004, The 26th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[27]  Lalat Indu Giri,et al.  Non-contact monitoring of human respiration using infrared thermography and machine learning , 2020 .

[28]  Rafik A. Goubran,et al.  The detection of breathing behavior using Eulerian-enhanced thermal video , 2015, 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[29]  Beena Ahmed,et al.  Periodic leg movement (PLM) monitoring using a distributed body sensor network , 2015, 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[30]  Steffen Leonhardt,et al.  Noncontact Monitoring of Respiratory Rate in Newborn Infants Using Thermal Imaging , 2019, IEEE Transactions on Biomedical Engineering.

[31]  Sander Stuijk,et al.  Unobtrusive respiratory flow monitoring using a thermopile array: a feasibility study , 2019 .

[32]  Patrick Garda,et al.  Thermal-Signature-Based Sleep Analysis Sensor , 2017, Informatics.

[33]  Reza Saatchi,et al.  Tracking human face features in thermal images for respiration monitoring , 2010, ACS/IEEE International Conference on Computer Systems and Applications - AICCSA 2010.

[34]  Ronald W. Schafer,et al.  What Is a Savitzky-Golay Filter? [Lecture Notes] , 2011, IEEE Signal Processing Magazine.

[35]  Ioannis T. Pavlidis,et al.  Thermal Vision for Sleep Apnea Monitoring , 2009, MICCAI.

[36]  Zhangjie Chen,et al.  Sleep monitoring using an infrared thermal array sensor , 2019, Smart Structures and Materials + Nondestructive Evaluation and Health Monitoring.

[37]  L. Tarassenko,et al.  Non-contact video-based vital sign monitoring using ambient light and auto-regressive models , 2014, Physiological measurement.

[38]  Reza Saatchi,et al.  Tracing the Region of Interest in Thermal Human Face for Respiration Monitoring , 2015 .

[39]  M. Sheppard,et al.  Principles and Practice of High Dependency Nursing , 2006 .

[40]  Yee Siong Lee,et al.  Monitoring and Analysis of Respiratory Patterns Using Microwave Doppler Radar , 2014, IEEE Journal of Translational Engineering in Health and Medicine.