Vision Transformers (ViT) for Blanket-Penetrating Sleep Posture Recognition Using a Triple Ultra-Wideband (UWB) Radar System

Sleep posture has a crucial impact on the incidence and severity of obstructive sleep apnea (OSA). Therefore, the surveillance and recognition of sleep postures could facilitate the assessment of OSA. The existing contact-based systems might interfere with sleeping, while camera-based systems introduce privacy concerns. Radar-based systems might overcome these challenges, especially when individuals are covered with blankets. The aim of this research is to develop a nonobstructive multiple ultra-wideband radar sleep posture recognition system based on machine learning models. We evaluated three single-radar configurations (top, side, and head), three dual-radar configurations (top + side, top + head, and side + head), and one tri-radar configuration (top + side + head), in addition to machine learning models, including CNN-based networks (ResNet50, DenseNet121, and EfficientNetV2) and vision transformer-based networks (traditional vision transformer and Swin Transformer V2). Thirty participants (n = 30) were invited to perform four recumbent postures (supine, left side-lying, right side-lying, and prone). Data from eighteen participants were randomly chosen for model training, another six participants’ data (n = 6) for model validation, and the remaining six participants’ data (n = 6) for model testing. The Swin Transformer with side and head radar configuration achieved the highest prediction accuracy (0.808). Future research may consider the application of the synthetic aperture radar technique.

[1]  Andy Yiu-Chau Tam,et al.  Dual ultra-wideband (UWB) radar-based sleep posture recognition system: Towards ubiquitous sleep monitoring , 2023, Engineered Regeneration.

[2]  Abir Rahali,et al.  End-to-End Transformer-Based Models in Textual-Based NLP , 2023, AI.

[3]  Xiang Li,et al.  Sleep posture recognition based on machine learning: A systematic review , 2023, Pervasive Mob. Comput..

[4]  F. D. L. Torre,et al.  DensePose From WiFi , 2022, ArXiv.

[5]  Fangqiang Hu,et al.  CSI-Former: Pay More Attention to Pose Estimation with WiFi , 2022, Entropy.

[6]  Yuh-Shyan Chen,et al.  Multi-Feature Transformer-Based Learning for Continuous Human Motion Recognition with High Similarity Using mmWave FMCW Radar , 2022, Sensors.

[7]  Andy Yiu-Chau Tam,et al.  Depth-Camera-Based Under-Blanket Sleep Posture Classification Using Anatomical Landmark-Guided Deep Learning Model , 2022, International journal of environmental research and public health.

[8]  Jashila Nair Mogan,et al.  Gait-ViT: Gait Recognition with Vision Transformer , 2022, Sensors.

[9]  Bryan Pak-Hei So,et al.  Wrist accelerometry for monitoring dementia agitation behaviour in clinical settings: A scoping review , 2022, Frontiers in Psychiatry.

[10]  M. Ghavami,et al.  Markerless Gait Classification Employing 3D IR-UWB Physiological Motion Sensing , 2022, IEEE Sensors Journal.

[11]  M. Baity-Jesi,et al.  Ensembles of data-efficient vision transformers as a new paradigm for automated classification in ecology , 2022, Scientific Reports.

[12]  Khawar Islam Recent Advances in Vision Transformer: A Survey and Outlook of Recent Work , 2022, ArXiv.

[13]  Yongjun Zheng,et al.  A Night-Time Monitoring System (eNightLog) to Prevent Elderly Wandering in Hostels: A Three-Month Field Study , 2022, International journal of environmental research and public health.

[14]  Xudong Jiang,et al.  Attention-Based Dual-Stream Vision Transformer for Radar Gait Recognition , 2021, ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[15]  Raphael Couturier,et al.  Convolutional Neural Network (CNN) vs Vision Transformer (ViT) for Digital Holography , 2021, 2022 2nd International Conference on Computer, Control and Robotics (ICCCR).

[16]  D. Tao,et al.  A Survey on Vision Transformer , 2020, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[17]  Byung Cheol Song,et al.  Vision Transformer for Small-Size Datasets , 2021, ArXiv.

[18]  F. Xu,et al.  Human Sleep Posture Recognition Based on Millimeter-Wave Radar , 2021, 2021 Signal Processing Symposium (SPSympo).

[19]  Duo Wai-Chi Wong,et al.  A Blanket Accommodative Sleep Posture Classification System Using an Infrared Depth Camera: A Deep Learning Approach with Synthetic Augmentation of Blanket Conditions , 2021, Sensors.

[20]  Beatriz Ortega,et al.  Low-cost plastic optical fiber sensor embedded in mattress for sleep performance monitoring , 2021, Optical Fiber Technology.

[21]  E. Wickwire,et al.  Value-based sleep and breathing: health economic aspects of obstructive sleep apnea , 2021, Faculty reviews.

[22]  Quoc V. Le,et al.  EfficientNetV2: Smaller Models and Faster Training , 2021, ICML.

[23]  Chung-Wai James Cheung,et al.  Night-Time Monitoring System (eNightLog) for Elderly Wandering Behavior , 2021, Sensors.

[24]  A. Yuille,et al.  MaX-DeepLab: End-to-End Panoptic Segmentation with Mask Transformers , 2020, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[25]  S. Gelly,et al.  An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale , 2020, ICLR.

[26]  Brian Kenji Iwana,et al.  An empirical survey of data augmentation for time series classification with neural networks , 2020, PloS one.

[27]  Subhas Chandra Mukhopadhyay,et al.  SleepPoseNet: Multi-View Learning for Sleep Postural Transition Recognition Using UWB , 2020, IEEE Journal of Biomedical and Health Informatics.

[28]  Victor M. Lubecke,et al.  Sleep Posture Recognition With a Dual-Frequency Cardiopulmonary Doppler Radar , 2021, IEEE Access.

[29]  Stephen Lin,et al.  Swin Transformer: Hierarchical Vision Transformer using Shifted Windows , 2021, 2021 IEEE/CVF International Conference on Computer Vision (ICCV).

[30]  Nicolas Usunier,et al.  End-to-End Object Detection with Transformers , 2020, ECCV.

[31]  Baker Mohammad,et al.  Human Vital Signs Detection Methods and Potential Using Radars: A Review , 2020, Sensors.

[32]  Sung Ho Cho,et al.  Hand Gesture Recognition Using an IR-UWB Radar with an Inception Module-Based Classifier , 2020, Sensors.

[33]  W. Zuo,et al.  ECA-Net: Efficient Channel Attention for Deep Convolutional Neural Networks , 2019, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[34]  Ji Liu,et al.  Hand-Transformer: Non-Autoregressive Structured Modeling for 3D Hand Pose Estimation , 2020, ECCV.

[35]  Ou Ma,et al.  Human Posture Recognition Using a Hybrid of Fuzzy Logic and Machine Learning Approaches , 2020, IEEE Access.

[36]  Ali Etemad,et al.  Identity and Posture Recognition in Smart Beds with Deep Multitask Learning , 2019, 2019 IEEE International Conference on Systems, Man and Cybernetics (SMC).

[37]  Liming Chen,et al.  Computational Sleep Behavior Analysis: A Survey , 2019, IEEE Access.

[38]  Quoc V. Le,et al.  EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks , 2019, ICML.

[39]  Hongmei Li,et al.  Bearing Fault Diagnosis with a Feature Fusion Method Based on an Ensemble Convolutional Neural Network and Deep Neural Network , 2019, Sensors.

[40]  Sung Ho Cho,et al.  Quantified Activity Measurement for Medical Use in Movement Disorders through IR-UWB Radar Sensor † , 2019, Sensors.

[41]  Ming Zhang,et al.  Sleeping mattress determinants and evaluation: a biomechanical review and critique , 2019, PeerJ.

[42]  Jun-Young Park,et al.  A Novel Non-contact Heart Rate Monitor Using Impulse-Radio Ultra-Wideband (IR-UWB) Radar Technology , 2018, Scientific Reports.

[43]  James M. Keller,et al.  Sleep Posture Classification Using Bed Sensor Data and Neural Networks , 2018, 2018 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[44]  Adrian Hilton,et al.  Sleep Posture Classification using a Convolutional Neural Network , 2018, 2018 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[45]  Xiang Li,et al.  Mixed Link Networks , 2018, IJCAI.

[46]  Iasonas Kokkinos,et al.  DensePose: Dense Human Pose Estimation in the Wild , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[47]  Martin Masek,et al.  Sleep monitor: A tool for monitoring and categorical scoring of lying position using 3D camera data , 2018, SoftwareX.

[48]  Dong Seog Han,et al.  A real-time sleeping position recognition system using IMU sensor motion data , 2018, 2018 IEEE International Conference on Consumer Electronics (ICCE).

[49]  A. Lowe,et al.  Prevalence of obstructive sleep apnea in the general population: A systematic review. , 2017, Sleep medicine reviews.

[50]  Dana Kulic,et al.  Data augmentation of wearable sensor data for parkinson’s disease monitoring using convolutional neural networks , 2017, ICMI.

[51]  Jong-Hun Lee,et al.  Moving and stationary target detection scheme using coherent integration and subtraction for automotive FMCW radar systems , 2017, 2017 IEEE Radar Conference (RadarConf).

[52]  Kilian Q. Weinberger,et al.  Densely Connected Convolutional Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[53]  Hao Zhang,et al.  An Improved Algorithm for Through-Wall Target Detection Using Ultra-Wideband Impulse Radar , 2017, IEEE Access.

[54]  Aske Plaat,et al.  On the Impact of Data Set Size in Transfer Learning Using Deep Neural Networks , 2016, IDA.

[55]  Thiemo Voigt,et al.  3DinSAR: Object 3D localization for indoor RFID applications , 2016, 2016 IEEE International Conference on RFID (RFID).

[56]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[57]  Manoj Kumar,et al.  Influence of Body Position on Severity of Obstructive Sleep Apnea: A Systematic Review , 2013, ISRN otolaryngology.

[58]  Matthew L. Ho,et al.  Obstructive sleep apnea , 2011, Neurology international.

[59]  Fei-Fei Li,et al.  ImageNet: A large-scale hierarchical image database , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[60]  M. Skubic,et al.  Senior residents’ perceived need of and preferences for “smart home” sensor technologies , 2008, International Journal of Technology Assessment in Health Care.

[61]  H. Tinteren,et al.  The role of sleep position in obstructive sleep apnea syndrome , 2006, European Archives of Oto-Rhino-Laryngology and Head & Neck.

[62]  Michael F. Otero,et al.  Application of a continuous wave radar for human gait recognition , 2005, SPIE Defense + Commercial Sensing.

[63]  V. Somers,et al.  Obstructive Sleep Apnea , 2005, Annals of Internal Medicine.

[64]  T. Nishino,et al.  Comparison of static mechanical properties of the passive pharynx between normal children and children with sleep-disordered breathing. , 1998, American journal of respiratory and critical care medicine.

[65]  R. Rogers,et al.  Obstructive sleep apnea. , 1996, The New England journal of medicine.

[66]  J. Haze,et al.  Obstructive sleep apnea. , 1987, Cranio : the journal of craniomandibular practice.