Real-Time Action Recognition System for Elderly People Using Stereo Depth Camera

Smart technologies are necessary for ambient assisted living (AAL) to help family members, caregivers, and health-care professionals in providing care for elderly people independently. Among these technologies, the current work is proposed as a computer vision-based solution that can monitor the elderly by recognizing actions using a stereo depth camera. In this work, we introduce a system that fuses together feature extraction methods from previous works in a novel combination of action recognition. Using depth frame sequences provided by the depth camera, the system localizes people by extracting different regions of interest (ROI) from UV-disparity maps. As for feature vectors, the spatial-temporal features of two action representation maps (depth motion appearance (DMA) and depth motion history (DMH) with a histogram of oriented gradients (HOG) descriptor) are used in combination with the distance-based features, and fused together with the automatic rounding method for action recognition of continuous long frame sequences. The experimental results are tested using random frame sequences from a dataset that was collected at an elder care center, demonstrating that the proposed system can detect various actions in real-time with reasonable recognition rates, regardless of the length of the image sequences.

[1]  Muhammad Mahadi Abdul Jamil,et al.  Human Fall Detection from Depth Images using Position and Velocity of Subject , 2017 .

[2]  Thi Thi Zin,et al.  Image Processing Technique and Hidden Markov Model for an Elderly Care Monitoring System , 2020, J. Imaging.

[3]  Sergiu Nedevschi,et al.  UV disparity based obstacle detection and pedestrian classification in urban traffic scenarios , 2014, 2014 IEEE 10th International Conference on Intelligent Computer Communication and Processing (ICCP).

[4]  Daijin Kim,et al.  A Depth Video Sensor-Based Life-Logging Human Activity Recognition System for Elderly Care in Smart Indoor Environments , 2014, Sensors.

[5]  Thi Thi Zin,et al.  Elderly Monitoring and Action Recognition System Using Stereo Depth Camera , 2020, 2020 IEEE 9th Global Conference on Consumer Electronics (GCCE).

[6]  Xiaodong Yang,et al.  Recognizing actions using depth motion maps-based histograms of oriented gradients , 2012, ACM Multimedia.

[7]  Thaier Hayajneh,et al.  Smartphone and Smartwatch-Based Biometrics Using Activities of Daily Living , 2019, IEEE Access.

[8]  Kun-Lin Lu,et al.  An Image-Based Fall Detection System for the Elderly , 2018, Applied Sciences.

[9]  Z. Hu,et al.  U-V-disparity: an efficient algorithm for stereovision based scene analysis , 2005, IEEE Proceedings. Intelligent Vehicles Symposium, 2005..

[10]  Nasser Kehtarnavaz,et al.  Real-time human action recognition based on depth motion maps , 2013, Journal of Real-Time Image Processing.

[11]  Arif Mahmood,et al.  HOPC: Histogram of Oriented Principal Components of 3D Pointclouds for Action Recognition , 2014, ECCV.

[12]  Max Mignotte,et al.  Fall Detection from Depth Map Video Sequences , 2011, ICOST.

[13]  Thomas Brunschwiler,et al.  An Unsupervised Behavioral Modeling and Alerting System Based on Passive Sensing for Elderly Care , 2020, Future Internet.

[14]  Shaogang Gong,et al.  Video behaviour profiling and abnormality detection without manual labelling , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[15]  Marian Wysocki,et al.  Recognition of Human Activities Using Depth Maps and the Viewpoint Feature Histogram Descriptor , 2020, Sensors.

[16]  Balasubramanian Raman,et al.  Privacy-preserving human action recognition as a remote cloud service using RGB-D sensors and deep CNN , 2020, Expert Syst. Appl..

[17]  Gary M. Weiss,et al.  Smartwatch-based activity recognition: A machine learning approach , 2016, 2016 IEEE-EMBS International Conference on Biomedical and Health Informatics (BHI).

[18]  Md. Atiqur Rahman Ahad,et al.  Motion history image: its variants and applications , 2012, Machine Vision and Applications.

[19]  Alex Mihailidis,et al.  A Survey on Ambient-Assisted Living Tools for Older Adults , 2013, IEEE Journal of Biomedical and Health Informatics.

[20]  Weria Khaksar,et al.  Ambient Sensors for Elderly Care and Independent Living: A Survey , 2018, Sensors.

[21]  Zhenjiang Miao,et al.  Real Time Human Action Recognition in a Long Video Sequence , 2010, 2010 7th IEEE International Conference on Advanced Video and Signal Based Surveillance.

[22]  Dimitrios Tzovaras,et al.  Abnormal Behavior Detection for Elderly People Living Alone Leveraging IoT Sensors , 2019, 2019 IEEE 19th International Conference on Bioinformatics and Bioengineering (BIBE).

[23]  Wenjun Zeng,et al.  Spatio-Temporal Attention-Based LSTM Networks for 3D Action Recognition and Detection , 2018, IEEE Transactions on Image Processing.

[24]  Ho-Sub Yoon,et al.  Action Recogntion with Depth Maps Using HOG Descriptors of Multi-view Motion Appearance and History , 2014 .

[25]  P. Remagnino,et al.  Action recognition on continuous video , 2020, Neural Computing and Applications.

[26]  Mohammed Sadgal,et al.  Skeleton-based human activity recognition for elderly monitoring systems , 2018, IET Comput. Vis..

[27]  Hong Liu,et al.  Action Recognition Based on Optimal Joint Selection and Discriminative Depth Descriptor , 2016, ACCV.

[28]  Sabu Emmanuel,et al.  Video surveillance for elderly monitoring and safety , 2009, TENCON 2009 - 2009 IEEE Region 10 Conference.

[29]  Martin Kampel,et al.  Computer vision for active and assisted living , 2016 .

[30]  Patrick Pérez,et al.  Retrieving actions in movies , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[31]  Juan Carlos Niebles,et al.  Unsupervised Learning of Human Action Categories Using Spatial-Temporal Words , 2008, International Journal of Computer Vision.

[32]  Marco Buzzelli,et al.  A Vision-Based System for Monitoring Elderly People at Home , 2020, Applied Sciences.