Skeleton-Based Detection of Abnormalities in Human Actions Using Graph Convolutional Networks

Human action abnormality detection has been attempted by various sensors for application domains like rehabilitation, healthcare, and assisted living. Since the release of motion sensors that ease the human body skeleton retrieval, skeleton-based human action recognition has recently been an active topic in the area of artificial intelligence. Unlike human action recognition, human action abnormality detection is an emerging field that aims to detect the incorrect action from the same action class. Graph convolutional network has been widely adopted for human action recognition. However, to the best of our knowledge, whether it could be effective for the task of human action abnormality detection has not been attempted. To advance prior work in the emerging field of human action abnormality detection, we propose a novel method that uses graph convolutional network to detect abnormal actions in skeleton data. To validate the effectiveness of our proposed method, we conduct extensive experiments on a public dataset called UI-PRMD. Based on the experimental results, our proposed method achieved superior action abnormality detection performance comparing with existing deep learning methods.

[1]  Zhenghao Chen,et al.  Disentangling and Unifying Graph Convolutions for Skeleton-Based Action Recognition , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[2]  Harry Prapavessis,et al.  Home-Based Physical Therapy Intervention With Adherence-Enhancing Strategies Versus Clinic-Based Management for Patients With Ankle Sprains , 2007, Physical Therapy.

[3]  Omar Shahid,et al.  Vision-based Action Understanding for Assistive Healthcare: A Short Review , 2019, CVPR Workshops.

[4]  Geoffrey E. Hinton,et al.  Distilling the Knowledge in a Neural Network , 2015, ArXiv.

[5]  J. Ratcliffe,et al.  Long-term clinical benefits and costs of an integrated rehabilitation programme compared with outpatient physiotherapy for chronic knee pain. , 2009, Physiotherapy.

[6]  Majid Mirmehdi,et al.  View-Invariant Pose Analysis for Human Movement Assessment from RGB Data , 2019, ICIAP.

[7]  William W Yu,et al.  Determinants of Utilization and Expenditures for Episodes of Ambulatory Physical Therapy Among Adults , 2011, Physical Therapy.

[8]  Crina Grosan,et al.  Gesture Correctness Estimation with Deep Neural Networks and Rough Path Descriptors , 2019, 2019 International Conference on Data Mining Workshops (ICDMW).

[9]  Jeanna Basnett,et al.  Quality and Quantity of Rehabilitation Exercises Delivered By A 3-D Motion Controlled Camera: A Pilot Study , 2014, International journal of physical medicine & rehabilitation.

[10]  Dima Damen,et al.  Efficient and Robust Skeleton-Based Quality Assessment and Abnormality Detection in Human Action Performance , 2020, IEEE Journal of Biomedical and Health Informatics.

[11]  Dahua Lin,et al.  Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognition , 2018, AAAI.

[12]  Jake K. Aggarwal,et al.  Human activity recognition from 3D data: A review , 2014, Pattern Recognit. Lett..

[13]  Marco La Cascia,et al.  3D skeleton-based human action classification: A survey , 2016, Pattern Recognit..

[14]  Shah Ebrahim,et al.  Promoting patient uptake and adherence in cardiac rehabilitation. , 2014, The Cochrane database of systematic reviews.

[15]  Min Xian,et al.  A Deep Learning Framework for Assessing Physical Rehabilitation Exercises , 2019, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[16]  Dinesh Kumar Vishwakarma,et al.  A Robust Framework for Abnormal Human Action Recognition Using $\boldsymbol{\mathcal{R}}$ -Transform and Zernike Moments in Depth Videos , 2019, IEEE Sensors Journal.

[17]  Majid Mirmehdi,et al.  A comparative study of pose representation and dynamics modelling for online motion quality assessment , 2016, Comput. Vis. Image Underst..

[18]  Qing Lei,et al.  A Survey of Vision-Based Human Action Evaluation Methods , 2019, Sensors.

[19]  Russell Baker,et al.  A Data Set of Human Body Movements for Physical Rehabilitation Exercises , 2018, Data.

[20]  Margrit Betke,et al.  ExerciseCheck: Remote Monitoring and Evaluation Platform for Home Based Physical Therapy , 2017, PETRA.

[21]  Lei Shi,et al.  Two-Stream Adaptive Graph Convolutional Networks for Skeleton-Based Action Recognition , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[22]  Daniel P. Siewiorek,et al.  AHA-3D: A Labelled Dataset for Senior Fitness Exercise Recognition and Segmentation from 3D Skeletal Data , 2018, BMVC.