Gesture Recognition Intermediary Robot for Abnormality Detection in Human Activities

The world ageing population is increasing, giving rise to research targeted towards improving the quality of life and promoting the independent living of older adults. Detecting abnormalities in the daily activities of the older adults is relevant since abnormalities can be an early sign of health decline, prompting for the need for intervention. Current approaches to abnormality detection involve modelling the usual behavioural routine of the individual as a baseline and comparing subsequent behaviour to the baseline to detect abnormalities. This approach is prone to errors and not flexible since it does not take into account changes in human behavioural routine. Training is usually performed on pre-existing data making the abnormality detection model non-adaptive to new incoming data. An intermediary can be incorporated to enable model predictions to be communicated to humans for verification of the detected anomalies. This paper proposes a gesture recognition approach for facilitating interaction between humans and a robot intermediary. A model capable of recognising hand gestures corresponding to affirmations and denials is implemented. Preliminary evaluation shows that the proposed gesture recognition approach has the potential of being utilised in an assistive robot intermediary.

[1]  Lola Cañamero,et al.  Making New “New AI” Friends: Designing a Social Robot for Diabetic Children from an Embodied AI Perspective , 2016, Int. J. Soc. Robotics.

[2]  Ren C. Luo,et al.  Multimodal information fusion for human-robot interaction , 2015, 2015 IEEE 10th Jubilee International Symposium on Applied Computational Intelligence and Informatics.

[3]  Li Jinghua,et al.  A Robust Hand Gesture Recognition Method via Convolutional Neural Network , 2016, 2016 6th International Conference on Digital Home (ICDH).

[4]  Seong-Whan Lee,et al.  Real-time 3D pointing gesture recognition for mobile robots with cascade HMM and particle filter , 2011, Image Vis. Comput..

[5]  Marcelo H. Ang,et al.  Why Robots? A Survey on the Roles and Benefits of Social Robots in the Therapy of Children with Autism , 2013, International Journal of Social Robotics.

[6]  Ali Farhadi,et al.  You Only Look Once: Unified, Real-Time Object Detection , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[7]  Ahmad Lotfi,et al.  A Framework for Anomaly Detection in Activities of Daily Living using an Assistive Robot , 2019, UK-RAS19 Conference: "Embedded Intelligence: Enabling and Supporting RAS Technologies" Proceedings.

[8]  Enamul Hoque,et al.  Holmes: A Comprehensive Anomaly Detection System for Daily In-home Activities , 2015, 2015 International Conference on Distributed Computing in Sensor Systems.

[9]  Hongnian Yu,et al.  Elderly activities recognition and classification for applications in assisted living , 2013, Expert Syst. Appl..

[10]  Luca Maria Gambardella,et al.  Max-pooling convolutional neural networks for vision-based hand gesture recognition , 2011, 2011 IEEE International Conference on Signal and Image Processing Applications (ICSIPA).

[11]  Cleber Zanchettin,et al.  Gesture recognition: A review focusing on sign language in a mobile context , 2018, Expert Syst. Appl..

[12]  Horst-Michael Groß,et al.  I'll keep an eye on you: Home robot companion for elderly people with cognitive impairment , 2011, 2011 IEEE International Conference on Systems, Man, and Cybernetics.

[13]  Diane J. Cook,et al.  Automatic assessment of functional health decline in older adults based on smart home data , 2018, J. Biomed. Informatics.

[14]  Maja J. Mataric,et al.  The role of physical embodiment in human-robot interaction , 2006, ROMAN 2006 - The 15th IEEE International Symposium on Robot and Human Interactive Communication.

[15]  Andrew Zisserman,et al.  Convolutional Two-Stream Network Fusion for Video Action Recognition , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[16]  Ahmad Lotfi,et al.  Socially Assistive Robotics: Robot Exercise Trainer for Older Adults , 2018 .

[17]  Ahmad Lotfi,et al.  A Consensus Novelty Detection Ensemble Approach for Anomaly Detection in Activities of Daily Living , 2019, Appl. Soft Comput..

[18]  Ahmad Lotfi,et al.  Smart homes for the elderly dementia sufferers: identification and prediction of abnormal behaviour , 2012, J. Ambient Intell. Humaniz. Comput..

[19]  Damla Arifoglu,et al.  Detection of abnormal behaviour for dementia sufferers using Convolutional Neural Networks , 2019, Artif. Intell. Medicine.

[20]  Jie Liu,et al.  3D separable convolutional neural network for dynamic hand gesture recognition , 2018, Neurocomputing.