Benefits of automatic human action recognition in an assistive system for people with dementia

In the context of Activities of Daily Living, a human action can be defined as any interaction between an individual and his or her environment during a task (e.g., to use the soap during handwashing). When an assistive system is designed to recall users what to do during a task, one of its goals is to properly track and detect their actions in order to provide accurate guidance. This paper describes a k-Nearest Neighbor (kNN) based Action Recognition System (ARS) for use in COACH, which is an assistive technology designed for people with dementia. The kNN-based ARS is able to recognize 6 main actions related to the handwashing task. The recognition is done in real-time and uses continuous sequences of discrete hand positions output by a handtracker. The aims of this new ARS are (1) to verify the benefits of enabling automatic human action recognition in COACH, (2) to evaluate its ability to overcome the limitations experienced during previous clinical trials.

[1]  E. Jean-Baptiste.  Statistical task modeling of activities of daily living for rehabilitation , 2016 .

[2]  Jesse Hoey,et al.  Automatic Task Assistance for People with Cognitive Disabilities in Brushing Teeth - A User Study with the TEBRA System , 2014, TACC.

[3]  A. Mihailidis,et al.  The COACH prompting system to assist older adults with dementia through handwashing: An efficacy study , 2008, BMC geriatrics.

[4]  Dan Högberg,et al.  k-Nearest-Neighbour based Numerical Hand Posture Recognition using a Smart Textile Glove , 2015 .

[5]  Diane J. Cook,et al.  Human Activity Recognition and Pattern Discovery , 2010, IEEE Pervasive Computing.

[6]  Abdelsalam Helal,et al.  Training-Free Fuzzy Logic Based Human Activity Recognition , 2014, J. Inf. Process. Syst..

[7]  Gerhard Tröster,et al.  Incremental kNN Classifier Exploiting Correct-Error Teacher for Activity Recognition , 2010, 2010 Ninth International Conference on Machine Learning and Applications.

[8]  Martin J. Russell,et al.  Intelligent Assistive System Using Real-Time Action Recognition for Stroke Survivors , 2014, 2014 IEEE International Conference on Healthcare Informatics.

[9]  Kilian Q. Weinberger,et al.  Distance Metric Learning for Large Margin Nearest Neighbor Classification , 2005, NIPS.

[10]  Naokazu Yokoya,et al.  Flexible human action recognition in depth video sequences using masked joint trajectories , 2016, EURASIP J. Image Video Process..

[11]  Pia Rotshtein,et al.  CogWatch: Intelligent agent-based system to assist stroke survivors during tea-making , 2015, 2015 SAI Intelligent Systems Conference (IntelliSys).

[12]  Alex Mihailidis,et al.  A real-world deployment of the COACH prompting system , 2013, J. Ambient Intell. Smart Environ..

[13]  Martin J. Russell,et al.  Object-Centred Recognition of Human Activity , 2015, 2015 International Conference on Healthcare Informatics.

[14]  Alex Mihailidis,et al.  Development and evaluation of a hand tracker using depth images captured from an overhead perspective , 2016, Disability and rehabilitation. Assistive technology.

[15]  Jesse Hoey.  Tracking using Flocks of Features, with Application to Assisted Handwashing , 2006, BMVC.

[16]  Martin J. Russell,et al.  CogWatch - Automated Assistance and Rehabilitation of Stroke-Induced Action Disorders in the Home Environment , 2013, HCI.

[17]  Peter E. Hart,et al.  Nearest neighbor pattern classification , 1967, IEEE Trans. Inf. Theory.

[18]  José M. Cogollor,et al.  Using Human-Computer Interface for Rehabilitation of Activities of Daily Living (ADL) in Stroke Patients: Lessons from the First Prototype , 2014 .

[19]  Martin J. Russell,et al.  POMDP Based Action Planning and Human Error Detection , 2015, AIAI.

[20]  Rama Chellappa,et al.  Machine Recognition of Human Activities: A Survey , 2008, IEEE Transactions on Circuits and Systems for Video Technology.

[21]  Jesse Hoey,et al.  Automated handwashing assistance for persons with dementia using video and a partially observable Markov decision process , 2010, Comput. Vis. Image Underst..