Cleaning Behavior Estimation for Self-Supported Cognitive Rehabilitation System

People suffering from brain injury can present complications such as attention disorder, memory disorder, functional disorder, and aphasia, which limit their capacity to perform activities of daily living. We aim to implement a system for cognitive rehabilitation based on daily routines. For instance, we previously reported the benefits of cognitive rehabilitation through cooking. In this paper, we focus on cognitive rehabilitation based on cleaning tasks, which are as important as cooking for daily living. We tested the system with the help of a patient from the Osaka Prefecture Self-Reliance Center for People with Disabilities on November 9, 16 and 30, 2017. We propose and test a system that detects some cleaning tools and a human skeleton to assess the patients' behavior while cleaning, and rehabilitation is guided using indoor navigation. Object recognition allowed to integrate information about the cleaning tools to determine different cleaning behaviors. In addition, we tracked the patient to determine her state from a library of five states for the recognized skeleton in different cleaning areas. By detecting the cleaning tools and person's state, we estimated the cleaning behavior. As state estimation may retrieve a high rate of misidentification of the person's state, we implemented an error correction algorithm for estimating the cleaning behavior based on previous states, thus improving the system accuracy. We evaluated the university environment before testing the system, and determined the accuracy of both object recognition and person's state for estimating the cleaning behavior.

[1]  Varun Ramakrishna,et al.  Convolutional Pose Machines , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[2]  Shengrui Wang,et al.  A Frequent Pattern Mining Approach for ADLs Recognition in Smart Environments , 2011, 2011 IEEE International Conference on Advanced Information Networking and Applications.

[3]  Yaser Sheikh,et al.  OpenPose: Realtime Multi-Person 2D Pose Estimation Using Part Affinity Fields , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[4]  Fillia Makedon,et al.  Short-Term Recognition of Human Activities Using Convolutional Neural Networks , 2016, 2016 12th International Conference on Signal-Image Technology & Internet-Based Systems (SITIS).

[5]  Ali Farhadi,et al.  You Only Look Once: Unified, Real-Time Object Detection , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[6]  Amitava Chatterjee,et al.  Recognition of Human Behavior for Assisted Living Using Dictionary Learning Approach , 2018, IEEE Sensors Journal.

[7]  Wei Liu,et al.  SSD: Single Shot MultiBox Detector , 2015, ECCV.

[8]  Mutsuo Sano,et al.  Cooking Behavior Recognition Using Egocentric Vision for Cooking Navigation , 2017, J. Robotics Mechatronics.

[9]  Trevor Darrell,et al.  Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation , 2013, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[10]  大井 翔,et al.  Detect Cooking Action Based on Skeleton for Cognitive Rehabilitation , 2015 .