Geoffrey: An Automated Schedule System on a Social Robot for the Intellectually Challenged

The accelerated growth of the percentage of elder people and persons with brain injury-related conditions and who are intellectually challenged are some of the main concerns of the developed countries. These persons often require special cares and even almost permanent overseers that help them to carry out diary tasks. With this issue in mind, we propose an automated schedule system which is deployed on a social robot. The robot keeps track of the tasks that the patient has to fulfill in a diary basis. When a task is triggered, the robot guides the patient through its completion. The system is also able to detect if the steps are being properly carried out or not, issuing alerts in that case. To do so, an ensemble of deep learning techniques is used. The schedule is customizable by the carers and authorized relatives. Our system could enhance the quality of life of the patients and improve their self-autonomy. The experimentation, which was supervised by the ADACEA foundation, validates the achievement of these goals.

[1]  Geoffrey A. Hollinger,et al.  HERB: a home exploring robotic butler , 2010, Auton. Robots.

[2]  Fumihito Arai,et al.  In vitro patient-tailored anatomical model of cerebral artery for evaluating medical robots and systems for intravascular neurosurgery , 2005, 2005 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[3]  Huanhuan Zhang,et al.  A Home Nursing Robot System , 2011 .

[4]  Vicente Julián,et al.  PHAROS—PHysical Assistant RObot System , 2018, Sensors.

[5]  Gang Chen,et al.  A New Remote Health-Care System Based on Moving Robot Intended for the Elderly at Home , 2018, Journal of healthcare engineering.

[6]  J. Christopher Beck,et al.  An autonomous assistive robot for planning, scheduling and facilitating multi-user activities , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[7]  Yeh-Liang Hsu,et al.  Mobility Assistance Design of the Intelligent Robotic Wheelchair , 2012 .

[8]  Rory A Cooper,et al.  Intelligent walkers for the elderly: performance and safety testing of VA-PAMAID robotic walker. , 2003, Journal of rehabilitation research and development.

[9]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[10]  Albert Ali Salah,et al.  An autonomous robotic exercise tutor for elderly people , 2017, Auton. Robots.

[11]  Ross A. Knepper,et al.  Herb 2.0: Lessons Learned From Developing a Mobile Manipulator for the Home , 2012, Proceedings of the IEEE.

[12]  Fumihito Arai,et al.  Patient-specific neurovascular simulator for evaluating the performance of medical robots and instrumens , 2006, Proceedings 2006 IEEE International Conference on Robotics and Automation, 2006. ICRA 2006..

[13]  Yaser Sheikh,et al.  Hand Keypoint Detection in Single Images Using Multiview Bootstrapping , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[14]  Diego Reforgiato Recupero,et al.  Interacting with Dementia: The MARIO Approach , 2017, AAATE Conf..

[15]  Carlos Balaguer,et al.  Personal Autonomy Rehabilitation in Home Environments by a Portable Assistive Robot , 2012, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[16]  Gary R. Bradski,et al.  ORB: An efficient alternative to SIFT or SURF , 2011, 2011 International Conference on Computer Vision.

[17]  Maja J. Mataric,et al.  Robot exercise instructor: A socially assistive robot system to monitor and encourage physical exercise for the elderly , 2010, 19th International Symposium in Robot and Human Interactive Communication.

[18]  S. A. M. Gilani,et al.  Object Recognition by Modified Scale Invariant Feature Transform , 2008, 2008 Third International Workshop on Semantic Media Adaptation and Personalization.

[19]  Roger Orpwood,et al.  The Weston wheelchair mounted assistive robot - the design story , 2002, Robotica.

[20]  Pieter P. Jonker,et al.  Keypoint Extraction and Selection for Object Recognition , 2009, MVA.

[21]  Wolfram Burgard,et al.  Improved Techniques for Grid Mapping With Rao-Blackwellized Particle Filters , 2007, IEEE Transactions on Robotics.

[22]  John Hu,et al.  Giving patients a lift - the robotic nursing assistant (RoNA) , 2014, 2014 IEEE International Conference on Technologies for Practical Robot Applications (TePRA).

[23]  Michael S. Bernstein,et al.  ImageNet Large Scale Visual Recognition Challenge , 2014, International Journal of Computer Vision.

[24]  Sergey Ioffe,et al.  Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning , 2016, AAAI.

[25]  José García Rodríguez,et al.  Finding the Place: How to Train and Use Convolutional Neural Networks for a Dynamically Learning Robot , 2018, 2018 International Joint Conference on Neural Networks (IJCNN).

[26]  Oussama Khatib,et al.  ProVAR assistive robot system architecture , 1999, Proceedings 1999 IEEE International Conference on Robotics and Automation (Cat. No.99CH36288C).

[27]  Wolfram Burgard,et al.  Monte Carlo Localization: Efficient Position Estimation for Mobile Robots , 1999, AAAI/IAAI.

[28]  Joelle Pineau,et al.  Towards robotic assistants in nursing homes: Challenges and results , 2003, Robotics Auton. Syst..

[29]  Ali Farhadi,et al.  YOLOv3: An Incremental Improvement , 2018, ArXiv.

[30]  Illah R. Nourbakhsh,et al.  The Personal Rover Project: : The comprehensive design of a domestic personal robot , 2003, Robotics Auton. Syst..

[31]  Manuela M. Veloso,et al.  Setting Up Pepper For Autonomous Navigation And Personalized Interaction With Users , 2017, ArXiv.

[32]  Helen Petrie,et al.  PAM-AID. Personal Adaptive Mobility Aid for the Frail and Elderly Visually Impaired. D3.1. User Requirement Study , 1997 .

[33]  Martha E. Pollack,et al.  Autominder: an intelligent cognitive orthotic system for people with memory impairment , 2003, Robotics Auton. Syst..

[34]  Michael Blumenstein,et al.  What Effect Does an Animal Robot Called CuDDler Have on the Engagement and Emotional Response of Older People with Dementia? A Pilot Feasibility Study , 2016, Int. J. Soc. Robotics.

[35]  Varun Ramakrishna,et al.  Convolutional Pose Machines , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[36]  Yaser Sheikh,et al.  OpenPose: Realtime Multi-Person 2D Pose Estimation Using Part Affinity Fields , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[37]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).