Active Robot-Assisted Feeding with a General-Purpose Mobile Manipulator: Design, Evaluation, and Lessons Learned.

Eating is an essential activity of daily living (ADL) for staying healthy and living at home independently. Although numerous assistive devices have been introduced, many people with disabilities are still restricted from independent eating due to the devices' physical or perceptual limitations. In this work, we present a new meal-assistance system and evaluations of this system with people with motor impairments. We also discuss learned lessons and design insights based on the evaluations. The meal-assistance system uses a general-purpose mobile manipulator, a Willow Garage PR2, which has the potential to serve as a versatile form of assistive technology. Our active feeding framework enables the robot to autonomously deliver food to the user's mouth, reducing the need for head movement by the user. The user interface, visually-guided behaviors, and safety tools allow people with severe motor impairments to successfully use the system. We evaluated our system with a total of 10 able-bodied participants and 9 participants with motor impairments. Both groups of participants successfully ate various foods using the system and reported high rates of success for the system's autonomous behaviors. In general, participants who operated the system reported that it was comfortable, safe, and easy-to-use.

[1]  Charles C. Kemp,et al.  In-home and remote use of robotic body surrogates by people with profound motor deficits , 2019, PloS one.

[2]  A C Phalangas,et al.  Powered feeding devices: an evaluation of three models. , 1999, Archives of physical medicine and rehabilitation.

[3]  Hokeun Kim,et al.  Multimodal anomaly detection for assistive robots , 2019, Auton. Robots.

[4]  Matei T. Ciocarlie,et al.  Mobile manipulation through an assistive home robot , 2012, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[5]  Charles C. Kemp,et al.  Multimodal execution monitoring for anomaly detection during robot manipulation , 2016, 2016 IEEE International Conference on Robotics and Automation (ICRA).

[6]  Noriyuki Tejima Rehabilitation Manipulator for Eating , 1996 .

[7]  Dae-Jin Kim,et al.  System Design and Implementation of UCF-MANUS—An Intelligent Assistive Robotic Manipulator , 2014, IEEE/ASME Transactions on Mechatronics.

[8]  Oussama Khatib,et al.  ProVAR assistive robot system architecture , 1999, Proceedings 1999 IEEE International Conference on Robotics and Automation (Cat. No.99CH36288C).

[9]  Matei T. Ciocarlie,et al.  ROS commander (ROSCo): Behavior creation for home robots , 2013, 2013 IEEE International Conference on Robotics and Automation.

[10]  Siddhartha S. Srinivasa,et al.  Shared autonomy via hindsight optimization for teleoperation and teaming , 2017, Int. J. Robotics Res..

[11]  Wolfram Burgard,et al.  An autonomous robotic assistant for drinking , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[12]  D. Feil-Seifer,et al.  Defining socially assistive robotics , 2005, 9th International Conference on Rehabilitation Robotics, 2005. ICORR 2005..

[13]  John A. Redmond,et al.  A noninvasive handheld assistive device to accommodate essential tremor: A pilot study , 2014, Movement disorders : official journal of the Movement Disorder Society.

[14]  Woo-Keun Yoon,et al.  User evaluation to apply the robotic arm RAPUDA for an upper-limb disabilities Patient's Daily Life , 2012, 2012 IEEE International Conference on Robotics and Biomimetics (ROBIO).

[15]  Michael Topping,et al.  An Overview of the Development of Handy 1, a Rehabilitation Robot to Assist the Severely Disabled , 2002, J. Intell. Robotic Syst..

[16]  Josephine Sullivan,et al.  One millisecond face alignment with an ensemble of regression trees , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[17]  Charles C. Kemp,et al.  Towards Assistive Feeding with a General-Purpose Mobile Manipulator , 2016, ArXiv.

[18]  Roger Orpwood,et al.  A WHEELCHAIR MOUNTED ASSISTIVE ROBOT , 1999 .

[19]  Advait Jain,et al.  Reaching in clutter with whole-arm tactile sensing , 2013, Int. J. Robotics Res..

[20]  S. Hart,et al.  Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research , 1988 .

[21]  Ginés García-Mateos,et al.  Estimating 3D facial pose in video with just three points , 2008, 2008 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[22]  Y. Takahashi,et al.  Easy Human Interface for Severely Handicapped Persons and Application for Eating Assist Robot , 2006, 2006 IEEE International Conference on Mechatronics.

[23]  Morgan Quigley,et al.  ROS: an open-source Robot Operating System , 2009, ICRA 2009.

[24]  Advait Jain,et al.  EL-E: an assistive mobile manipulator that autonomously fetches objects from flat surfaces , 2010, Auton. Robots.

[25]  Francois Routhier,et al.  Evaluation of the JACO robotic arm: Clinico-economic study for powered wheelchair users with upper-extremity disabilities , 2011, 2011 IEEE International Conference on Rehabilitation Robotics.

[26]  Odest Chadwicke Jenkins,et al.  Rosbridge: ROS for Non-ROS Users , 2011, ISRR.

[27]  Charles C. Kemp,et al.  Tracking Human Pose During Robot-Assisted Dressing Using Single-Axis Capacitive Proximity Sensing , 2017, IEEE Robotics and Automation Letters.

[28]  Chih-Hung King,et al.  Assistive mobile manipulation for self-care tasks around the head , 2014, 2014 IEEE Symposium on Computational Intelligence in Robotic Rehabilitation and Assistive Technologies (CIR2AT).

[29]  Won-Kyung Song,et al.  Novel Assistive Robot for Self-Feeding , 2012 .

[30]  Charles C. Kemp,et al.  A Multimodal Anomaly Detector for Robot-Assisted Feeding Using an LSTM-Based Variational Autoencoder , 2017, IEEE Robotics and Automation Letters.

[31]  Thilina Dulantha Lalitharatne,et al.  SSVEP based BMI for a meal assistance robot , 2016, 2016 IEEE International Conference on Systems, Man, and Cybernetics (SMC).

[32]  Manuela Veloso,et al.  Robot-driven Trajectory Improvement for Feeding Tasks , 2018, 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[33]  William D. Smart,et al.  Robots for humanity: using assistive robotics to empower people with disabilities , 2013, IEEE Robotics & Automation Magazine.

[34]  J. Wiener,et al.  Measuring the activities of daily living: comparisons across national surveys. , 1990, Journal of gerontology.

[35]  L. Leifer,et al.  Clinical evaluation of a desktop robotic assistant. , 1989, Journal of rehabilitation research and development.

[36]  Sven Behnke,et al.  NimbRo Rescue: Solving Disaster‐response Tasks with the Mobile Manipulation Robot Momaro , 2017, J. Field Robotics.

[37]  Kazuo Kiguchi,et al.  EEG-controlled meal assistance robot with camera-based automatic mouth position tracking and mouth open detection , 2017, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[38]  R. Dillmann,et al.  Development of an anthropomorphic hand for a mobile assistive robot , 2005, 9th International Conference on Rehabilitation Robotics, 2005. ICORR 2005..

[39]  Kimitoshi Yamazaki,et al.  Bottom dressing by a life-sized humanoid robot provided failure detection and recovery functions , 2014, 2014 IEEE/SICE International Symposium on System Integration.

[40]  Christophe Leroux,et al.  SAM, an Assistive Robotic Device Dedicated to Helping Persons with Quadriplegia: Usability Study , 2019, Int. J. Soc. Robotics.

[41]  Manuela M. Veloso,et al.  Personalized Assistance for Dressing Users , 2015, ICSR.

[42]  Carme Torras,et al.  Personalization Framework for Adaptive Robotic Feeding Assistance , 2016, ICSR.

[43]  Wendy A Rogers,et al.  Closing the Capacity-Ability Gap: Using Technology to Support Aging With Disability , 2018, Innovation in aging.

[44]  S. Leonhardt,et al.  A survey on robotic devices for upper limb rehabilitation , 2014, Journal of NeuroEngineering and Rehabilitation.

[45]  Yunyi Jia,et al.  Design of Robotic Human Assistance Systems Using a Mobile Manipulator , 2012 .

[46]  Atsushi Yamashita,et al.  Meal support system with spoon using laser range finder and manipulator , 2013, 2013 IEEE Workshop on Robot Vision (WORV).

[47]  Allen R. Hanson,et al.  Mobile manipulators for assisted living in residential settings , 2008, Auton. Robots.

[48]  Jörn Malzahn,et al.  WALK‐MAN: A High‐Performance Humanoid Platform for Realistic Environments , 2017, J. Field Robotics.

[49]  João Paulo Costeira,et al.  Vision Augmented Robot Feeding , 2018, ECCV Workshops.

[50]  Akira Yamazaki,et al.  Autonomous Foods Handling by Chopsticks for Meal Assistant Robot , 2012, ROBOTIK.

[51]  Hokeun Kim,et al.  A multimodal execution monitor with anomaly classification for robot-assisted feeding , 2017, 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[52]  Thilina Dulantha Lalitharatne,et al.  Meal assistance robots: A review on current status, challenges and future directions , 2015, 2015 IEEE/SICE International Symposium on System Integration (SII).

[53]  Carlos Balaguer,et al.  A portable light-weight climbing robot for personal assistance applications , 2006, Ind. Robot.

[54]  Alex Mihailidis,et al.  A Survey on Ambient-Assisted Living Tools for Older Adults , 2013, IEEE Journal of Biomedical and Health Informatics.

[55]  Davis E. King,et al.  Dlib-ml: A Machine Learning Toolkit , 2009, J. Mach. Learn. Res..

[56]  Martin Hägele,et al.  Robotic home assistant Care-O-bot® 3 - product vision and innovation platform , 2009, 2009 IEEE Workshop on Advanced Robotics and its Social Impacts.

[57]  Manfred Tscheligi,et al.  The USUS Evaluation Framework for Human-Robot Interaction , 2009 .

[58]  Charles C. Kemp,et al.  Task-centric selection of robot and environment initial configurations for assistive tasks , 2015, 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).