Assistive Gym: A Physics Simulation Framework for Assistive Robotics

Autonomous robots have the potential to serve as versatile caregivers that improve quality of life for millions of people worldwide. Yet, conducting research in this area presents numerous challenges, including the risks of physical interaction between people and robots. Physics simulations have been used to optimize and train robots for physical assistance, but have typically focused on a single task. In this paper, we present Assistive Gym, an open source physics simulation framework for assistive robots that models multiple tasks. It includes six simulated environments in which a robotic manipulator can attempt to assist a person with activities of daily living (ADLs): itch scratching, drinking, feeding, body manipulation, dressing, and bathing. Assistive Gym models a person’s physical capabilities and preferences for assistance, which are used to provide a reward function. We present baseline policies trained using reinforcement learning for four different commercial robots in the six environments. We demonstrate that modeling human motion results in better assistance and we compare the performance of different robots. Overall, we show that Assistive Gym is a promising tool for assistive robotics research.

[1]  Sergey Levine,et al.  Using Simulation and Domain Adaptation to Improve Efficiency of Deep Robotic Grasping , 2018, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[2]  Sergey Levine,et al.  Sim2Real View Invariant Visual Servoing by Recurrent Control , 2017 .

[3]  Johannes Fürnkranz,et al.  Preference-Based Monte Carlo Tree Search , 2018, KI.

[4]  Darwin G. Caldwell,et al.  Learning-based control strategy for safe human-robot interaction exploiting task and robot redundancies , 2010, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[5]  Alec Radford,et al.  Proximal Policy Optimization Algorithms , 2017, ArXiv.

[6]  Siddhartha S. Srinivasa,et al.  Legibility and predictability of robot motion , 2013, 2013 8th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[7]  Pradeep K. Khosla,et al.  Dexterity measures for design and control of manipulators , 1991, Proceedings IROS '91:IEEE/RSJ International Workshop on Intelligent Robots and Systems '91.

[8]  C. Karen Liu,et al.  What does the person feel? Learning to infer applied forces during robot-assisted dressing , 2017, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[9]  Manuela Veloso,et al.  Robot-driven Trajectory Improvement for Feeding Tasks , 2018, 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[10]  William D. Smart,et al.  Robots for humanity: using assistive robotics to empower people with disabilities , 2013, IEEE Robotics & Automation Magazine.

[11]  Atil Iscen,et al.  Sim-to-Real: Learning Agile Locomotion For Quadruped Robots , 2018, Robotics: Science and Systems.

[12]  Rachid Alami,et al.  Synthesizing Robot Motions Adapted to Human Presence , 2010 .

[13]  Yaser Sheikh,et al.  OpenPose: Realtime Multi-Person 2D Pose Estimation Using Part Affinity Fields , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[14]  Carme Torras,et al.  Joining High-Level Symbolic Planning with Low-Level Motion Primitives in Adaptive HRI: Application to Dressing Assistance , 2018, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[15]  Silvio Savarese,et al.  SURREAL: Open-Source Reinforcement Learning Framework and Robot Manipulation Benchmark , 2018, CoRL.

[16]  C. Karen Liu,et al.  Haptic simulation for robot-assisted dressing , 2017, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[17]  Kimitoshi Yamazaki,et al.  Bottom dressing by a life-sized humanoid robot provided failure detection and recovery functions , 2014, 2014 IEEE/SICE International Symposium on System Integration.

[18]  Alejandro Hernández Cordero,et al.  Extending the OpenAI Gym for robotics: a toolkit for reinforcement learning using ROS and Gazebo , 2016, ArXiv.

[19]  Yiannis Demiris,et al.  Iterative path optimisation for personalised dressing assistance using vision and force information , 2016, 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[20]  Greg Turk,et al.  Learning to Collaborate From Simulation for Robot-Assisted Dressing , 2019, IEEE Robotics and Automation Letters.

[21]  Jan Peters,et al.  Sample and Feedback Efficient Hierarchical Reinforcement Learning from Human Preferences , 2018, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[22]  Ali Farhadi,et al.  AI2-THOR: An Interactive 3D Environment for Visual AI , 2017, ArXiv.

[23]  Charles C. Kemp,et al.  A Multimodal Anomaly Detector for Robot-Assisted Feeding Using an LSTM-Based Variational Autoencoder , 2017, IEEE Robotics and Automation Letters.

[24]  Greg Chance,et al.  “Elbows Out”—Predictive Tracking of Partially Occluded Pose for Robot-Assisted Dressing , 2018, IEEE Robotics and Automation Letters.

[25]  Thilina Dulantha Lalitharatne,et al.  Meal assistance robots: A review on current status, challenges and future directions , 2015, 2015 IEEE/SICE International Symposium on System Integration (SII).

[26]  Wendy A. Rogers,et al.  Identifying the Potential for Robotics to Assist Older Adults in Different Living Environments , 2014, Int. J. Soc. Robotics.

[27]  Marcin Andrychowicz,et al.  Multi-Goal Reinforcement Learning: Challenging Robotics Environments and Request for Research , 2018, ArXiv.

[28]  Manuela M. Veloso,et al.  Personalized Assistance for Dressing Users , 2015, ICSR.

[29]  Shane Legg,et al.  Deep Reinforcement Learning from Human Preferences , 2017, NIPS.

[30]  Carme Torras,et al.  Personalization Framework for Adaptive Robotic Feeding Assistance , 2016, ICSR.

[31]  Hriday Bavle,et al.  A Deep Reinforcement Learning Strategy for UAV Autonomous Landing on a Moving Platform , 2018, Journal of Intelligent & Robotic Systems.

[32]  Pieter Abbeel,et al.  Benchmarking Deep Reinforcement Learning for Continuous Control , 2016, ICML.

[33]  Carme Torras,et al.  Adapting robot task planning to user preferences: an assistive shoe dressing example , 2019, Auton. Robots.

[34]  Michael J. Black,et al.  Pose-conditioned joint angle limits for 3D human pose reconstruction , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[35]  Wojciech Zaremba,et al.  OpenAI Gym , 2016, ArXiv.

[36]  Anca D. Dragan,et al.  Active Preference-Based Learning of Reward Functions , 2017, Robotics: Science and Systems.

[37]  Charles C. Kemp,et al.  Task-centric optimization of configurations for assistive robots , 2018, Autonomous Robots.

[38]  Javier García,et al.  A comprehensive survey on safe reinforcement learning , 2015, J. Mach. Learn. Res..

[39]  Carme Torras,et al.  A taxonomy of preferences for physically assistive robots , 2017, 2017 26th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN).

[40]  Charles C. Kemp,et al.  3D Human Pose Estimation on a Configurable Bed from a Pressure Image , 2018, 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[41]  Alberto Rodriguez,et al.  TossingBot: Learning to Throw Arbitrary Objects With Residual Physics , 2019, IEEE Transactions on Robotics.

[42]  Michèle Sebag,et al.  Preference-Based Policy Learning , 2011, ECML/PKDD.

[43]  Yuval Tassa,et al.  MuJoCo: A physics engine for model-based control , 2012, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[44]  Charles C. Kemp,et al.  The wouse: A wearable wince detector to stop assistive robots , 2012, IEEE International Symposium on Robot and Human Interactive Communication.

[45]  Charles A. Klein,et al.  Dexterity Measures for the Design and Control of Kinematically Redundant Manipulators , 1987 .

[46]  Siddhartha S. Srinivasa,et al.  DART: Dynamic Animation and Robotics Toolkit , 2018, J. Open Source Softw..

[47]  Fan Zhang,et al.  Probabilistic Real-Time User Posture Tracking for Personalized Robot-Assisted Dressing , 2019, IEEE Transactions on Robotics.

[48]  Jitendra Malik,et al.  Habitat: A Platform for Embodied AI Research , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[49]  Rachid Alami,et al.  Planning human-aware motions using a sampling-based costmap planner , 2011, 2011 IEEE International Conference on Robotics and Automation.

[50]  Chih-Hung King,et al.  Towards an assistive robot that autonomously performs bed baths for patient hygiene , 2010, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[51]  M. Lawton,et al.  Assessment of older people: self-maintaining and instrumental activities of daily living. , 1969, The Gerontologist.

[52]  Henry Dreyfuss,et al.  Measure of Man and Woman: Human Factors in Design , 1993 .

[53]  C. Karen Liu,et al.  Deep Haptic Model Predictive Control for Robot-Assisted Dressing , 2017, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[54]  Siddhartha S. Srinivasa,et al.  Human preferences for robot-human hand-over configurations , 2011, 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[55]  Johannes Fürnkranz,et al.  A Survey of Preference-Based Reinforcement Learning Methods , 2017, J. Mach. Learn. Res..

[56]  Rachid Alami,et al.  Human-aware robot navigation: A survey , 2013, Robotics Auton. Syst..

[57]  Wolfram Burgard,et al.  An autonomous robotic assistant for drinking , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[58]  Johannes Fürnkranz,et al.  Model-Free Preference-Based Reinforcement Learning , 2016, AAAI.

[59]  C. Karen Liu,et al.  Data-Driven Approach to Simulating Realistic Human Joint Constraints , 2018, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[60]  C. Karen Liu,et al.  Personalized collaborative plans for robot-assisted dressing via optimization and simulation , 2019, Autonomous Robots.

[61]  Sylvain Calinon,et al.  Learning adaptive dressing assistance from human demonstration , 2017, Robotics Auton. Syst..

[62]  Takamitsu Matsubara,et al.  Reinforcement learning of clothing assistance with a dual-arm robot , 2011, 2011 11th IEEE-RAS International Conference on Humanoid Robots.

[63]  Sergey Levine,et al.  Sim2Real View Invariant Visual Servoing by Recurrent Control , 2017, ArXiv.

[64]  C. Karen Liu,et al.  Data-driven haptic perception for robot-assisted dressing , 2016, 2016 25th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN).

[65]  Patricia C. Higginbottom,et al.  Americans with Disabilities , 2001 .

[66]  C. Karen Liu,et al.  Multidimensional Capacitive Sensing for Robot-Assisted Dressing and Bathing , 2019, 2019 IEEE 16th International Conference on Rehabilitation Robotics (ICORR).

[67]  Andrea Lockerd Thomaz,et al.  An Investigation of Responses to Robot-Initiated Touch in a Nursing Context , 2014, Int. J. Soc. Robotics.

[68]  Hokeun Kim,et al.  A multimodal execution monitor with anomaly classification for robot-assisted feeding , 2017, 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[69]  Maria Kyrarini,et al.  Autonomous Multi-Sensory Robotic Assistant for a Drinking Task , 2019, 2019 IEEE 16th International Conference on Rehabilitation Robotics (ICORR).

[70]  Siddhartha S. Srinivasa,et al.  Transfer Depends on Acquisition: Analyzing Manipulation Strategies for Robotic Feeding , 2019, 2019 14th ACM/IEEE International Conference on Human-Robot Interaction (HRI).