Personalized robot-assisted dressing using user modeling in latent spaces

Robots have the potential to provide tremendous support to disabled and elderly people in their everyday tasks, such as dressing. Many recent studies on robotic dressing assistance usually view dressing as a trajectory planning problem. However, the user movements during the dressing process are rarely taken into account, which often leads to the failures of the planned trajectory and may put the user at risk. The main difficulty of taking user movements into account is caused by severe occlusions created by the robot, the user, and the clothes during the dressing process, which prevent vision sensors from accurately detecting the postures of the user in real time. In this paper, we address this problem by introducing an approach that allows the robot to automatically adapt its motion according to the force applied on the robot's gripper caused by user movements. There are two main contributions introduced in this paper: 1) the use of a hierarchical multi-task control strategy to automatically adapt the robot motion and minimize the force applied between the user and the robot caused by user movements; 2) the online update of the dressing trajectory based on the user movement limitations modeled with the Gaussian Process Latent Variable Model in a latent space, and the density information extracted from such latent space. The combination of these two contributions leads to a personalized dressing assistance that can cope with unpredicted user movements during the dressing while constantly minimizing the force that the robot may apply on the user. The experimental results demonstrate that the proposed method allows the Baxter humanoid robot to provide personalized dressing assistance for human users with simulated upper-body impairments.

[1]  David J. Fleet,et al.  Gaussian Process Dynamical Models , 2005, NIPS.

[2]  Jean-Christophe Nebel,et al.  Generalized Laplacian Eigenmaps for Modeling and Tracking Human Motions , 2014, IEEE Transactions on Cybernetics.

[3]  Matthew T. Mason,et al.  Compliance and Force Control for Computer Controlled Manipulators , 1981, IEEE Transactions on Systems, Man, and Cybernetics.

[4]  Takamitsu Matsubara,et al.  Reinforcement learning of clothing assistance with a dual-arm robot , 2011, 2011 11th IEEE-RAS International Conference on Humanoid Robots.

[5]  Giuseppe Oriolo,et al.  Kinematically Redundant Manipulators , 2008, Springer Handbook of Robotics.

[6]  Takamitsu Matsubara,et al.  Reinforcement learning of a motor skill for wearing a T-shirt using topology coordinates , 2013, Adv. Robotics.

[7]  James L. Patton,et al.  Evaluation of force field training customized according to individual movement deficit patterns , 2015, 2015 IEEE International Conference on Rehabilitation Robotics (ICORR).

[8]  J. Patton,et al.  Movement distributions of stroke survivors exhibit distinct patterns that evolve with training , 2016, Journal of NeuroEngineering and Rehabilitation.

[9]  Takamitsu Matsubara,et al.  Estimation of Human Cloth Topological Relationship using Depth Sensor for Robotic Clothing Assistance , 2013, AIR '13.

[10]  John T. Wen,et al.  Collaborative human-robot manipulation of highly deformable materials , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[11]  Lars-Peter Ellekilde,et al.  Kernel density estimation based self-learning sampling strategy for motion planning of repetitive tasks , 2016, 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[12]  Yiannis Demiris,et al.  Iterative path optimisation for personalised dressing assistance using vision and force information , 2016, 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[13]  Bernhard Schölkopf,et al.  Kernel Principal Component Analysis , 1997, ICANN.

[14]  Yiannis Demiris,et al.  User modelling for personalised dressing assistance by humanoid robots , 2015, 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[15]  Neil D. Lawrence,et al.  Gaussian Process Latent Variable Models for Visualisation of High Dimensional Data , 2003, NIPS.

[16]  Nishanth Koganti,et al.  Bayesian Nonparametric Learning of Cloth Models for Real-Time State Estimation , 2017, IEEE Transactions on Robotics.

[17]  Greg Chance,et al.  An assistive robot to support dressing - strategies for planning and error handling , 2016, 2016 6th IEEE International Conference on Biomedical Robotics and Biomechatronics (BioRob).

[18]  Nishanth Koganti,et al.  Cloth dynamics modeling in latent spaces and its application to robotic clothing assistance , 2015, 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[19]  David J. Fleet,et al.  This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE Gaussian Process Dynamical Model , 2007 .

[20]  C. Karen Liu,et al.  Data-driven haptic perception for robot-assisted dressing , 2016, 2016 25th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN).

[21]  Carme Torras,et al.  A friction-model-based framework for Reinforcement Learning of robotic tasks in non-rigid environments , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).