Data-efficient learning of robotic clothing assistance using Bayesian Gaussian process latent variable model

ABSTRACT Motor-skill learning for complex robotic tasks is a challenging problem due to the high task variability. Robotic clothing assistance is one such challenging problem that can greatly improve the quality-of-life for the elderly and disabled. In this study, we propose a data-efficient representation to encode task-specific motor-skills of the robot using Bayesian nonparametric latent variable models. The effectivity of the proposed motor-skill representation is demonstrated in two ways: (1) through a real-time controller that can be used as a tool for learning from demonstration to impart novel skills to the robot and (2) by demonstrating that policy search reinforcement learning in such a task-specific latent space outperforms learning in the high-dimensional joint configuration space of the robot. We implement our proposed framework in a practical setting with a dual-arm robot performing clothing assistance tasks. GRAPHICAL ABSTRACT

[1]  Jan Peters,et al.  Reinforcement learning in robotics: A survey , 2013, Int. J. Robotics Res..

[2]  Winslow Burleson,et al.  Assistive Dressing System: A Capabilities Study for Personalized Support of Dressing Activities for People Living with Dementia , 2015 .

[3]  Shigeki Sugano,et al.  Repeatable Folding Task by Humanoid Robot Worker Using Deep Learning , 2017, IEEE Robotics and Automation Letters.

[4]  Heng Tao Shen,et al.  Principal Component Analysis , 2009, Encyclopedia of Biometrics.

[5]  Bernhard Schölkopf,et al.  Probabilistic movement modeling for intention inference in human–robot interaction , 2013, Int. J. Robotics Res..

[6]  Martin A. Riedmiller,et al.  Embed to Control: A Locally Linear Latent Dynamics Model for Control from Raw Images , 2015, NIPS.

[7]  C. Karen Liu,et al.  Data-driven haptic perception for robot-assisted dressing , 2016, 2016 25th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN).

[8]  Sandy H. Huang,et al.  Leveraging appearance priors in non-rigid registration, with application to manipulation of deformable objects , 2015, 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[9]  Nicholas Roy,et al.  SLAM using Incremental Probabilistic PCA and Dimensionality Reduction , 2005, Proceedings of the 2005 IEEE International Conference on Robotics and Automation.

[10]  Jan Peters,et al.  A Survey on Policy Search for Robotics , 2013, Found. Trends Robotics.

[11]  Nishanth Koganti,et al.  A study on human-robot collaboration for table-setting task , 2017, 2017 IEEE International Conference on Robotics and Biomimetics (ROBIO).

[12]  Kimitoshi Yamazaki,et al.  Bottom dressing by a life-sized humanoid robot provided failure detection and recovery functions , 2014, 2014 IEEE/SICE International Symposium on System Integration.

[13]  Yiannis Demiris,et al.  Iterative path optimisation for personalised dressing assistance using vision and force information , 2016, 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[14]  Carl E. Rasmussen,et al.  Gaussian Processes for Data-Efficient Learning in Robotics and Control , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[15]  Nishanth Koganti,et al.  A framework for robotic clothing assistance by imitation learning , 2019, Adv. Robotics.

[16]  Panos E. Trahanias,et al.  Learning from Demonstration facilitates Human-Robot Collaborative task execution , 2016, 2016 11th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[17]  Aude Billard,et al.  On Learning, Representing, and Generalizing a Task in a Humanoid Robot , 2007, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[18]  Jan Peters,et al.  Stable reinforcement learning with autoencoders for tactile and visual data , 2016, 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[19]  Nishanth Koganti,et al.  Bayesian Nonparametric Learning of Cloth Models for Real-Time State Estimation , 2017, IEEE Transactions on Robotics.

[20]  Matei T. Ciocarlie,et al.  Dimensionality reduction for hand-independent dexterous robotic grasping , 2007, 2007 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[21]  Stefano Carpin,et al.  Combining imitation and reinforcement learning to fold deformable planar objects , 2011, 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[22]  Dieter Fox,et al.  Learning GP-BayesFilters via Gaussian process latent variable models , 2009, Auton. Robots.

[23]  Huiru Zheng,et al.  A Multi-agent Approach to Assist with Dressing in a Smart Environment , 2016, eHealth 360°.

[24]  Takamitsu Matsubara,et al.  Reinforcement learning of clothing assistance with a dual-arm robot , 2011, 2011 11th IEEE-RAS International Conference on Humanoid Robots.

[25]  Manuela M. Veloso,et al.  Personalized Assistance for Dressing Users , 2015, ICSR.

[26]  Carme Torras,et al.  POMDP approach to robotized clothes separation , 2012, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[27]  C. Karen Liu,et al.  Animating human dressing , 2015, ACM Trans. Graph..

[28]  Carme Torras,et al.  A friction-model-based framework for Reinforcement Learning of robotic tasks in non-rigid environments , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[29]  C. Karen Liu,et al.  What does the person feel? Learning to infer applied forces during robot-assisted dressing , 2017, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[30]  Nishanth Koganti,et al.  Cloth dynamics modeling in latent spaces and its application to robotic clothing assistance , 2015, 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[31]  Jan Peters,et al.  Noname manuscript No. (will be inserted by the editor) Policy Search for Motor Primitives in Robotics , 2022 .

[32]  Tomohiro Shibata,et al.  Clothing Extremity Identification Using Convolutional Neural Network Regressor , 2018, 2018 Joint 7th International Conference on Informatics, Electronics & Vision (ICIEV) and 2018 2nd International Conference on Imaging, Vision & Pattern Recognition (icIVPR).

[33]  Christopher G. Atkeson,et al.  Constructive Incremental Learning from Only Local Information , 1998, Neural Computation.

[34]  F. Wilcoxon.  Individual Comparisons by Ranking Methods , 1945 .

[35]  C. Karen Liu,et al.  Haptic simulation for robot-assisted dressing , 2017, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[36]  Panagiotis K. Artemiadis,et al.  EMG-Based Control of a Robot Arm Using Low-Dimensional Embeddings , 2010, IEEE Transactions on Robotics.

[37]  Sethu Vijayakumar,et al.  Using dimensionality reduction to exploit constraints in reinforcement learning , 2010, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[38]  Neil D. Lawrence,et al.  Bayesian Gaussian Process Latent Variable Model , 2010, AISTATS.

[39]  Jun Nakanishi,et al.  Dynamical Movement Primitives: Learning Attractor Models for Motor Behaviors , 2013, Neural Computation.

[40]  Rajesh P. N. Rao,et al.  Learning Shared Latent Structure for Image Synthesis and Robotic Imitation , 2005, NIPS.

[41]  Neil D. Lawrence,et al.  Probabilistic Non-linear Principal Component Analysis with Gaussian Process Latent Variable Models , 2005, J. Mach. Learn. Res..

[42]  Carl E. Rasmussen,et al.  Gaussian processes for machine learning , 2005, Adaptive computation and machine learning.

[43]  Mitsuo Kawato,et al.  A theory for cursive handwriting based on the minimization principle , 1995, Biological Cybernetics.

[44]  Lynne E. Parker,et al.  Feature Space Decomposition for effective robot adaptation , 2015, 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[45]  Neil D. Lawrence,et al.  Variational Inference for Latent Variables and Uncertain Inputs in Gaussian Processes , 2016, J. Mach. Learn. Res..

[46]  Jan Peters,et al.  Latent space policy search for robotics , 2014, 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[47]  Sergey Levine,et al.  Learning force-based manipulation of deformable objects from multiple demonstrations , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[48]  Tae-Kyun Kim,et al.  Autonomous active recognition and unfolding of clothes using random decision forests and probabilistic planning , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[49]  Jia Pan,et al.  Three-Dimensional Deformable Object Manipulation Using Fast Online Gaussian Process Regression , 2017, IEEE Robotics and Automation Letters.

[50]  Masahide Kaneko,et al.  Segmenting Continuous Motions with Hidden Semi-markov Models and Gaussian Processes , 2017, Front. Neurorobot..