Imitation Learning for Human-robot Cooperation Using Bilateral Control

Robots are required to operate autonomously in response to changing situations. Previously, imitation learning using 4ch-bilateral control was demonstrated to be suitable for imitation of object manipulation. However, cooperative work between humans and robots has not yet been verified in these studies. In this study, the task was expanded by cooperative work between a human and a robot. 4ch-bilateral control was used to collect training data for training robot motion. We focused on serving salad as a task in the home. The task was executed with a spoon and a fork fixed to robots. Adjustment of force was indispensable in manipulating indefinitely shaped objects such as salad. Results confirmed the effectiveness of the proposed method as demonstrated by the success of the task.

[1]  Toshiaki Tsuji,et al.  Estimation and Kinetic Modeling of Human Arm using Wearable Robot Arm , 2017 .

[2]  Kouhei Ohnishi,et al.  Multi-DOF Micro-Macro Bilateral Controller Using Oblique Coordinate Control , 2011, IEEE Transactions on Industrial Informatics.

[3]  Claudio Pacchierotti,et al.  Cutaneous haptic feedback to ensure the stability of robotic teleoperation systems , 2015, Int. J. Robotics Res..

[4]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[5]  Kouhei Ohnishi,et al.  Motion control for advanced mechatronics , 1996 .

[6]  Sergey Levine,et al.  One-Shot Imitation from Observing Humans via Domain-Adaptive Meta-Learning , 2018, Robotics: Science and Systems.

[7]  Carme Torras,et al.  Learning Physical Collaborative Robot Behaviors From Human Demonstrations , 2016, IEEE Transactions on Robotics.

[8]  Sergey Levine,et al.  Learning hand-eye coordination for robotic grasping with deep learning and large-scale data collection , 2016, Int. J. Robotics Res..

[9]  Oliver Brock,et al.  Lessons from the Amazon Picking Challenge: Four Aspects of Building Robotic Systems , 2016, Robotics: Science and Systems.

[10]  Pieter Abbeel,et al.  An Algorithmic Perspective on Imitation Learning , 2018, Found. Trends Robotics.

[11]  Shigeki Sugano,et al.  Repeatable Folding Task by Humanoid Robot Worker Using Deep Learning , 2017, IEEE Robotics and Automation Letters.

[12]  D. Floreano,et al.  Soft Robotic Grippers , 2018, Advanced materials.

[13]  Sergey Levine,et al.  Learning force-based manipulation of deformable objects from multiple demonstrations , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[14]  Toshiyuki Murakami,et al.  Torque sensorless control in multidegree-of-freedom manipulator , 1993, IEEE Trans. Ind. Electron..

[15]  Panfeng Huang,et al.  Convolutional multi-grasp detection using grasp path for RGBD images , 2019, Robotics Auton. Syst..

[16]  Kouhei Ohnishi,et al.  A Novel Motion Equation for General Task Description and Analysis of Mobile-Hapto , 2013, IEEE Transactions on Industrial Electronics.

[17]  Rouhollah Rahmatizadeh,et al.  Vision-Based Multi-Task Manipulation for Inexpensive Robots Using End-to-End Learning from Demonstration , 2017, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[18]  Tsuyoshi Adachi,et al.  Imitation Learning for Object Manipulation Based on Position/Force Information Using Bilateral Control , 2018, 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[19]  Carme Torras,et al.  A robot learning from demonstration framework to perform force-based manipulation tasks , 2013, Intelligent Service Robotics.

[20]  Darwin G. Caldwell,et al.  Imitation Learning of Positional and Force Skills Demonstrated via Kinesthetic Teaching and Haptic Input , 2011, Adv. Robotics.

[21]  Sylvain Calinon,et al.  Improving dual-arm assembly by master-slave compliance , 2019, 2019 International Conference on Robotics and Automation (ICRA).

[22]  Ken Goldberg,et al.  Deep Imitation Learning for Complex Manipulation Tasks from Virtual Reality Teleoperation , 2017, ICRA.

[23]  Toshiaki Tsuji,et al.  Bilateral control using functional electrical stimulation , 2015, IECON 2015 - 41st Annual Conference of the IEEE Industrial Electronics Society.

[24]  Toshiaki Tsuji,et al.  Time Series Motion Generation Considering Long Short-Term Motion , 2019, 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[25]  Blake Hannaford,et al.  Improving control precision and motion adaptiveness for surgical robot with recurrent neural network , 2017, 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[26]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[27]  Oliver Brock,et al.  Lessons from the Amazon Picking Challenge: Four Aspects of Building Robotic Systems , 2016, IJCAI.

[28]  Geoffrey E. Hinton,et al.  Speech recognition with deep recurrent neural networks , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[29]  Kiyoshi Ohishi,et al.  Stability Analysis and Experimental Validation of a Motion-Copying System , 2009, IEEE Transactions on Industrial Electronics.

[30]  Jan Peters,et al.  Learning multiple collaborative tasks with a mixture of Interaction Primitives , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[31]  Joseph Redmon,et al.  Real-time grasp detection using convolutional neural networks , 2014, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[32]  Yoshua Bengio,et al.  Show, Attend and Tell: Neural Image Caption Generation with Visual Attention , 2015, ICML.