Motion Generation Using Bilateral Control-Based Imitation Learning With Autoregressive Learning

Imitation learning has been studied as an efficient and high-performance method to generate robot motion. Specifically, bilateral control-based imitation learning has been proposed as a method of realizing fast motion. However, the learning approach of this method leads to the accumulation of prediction errors during the prediction process and may not generate desirable long-term behavior. Therefore, in this paper, we propose a method of autoregressive learning for bilateral control-based imitation learning to reduce the accumulation of prediction errors. A new neural network model for implementing autoregressive learning is also proposed. Three types of experiments are conducted to verify the effectiveness of the proposed method, where the method is shown to have improved performance over those of conventional approaches. Due to the structure and method of autoregressive learning employed by the developed model, the proposed method can generate desirable long-term motion for successful tasks and has a high generalization ability for environmental changes based on the human demonstrations of tasks.

[1]  Jitendra Malik,et al.  Learning to Poke by Poking: Experiential Learning of Intuitive Physics , 2016, NIPS.

[2]  Toshiaki Tsuji,et al.  Bilateral Control in the Vertical Direction Using Functional Electrical Stimulation , 2016 .

[3]  Sergey Levine,et al.  One-Shot Imitation from Observing Humans via Domain-Adaptive Meta-Learning , 2018, Robotics: Science and Systems.

[4]  C. L. Philip Chen,et al.  Intelligent Decision Making and Bionic Movement Control of Self-Organized Swarm , 2021, IEEE Transactions on Industrial Electronics.

[5]  Chenguang Yang,et al.  Motor Learning and Generalization Using Broad Learning Adaptive Neural Control , 2020, IEEE Transactions on Industrial Electronics.

[6]  Hamid Reza Karimi,et al.  Improved recurrent neural network-based manipulator control with remote center of motion constraints: Experimental results , 2020, Neural Networks.

[7]  K. Ohnishi,et al.  Reproducibility and operationality in bilateral teleoperation , 2004, The 8th IEEE International Workshop on Advanced Motion Control, 2004. AMC '04..

[8]  Kouhei Ohnishi,et al.  Motion control for advanced mechatronics , 1996 .

[9]  Alberto Montebelli,et al.  Incrementally assisted kinesthetic teaching for programming by demonstration , 2016, 2016 11th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[10]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[11]  Darwin G. Caldwell,et al.  Upper-body kinesthetic teaching of a free-standing humanoid robot , 2011, 2011 IEEE International Conference on Robotics and Automation.

[12]  Hang Su,et al.  A Smartphone-Based Adaptive Recognition and Real-Time Monitoring System for Human Activities , 2020, IEEE Transactions on Human-Machine Systems.

[13]  Toshiaki Tsuji,et al.  Imitation Learning Based on Bilateral Control for Human–Robot Cooperation , 2020, IEEE Robotics and Automation Letters.

[14]  Samy Bengio,et al.  Scheduled Sampling for Sequence Prediction with Recurrent Neural Networks , 2015, NIPS.

[15]  Yoshua Bengio,et al.  Professor Forcing: A New Algorithm for Training Recurrent Networks , 2016, NIPS.

[16]  Ronald J. Williams,et al.  A Learning Algorithm for Continually Running Fully Recurrent Neural Networks , 1989, Neural Computation.

[17]  Sergey Levine,et al.  Learning hand-eye coordination for robotic grasping with deep learning and large-scale data collection , 2016, Int. J. Robotics Res..

[18]  Sergey Levine,et al.  Learning force-based manipulation of deformable objects from multiple demonstrations , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[19]  Sergey Levine,et al.  Relay Policy Learning: Solving Long-Horizon Tasks via Imitation and Reinforcement Learning , 2019, CoRL.

[20]  Darwin G. Caldwell,et al.  Imitation Learning of Positional and Force Skills Demonstrated via Kinesthetic Teaching and Haptic Input , 2011, Adv. Robotics.

[21]  Hang Su,et al.  An Incremental Learning Framework for Human-Like Redundancy Optimization of Anthropomorphic Manipulators , 2020, IEEE Transactions on Industrial Informatics.

[22]  Toshiaki Tsuji,et al.  Estimation and Kinetic Modeling of Human Arm using Wearable Robot Arm , 2017 .

[23]  Shigeki Sugano,et al.  Repeatable Folding Task by Humanoid Robot Worker Using Deep Learning , 2017, IEEE Robotics and Automation Letters.

[24]  Zheng Chen,et al.  Adaptive Fuzzy Backstepping Control for Stable Nonlinear Bilateral Teleoperation Manipulators With Enhanced Transparency Performance , 2020, IEEE Transactions on Industrial Electronics.

[25]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[26]  Darwin G. Caldwell,et al.  Learning optimal controllers in human-robot cooperative transportation tasks with position and force constraints , 2015, 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[27]  Yifan Xu,et al.  Rethinking Exposure Bias In Language Modeling , 2019, ArXiv.

[28]  Toshiyuki Murakami,et al.  Control Structure Determination of Bilateral System based on Reproducibility and Operationality , 2019 .

[29]  Guoying Zhao,et al.  Cross-Database Micro-Expression Recognition: A Benchmark , 2018, IEEE Transactions on Knowledge and Data Engineering.

[30]  Heni Ben Amor,et al.  A system for learning continuous human-robot interactions from human-human demonstrations , 2017, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[31]  Yang Feng,et al.  Bridging the Gap between Training and Inference for Neural Machine Translation , 2019, ACL.

[32]  Tsuyoshi Adachi,et al.  Imitation Learning for Object Manipulation Based on Position/Force Information Using Bilateral Control , 2018, 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[33]  Marc'Aurelio Ranzato,et al.  Sequence Level Training with Recurrent Neural Networks , 2015, ICLR.

[34]  Kouhei Ohnishi,et al.  Multi-DOF Micro-Macro Bilateral Controller Using Oblique Coordinate Control , 2011, IEEE Transactions on Industrial Informatics.

[35]  Fuchun Sun,et al.  Survey of imitation learning for robotic manipulation , 2019, International Journal of Intelligent Robotics and Applications.

[36]  Ken Goldberg,et al.  Deep Imitation Learning for Complex Manipulation Tasks from Virtual Reality Teleoperation , 2017, ICRA.

[37]  Rouhollah Rahmatizadeh,et al.  Vision-Based Multi-Task Manipulation for Inexpensive Robots Using End-to-End Learning from Demonstration , 2017, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[38]  Shuyuan Yang,et al.  A Survey of Deep Learning-Based Object Detection , 2019, IEEE Access.

[39]  Carme Torras,et al.  A robot learning from demonstration framework to perform force-based manipulation tasks , 2013, Intelligent Service Robotics.

[40]  Toshiyuki Murakami,et al.  Torque sensorless control in multidegree-of-freedom manipulator , 1993, IEEE Trans. Ind. Electron..

[41]  Kensuke Harada,et al.  Deep Learning Scooping Motion Using Bilateral Teleoperations , 2018, 2018 3rd International Conference on Advanced Robotics and Mechatronics (ICARM).

[42]  Sonia Chernova,et al.  Recent Advances in Robot Learning from Demonstration , 2020, Annu. Rev. Control. Robotics Auton. Syst..

[43]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[44]  Tetsuya Ogata,et al.  Learning Multiple Sensorimotor Units to Complete Compound Tasks using an RNN with Multiple Attractors , 2019, 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).