Task Learning Based on Reinforcement Learning in Virtual Environment

As a novel learning method, reinforced learning by which a robot acquires control rules through trial and error has gotten a lot of attention. However, it is quite difficult for robots to acquire control rules by reinforcement learning in real space because many learning trials are needed to achieve the control rules; the robot itself may lose control, or there may be safety problems with the control objects. In this paper, we propose a method in which a robot in real space learns a virtual task; then the task is transferred from virtual to real space. The robot eventually acquires the task in a real environment. We show that a real robot can acquire a task in virtual space with an input device by an example of an inverted pendulum. Next, we verify availability that the acquired task in virtual space can be applied to a real world task. We emphasize the utilization of virtual space to effectively obtain the real world task.

[1]  Richard S. Sutton,et al.  Neuronlike adaptive elements that can solve difficult learning control problems , 1983, IEEE Transactions on Systems, Man, and Cybernetics.

[2]  T. Flash,et al.  The coordination of arm movements: an experimentally confirmed mathematical model , 1985, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[3]  Jun Morimoto,et al.  A simple reinforcement learning algorithm for biped walking , 2004, IEEE International Conference on Robotics and Automation, 2004. Proceedings. ICRA '04. 2004.

[4]  Oliver Brock,et al.  Robotics and interactive simulation , 2002, CACM.

[5]  Richard S. Sutton,et al.  Reinforcement learning with replacing eligibility traces , 2004, Machine Learning.

[6]  Nikolaos G. Tsagarakis,et al.  Pneumatic muscle actuator technology: a light weight power system for a humanoid robot , 1998, Proceedings. 1998 IEEE International Conference on Robotics and Automation (Cat. No.98CH36146).

[7]  Tomoichi Takahashi,et al.  Robotic assembly operation teaching in a virtual environment , 1994, IEEE Trans. Robotics Autom..

[8]  Stefan Schaal,et al.  Robot Learning From Demonstration , 1997, ICML.

[9]  Kunikatsu Takase,et al.  Assembly motion teaching system using position/force simulator - extracting a sequence of contact state transition , 1995, Proceedings 1995 IEEE/RSJ International Conference on Intelligent Robots and Systems. Human Robot Interaction and Cooperative Robots.

[10]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[11]  Yoshiyuki Sankai,et al.  Virtual humanoid robot platform to develop controllers of real humanoid robots without porting , 2001, Proceedings 2001 IEEE/RSJ International Conference on Intelligent Robots and Systems. Expanding the Societal Role of Robotics in the the Next Millennium (Cat. No.01CH37180).

[12]  Toshi Takamori,et al.  Artificial tactile feel display using soft gel actuators , 2000, Proceedings 2000 ICRA. Millennium Conference. IEEE International Conference on Robotics and Automation. Symposia Proceedings (Cat. No.00CH37065).

[13]  Jun Morimoto,et al.  Conference on Intelligent Robots and Systems Reinforcement Le,arning of Dynamic Motor Sequence: Learning to Stand Up , 2022 .

[14]  Richard S. Sutton,et al.  Introduction to Reinforcement Learning , 1998 .