SkyAI: Highly modularized reinforcement learning library

This paper introduces a software library of reinforcement learning (RL) methods, named SkyAI. SkyAI is a highly modularized RL library for real/simulated robots to learn behaviors. Our ultimate goal is to develop an artificial intelligence (AI) program with which the robots can learn to behave as their users' wish. In this paper, we describe the concepts, the requirements, and the current implementation of SkyAI. SkyAI provides two conflicting features: high execution-speed enough for real robot systems and high flexibility to design learning systems. We also demonstrate the applications to crawling tasks of both a humanoid robot in simulation and a real spider robot.

[1]  Leemon C Baird,et al.  Reinforcement Learning With High-Dimensional, Continuous Actions , 1993 .

[2]  Stefan Schaal,et al.  Reinforcement Learning for Humanoid Robotics , 2003 .

[3]  Jing Peng,et al.  Incremental multi-step Q-learning , 1994, Machine Learning.

[4]  Mahesan Niranjan,et al.  On-line Q-learning using connectionist systems , 1994 .

[5]  Shahar Cohen,et al.  Reinforcement Learning with Hierarchical Decision-Making , 2006, Sixth International Conference on Intelligent Systems Design and Applications.

[6]  Jun Morimoto,et al.  Acquisition of stand-up behavior by a real robot using hierarchical reinforcement learning , 2000, Robotics Auton. Syst..

[7]  Brian Tanner,et al.  RL-Glue: Language-Independent Software for Reinforcement-Learning Experiments , 2009, J. Mach. Learn. Res..

[8]  Jan Peters,et al.  Learning motor primitives for robotics , 2009, 2009 IEEE International Conference on Robotics and Automation.

[9]  Shin Ishii,et al.  On-line EM Algorithm for the Normalized Gaussian Network , 2000, Neural Computation.

[10]  Tsukasa Ogasawara,et al.  Constructing action set from basis functions for reinforcement learning of robot control , 2009, 2009 IEEE International Conference on Robotics and Automation.

[11]  Csaba Szepesvári,et al.  Model-based and Model-free Reinforcement Learning for Visual Servoing , 2009, 2009 IEEE International Conference on Robotics and Automation.

[12]  Takashi Suehiro,et al.  A Software Platform for Component Based RT-System Development: OpenRTM-Aist , 2008, SIMPAR.

[13]  Pierre Geurts,et al.  Tree-Based Batch Mode Reinforcement Learning , 2005, J. Mach. Learn. Res..

[14]  Minoru Asada,et al.  Multi-layered learning systems for vision-based behavior acquisition of a real mobile robot , 2003, SICE 2003 Annual Conference (IEEE Cat. No.03TH8734).

[15]  Jun Morimoto,et al.  Low-dimensional feature extraction for humanoid locomotion using kernel dimension reduction , 2008, 2008 IEEE International Conference on Robotics and Automation.

[16]  Jianwei Zhang,et al.  Self-valuing learning and generalization with application in visually guided grasping of complex objects , 2004, Robotics Auton. Syst..