Towards Concurrent Q-Learning on Linked Multi-Component Robotic Systems

When conventional Q-Learning is applied to Multi-Component Robotic Systems (MCRS), increasing the number of components produces an exponential growth of state storage requirements. Modular approaches make the state size growth polynomial on the number of components, making more manageable its representation and manipulation. In this article, we give the first steps towards a modular Q-learning approach to learn the distributed control of a Linked MCRS, which is a specific type of MCRSs in which the individual robots are linked by a passive element. We have chosen a paradigmatic application of this kind of systems: a set of robots carrying the tip of a hose from some initial position to a desired goal. The hose dynamics is simplified to be a distance constraint on the robots positions.

[1]  Richard S. Sutton,et al.  Introduction to Reinforcement Learning , 1998 .

[2]  Ramón Moreno,et al.  Experiments on Robotic Multi-agent System for Hose Deployment and Transportation , 2010, PAAMS.

[3]  Bart De Schutter,et al.  A Comprehensive Survey of Multiagent Reinforcement Learning , 2008, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[4]  Manuel Graña,et al.  Hierarchically structured systems , 1986 .

[5]  Sean Luke,et al.  Cooperative Multi-Agent Learning: The State of the Art , 2005, Autonomous Agents and Multi-Agent Systems.

[6]  Ekaitz Zulueta,et al.  Linked Multicomponent Robotic Systems: Basic Assessment of Linking Element Dynamical Effect , 2010, HAIS.

[7]  Richard J. Duro,et al.  On the potential contributions of hybrid intelligent approaches to Multicomponent Robotic System development , 2010, Inf. Sci..

[8]  José Manuel Ferrández,et al.  Intelligent robotics and neuroscience , 2010, Robotics Auton. Syst..

[9]  Jonas Karlsson,et al.  Learning Multiple Goal Behavior via Task Decomposition and Dynamic Policy Merging , 1993 .

[10]  Sridhar Mahadevan,et al.  Robot Learning , 1993 .

[11]  Manuel Graña,et al.  Linked multi-component mobile robots: Modeling, simulation and control , 2010, Robotics Auton. Syst..

[12]  Javier de Lope,et al.  Hybridizing evolutionary computation and reinforcement learning for the design of almost universal controllers for autonomous robots , 2009 .

[13]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.