A homogeneous mobile robot team that is fault-tolerant

Abstract This paper introduces a design methodology of a fault-tolerant autonomous multi-robot system (MRS). An important fundamental topic for this type of system is the design of an on-line autonomous behavior acquisition mechanism that is capable of developing cooperative roles as well as assigning them to a robot appropriately in a noisy embedded environment. Our approach is to apply reinforcement learning that adopts the Bayesian discrimination method for segmenting a continuous state space and a continuous action space simultaneously. In addition, a neural network is provided for predicting the average of the other robots’ postures at the next time step in order to stabilize the reinforcement-learning environment. Computer simulations are conducted to illustrate the fault-tolerance of our MRS against a system change that occurs after the MRS achieves stable behavior.

[1]  András Lörincz,et al.  Event-learning and robust policy heuristics , 2003, Cognitive Systems Research.

[2]  Mikhail M. Svinin,et al.  Initial experiments on reinforcement learning control of cooperative manipulations , 2000, Proceedings. 2000 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2000) (Cat. No.00CH37113).

[3]  Long Ji Lin,et al.  Scaling Up Reinforcement Learning for Robot Control , 1993, International Conference on Machine Learning.

[4]  Shin Ishii,et al.  Multi-agent reinforcement learning: an approach based on the other agent's internal model , 2000, Proceedings Fourth International Conference on MultiAgent Systems.

[5]  Andrew W. Moore,et al.  Prioritized Sweeping: Reinforcement Learning with Less Data and Less Time , 1993, Machine Learning.

[6]  Vijay Kumar,et al.  An architecture for tightly coupled multi-robot cooperation , 2001, Proceedings 2001 ICRA. IEEE International Conference on Robotics and Automation (Cat. No.01CH37164).

[7]  Henrik I. Christensen,et al.  Multi-agent reinforcement learning: using macro actions to learn a mating task , 2004, 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (IEEE Cat. No.04CH37566).

[8]  Peter Stone,et al.  Scaling Reinforcement Learning toward RoboCup Soccer , 2001, ICML.

[9]  Ming Tan,et al.  Multi-Agent Reinforcement Learning: Independent versus Cooperative Agents , 1997, ICML.

[10]  Rolf Drechsler,et al.  Applications of Evolutionary Computing, EvoWorkshops 2008: EvoCOMNET, EvoFIN, EvoHOT, EvoIASP, EvoMUSART, EvoNUM, EvoSTOC, and EvoTransLog, Naples, Italy, March 26-28, 2008. Proceedings , 2008, EvoWorkshops.

[11]  Jun Morimoto,et al.  Acquisition of stand-up behavior by a real robot using hierarchical reinforcement learning , 2000, Robotics Auton. Syst..

[12]  Minoru Asada,et al.  Action-based sensor space categorization for robot learning , 1996, Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems. IROS '96.

[13]  Minoru Asada,et al.  Learning from conceptual aliasing caused by direct teaching , 1999, IEEE SMC'99 Conference Proceedings. 1999 IEEE International Conference on Systems, Man, and Cybernetics (Cat. No.99CH37028).

[14]  Michael L. Littman,et al.  Markov Games as a Framework for Multi-Agent Reinforcement Learning , 1994, ICML.

[15]  Leslie Pack Kaelbling,et al.  Planning and Acting in Partially Observable Stochastic Domains , 1998, Artif. Intell..

[16]  Richard O. Duda,et al.  Pattern classification and scene analysis , 1974, A Wiley-Interscience publication.

[17]  Andrew W. Moore,et al.  Memory-based Reinforcement Learning: Converging with Less Data and Less Real Time , 1993 .

[18]  Richard S. Sutton,et al.  Generalization in ReinforcementLearning : Successful Examples UsingSparse Coarse , 1996 .

[19]  Francesco Mondada,et al.  SWARM-BOT: from concept to implementation , 2003, Proceedings 2003 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2003) (Cat. No.03CH37453).

[20]  Maja J. Mataric,et al.  Pusher-watcher: an approach to fault-tolerant tightly-coupled robot coordination , 2002, Proceedings 2002 IEEE International Conference on Robotics and Automation (Cat. No.02CH37292).

[21]  András Lörincz,et al.  MDPs: Learning in Varying Environments , 2003, J. Mach. Learn. Res..

[22]  Minoru Asada,et al.  Skill acquisition and self-improvement for environmental change adaptation of mobile robot , 1998 .

[23]  Minoru Asada,et al.  Cooperative behavior acquisition by asynchronous policy renewal that enables simultaneous learning in multiagent environment , 2002, IEEE/RSJ International Conference on Intelligent Robots and Systems.

[24]  Kazuhiro Kosuge,et al.  Transportation of a single object by multiple decentralized-controlled nonholonomic mobile robots , 1999, Proceedings 1999 IEEE/RSJ International Conference on Intelligent Robots and Systems. Human and Environment Friendly Robots with High Intelligence and Emotional Quotients (Cat. No.99CH36289).

[25]  Minoru Asada,et al.  Cooperative Behavior Acquisition for Mobile Robots in Dynamically Changing Real Worlds Via Vision-Based Reinforcement Learning and Development , 1999, Artif. Intell..

[26]  Kazuhiro Ohkura,et al.  Cooperative behavior acquisition mechanism for a multi-robot system based on reinforcement learning in continuous space , 2003, Proceedings 2003 IEEE International Symposium on Computational Intelligence in Robotics and Automation. Computational Intelligence in Robotics and Automation for the New Millennium (Cat. No.03EX694).

[27]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[28]  Minoru Asada,et al.  Reasonable performance in less learning time by real robot based on incremental state space segmentation , 1996, Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems. IROS '96.

[29]  Kazuhiro Ohkura,et al.  Adaptive role development in a homogeneous connected robot group , 1999, IEEE SMC'99 Conference Proceedings. 1999 IEEE International Conference on Systems, Man, and Cybernetics (Cat. No.99CH37028).

[30]  Kazuhiro Ohkura,et al.  Improving the Robustness of Reinforcement Learning for a Multi-Robot System Environment , 2005, WSTST.

[31]  Kazuhiro Ohkura,et al.  Autonomous Role Assignment in a Homogeneous Multi-Robot System , 2005, J. Robotics Mechatronics.

[32]  Manuela M. Veloso,et al.  Multiagent Systems: A Survey from a Machine Learning Perspective , 2000, Auton. Robots.

[33]  Michael P. Wellman,et al.  Multiagent Reinforcement Learning: Theoretical Framework and an Algorithm , 1998, ICML.