A Framework for learning multi-agent dynamic formation strategy in real-time applications

Formation strategy is one of the most important parts of many multi-agent systems with many applications in real world problems. In this paper, a framework for learning this task in a limited domain (restricted environment) is proposed. In this framework, agents learn either directly by observing an expert behavior or indirectly by observing other agents or objects behavior. First, a group of algorithms for learning formation strategy based on limited features will be presented. Due to distributed and complex nature of many multi-agent systems, it is impossible to include all features directly in the learning process; thus, a modular scheme is proposed in order to reduce the number of features. In this method, some important features have indirect influence in learning instead of directly involving them as input features. This framework has the ability to dynamically assign a group of positions to a group of agents to improve system performance. In addition, it can change the formation strategy when the context changes. Finally, this framework is able to automatically produce many complex and flexible formation strategy algorithms without directly involving an expert to present and implement such complex algorithms.

[1]  Peter Vrancx,et al.  Learning multi-agent state space representations , 2010, AAMAS.

[2]  Wei Ding,et al.  Leader-following formation control based on pursuit strategies , 2009, 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[3]  Jianda Han,et al.  Multi-Unmanned Helicopter formation control on relative dynamics , 2009, 2009 International Conference on Mechatronics and Automation.

[4]  Ronald C. Arkin,et al.  An Behavior-based Robotics , 1998 .

[5]  Yoshitaka Sakurai,et al.  Using contexts to supervise a collaborative process , 2010, Artificial Intelligence for Engineering Design, Analysis and Manufacturing.

[6]  Avelino J. Gonzalez,et al.  Learning tactical human behavior through observation of human performance , 2006, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[7]  Philippe Fraisse,et al.  Decentralized strategy for car-like robot formations , 2007, 2007 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[8]  Chong Xiang,et al.  A Two-Stage Formation Flying Strategy to Reduce the Mission Time , 2007, 2007 IEEE International Conference on System of Systems Engineering.

[9]  Kenneth Sundaraj,et al.  A survey on team strategies in robot soccer: team strategies and role description , 2013, Artificial Intelligence Review.

[10]  Gerhard Weiss,et al.  Multiagent systems: a modern approach to distributed artificial intelligence , 1999 .

[11]  Wei Yan,et al.  A leader-follower formation control strategy for AUVs based on line-of-sight guidance , 2009, 2009 International Conference on Mechatronics and Automation.

[12]  Luís Paulo Reis,et al.  Situation Based Strategic Positioning for Coordinating a Team of Homogeneous Agents , 2000, Balancing Reactivity and Social Deliberation in Multi-Agent Systems.

[13]  Marc Halbrand Arne Voigtlander.  Brainstormers 3D { Team Description 2005 , 2005 .

[14]  SundarajKenneth,et al.  A survey on team strategies in robot soccer , 2013 .

[15]  N. Derbel,et al.  Nonlinear Formation Control of a Group of Underactuated Ships , 2007, OCEANS 2007 - Europe.

[16]  Brian S. Stensrud,et al.  Context-Based Reasoning: A Revised Specification , 2004, FLAIRS.

[17]  Yoav Shoham,et al.  If multi-agent learning is the answer, what is the question? , 2007, Artif. Intell..

[18]  L. Ljung,et al.  Overtraining, Regularization, and Searching for Minimum in Neural Networks , 1992 .

[19]  Lorenzo Sabattini,et al.  Potential based control strategy for arbitrary shape formations of mobile robots , 2009, 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[20]  Mohammad Bagher Menhaj,et al.  Training feedforward networks with the Marquardt algorithm , 1994, IEEE Trans. Neural Networks.

[21]  Ricardo Carelli,et al.  Decentralized control of leader-follower formations of mobile robots with obstacle avoidance , 2009, 2009 IEEE International Conference on Mechatronics.

[22]  Loizos Michael,et al.  Learning from Partial Observations , 2007, IJCAI.

[23]  A. Gonzalez,et al.  An approach towards building human behavior models automatically by observation , 2002 .

[24]  W. Ren.  Consensus strategies for cooperative control of vehicle formations , 2007 .

[25]  Roya Asadi,et al.  A Framework For Intelligent Multi Agent System Based Neural Network Classification Model , 2009, ArXiv.

[26]  Wei Ren,et al.  Consensus strategies for cooperative control of vehicle formations , 2007 .

[27]  Jeremy G. Siek,et al.  The Boost Graph Library - User Guide and Reference Manual , 2001, C++ in-depth series.

[28]  Alvin E. Roth,et al.  Multi-agent learning and the descriptive value of simple models , 2007, Artif. Intell..

[29]  Ali H. Sayed,et al.  Self-organization in bird flight formations using diffusion adaptation , 2009, 2009 3rd IEEE International Workshop on Computational Advances in Multi-Sensor Adaptive Processing (CAMSAP).

[30]  Yantao Tian,et al.  Swarm robots aggregation formation control inspired by fish school , 2007, 2007 IEEE International Conference on Robotics and Biomimetics (ROBIO).

[31]  Wei Ren,et al.  Consensus based formation control strategies for multi-vehicle systems , 2006, 2006 American Control Conference.

[32]  Yong Duan,et al.  A multi-agent reinforcement learning approach to robot soccer , 2012, Artificial Intelligence Review.

[33]  Peter Stone,et al.  Learning Complementary Multiagent Behaviors: A Case Study , 2009, RoboCup.

[34]  Lasheng Yu,et al.  Research on task decomposition and state abstraction in reinforcement learning , 2011, Artificial Intelligence Review.

[35]  Danwei Wang,et al.  Satellite formation keeping via real-time optimal control and iterative learning control , 2009, 2009 IEEE Aerospace conference.

[36]  Masaru Uchiyama,et al.  A symmetric hybrid position/force control scheme for the coordination of two robots , 1988, Proceedings. 1988 IEEE International Conference on Robotics and Automation.

[37]  John R. Anderson,et al.  MACHINE LEARNING An Artificial Intelligence Approach , 2009 .

[38]  G. Lewicki,et al.  Approximation by Superpositions of a Sigmoidal Function , 2003 .

[39]  F.Y. Hadaegh,et al.  A survey of spacecraft formation flying guidance and control. Part II: control , 2004, Proceedings of the 2004 American Control Conference.

[40]  Hiroaki Yamaguchi,et al.  A Cooperative Hunting Behavior by Mobile-Robot Troops , 1999, Int. J. Robotics Res..

[41]  Robert Hecht-Nielsen,et al.  Theory of the backpropagation neural network , 1989, International 1989 Joint Conference on Neural Networks.

[42]  Tucker R. Balch,et al.  Behavior-based coordination of large-scale robot formations , 2000, Proceedings Fourth International Conference on MultiAgent Systems.

[43]  P. J. Gmytrasiewicz,et al.  A Framework for Sequential Planning in Multi-Agent Settings , 2005, AI&M.

[44]  Kourosh Meshgi,et al.  Nemesis Team Description 2010 , 2010 .

[45]  Antonios Tsourdos,et al.  Centralized/decentralized control for spacecraft formation flying near Sun-Earth L2 point , 2009, 2009 4th IEEE Conference on Industrial Electronics and Applications.

[46]  H. Kuhn.  The Hungarian method for the assignment problem , 1955 .

[47]  A. Uusitalo,et al.  Overtraining , 2001, The Physician and sportsmedicine.