MLIMAS: A Framework for Machine Learning in Interactive Multi-agent Systems

Abstract Multi-agent systems in complex, real time domains require agents to act effectively both autonomously and as part of a team. The complexity of many tasks arising in these domains makes them difficult to solve with pre-programmed agent behaviors. The agents must instead discover a solution on their own, using learning. In this paper, we present MLIMAS a framework for Machine Learning in Interactive Multi-Agent Systems. The MLIMAS is proposed to provide answers to the issues arising from integrating machine learning algorithms in interactive multi-agent systems, focusing on three questions i) what are the learning targets for agents?, (ii) how can the machine learning system be integrated into the agent architecture?, and (iii) how can agents learn interactively?. MLIMAS addresses those three questions plus supporting multi-agent systems consisting of autonomous and adaptive agents acting in real-time and noisy environments. As a result of such required capabilities, MLIMAS allows dynamic and intelligent behavior of the agents to efficiently achieve their local and coalition goals such through modeling other agents actions, and interactively taking benefits of self and others preferences in learning and achieving the agents goals. We studied the proposed framework in the Taxi Domain compared with the traditional Q-Learning algorithm without interactive share of information. Our experiments showed 2 times improvement for the average award received per agents trail rather than the traditional Q-Learning approach. In addition, we have got %80 improvement for the same number of trials of the agents to reach the passengers.

[1]  Bartlomiej Sniezynski,et al.  Agent Strategy Generation by Rule Induction , 2014, Comput. Informatics.

[2]  Matthias Klusch,et al.  Distributed data mining and agents , 2005, Eng. Appl. Artif. Intell..

[3]  Gerhard Weiss,et al.  Multiagent systems: a modern approach to distributed artificial intelligence , 1999 .

[4]  Robert Craven,et al.  Argumentation-based reinforcement learning for robocup soccer takeaway , 2014, AAMAS.

[5]  Bartlomiej Sniezynski,et al.  Agent Strategy Generation by Rule Induction in Predator-Prey Problem , 2009, ICCS.

[6]  Leigh Tesfatsion,et al.  Agent-Based Computational Economics: Growing Economies From the Bottom Up , 2002, Artificial Life.

[7]  Peter Dayan,et al.  Technical Note: Q-Learning , 2004, Machine Learning.

[8]  Andrew W. Moore,et al.  Reinforcement Learning: A Survey , 1996, J. Artif. Intell. Res..

[9]  Csaba Szepesvári,et al.  A Unified Analysis of Value-Function-Based Reinforcement-Learning Algorithms , 1999, Neural Computation.

[10]  Daniel Kudenko,et al.  Learning in multi-agent systems , 2001, The Knowledge Engineering Review.

[11]  Daniel Kudenko,et al.  Multi-Agent Reinforcement Learning for Intrusion Detection: A Case Study and Evaluation , 2008, MATES.

[12]  Craig Boutilier,et al.  The Dynamics of Reinforcement Learning in Cooperative Multiagent Systems , 1998, AAAI/IAAI.

[13]  Peter Dayan,et al.  Q-learning , 1992, Machine Learning.

[14]  Manuela M. Veloso,et al.  Multiagent Systems: A Survey from a Machine Learning Perspective , 2000, Auton. Robots.

[15]  Michael P. Wellman,et al.  Multiagent Reinforcement Learning: Theoretical Framework and an Algorithm , 1998, ICML.

[16]  D. Fudenberg,et al.  Consistency and Cautious Fictitious Play , 1995 .

[17]  John N. Tsitsiklis,et al.  Asynchronous stochastic approximation and Q-learning , 1993, Proceedings of 32nd IEEE Conference on Decision and Control.

[18]  Thomas G. Dietterich.  Hierarchical Reinforcement Learning with the MAXQ Value Function Decomposition , 1999, J. Artif. Intell. Res..

[19]  Pattie Maes,et al.  Collaborative Interface Agents , 1994, AAAI.

[20]  Yoav Shoham,et al.  If multi-agent learning is the answer, what is the question? , 2007, Artif. Intell..