Optimizing information exchange in cooperative multi-agent systems

Decentralized control of a cooperative multi-agent system is the problem faced by multiple decision-makers that share a common set of objectives. The decision-makers may be robots placed at separate geographical locations or computational processes distributed in an information space. It may be impossible or undesirable for these decision-makers to share all their knowledge all the time. Furthermore, exchanging information may incur a cost associated with the required bandwidth or with the risk of revealing it to competing agents. Assuming that communication may not be reliable adds another dimension of complexity to the problem.This paper develops a decision-theoretic solution to this problem, treating both standard actions and communication as explicit choices that the decision maker must consider. The goal is to derive both action policies and communication policies that together optimize a global value function. We present an analytical model to evaluate the trade-off between the cost of communication and the value of the information received. Finally, to address the complexity of this hard optimization problem, we develop a practical approximation technique based on myopic meta-level control of communication.

[1]  Claudia V. Goldman,et al.  Emergent Coordination through the Use of Cooperative State-Changing Rules , 1994, AAAI.

[2]  Maja J. Mataric,et al.  Coordinating mobile robot group behavior using a model of interaction dynamics , 1999, AGENTS '99.

[3]  Jun Wang,et al.  Mutual online concept learning for multiple agents , 2002, AAMAS '02.

[4]  Sarit Kraus,et al.  Collaborative Plans for Complex Group Action , 1996, Artif. Intell..

[5]  Piotr J. Gmytrasiewicz,et al.  Toward automated evolution of agent communication languages , 2002, Proceedings of the 35th Annual Hawaii International Conference on System Sciences.

[6]  Michael L. Littman,et al.  Markov Games as a Framework for Multi-Agent Reinforcement Learning , 1994, ICML.

[7]  Kee-Eung Kim,et al.  Learning to Cooperate via Policy Search , 2000, UAI.

[8]  Edmund H. Durfee,et al.  Coordination of distributed problem solvers , 1988 .

[9]  Neil Immerman,et al.  The Complexity of Decentralized Control of Markov Decision Processes , 2000, UAI.

[10]  Craig Boutilier,et al.  Sequential Optimality and Coordination in Multiagent Systems , 1999, IJCAI.

[11]  Michael P. Wellman,et al.  Multiagent Reinforcement Learning: Theoretical Framework and an Algorithm , 1998, ICML.

[12]  Victor R. Lesser,et al.  Communication decisions in multi-agent cooperation: model and experiments , 2001, AGENTS '01.

[13]  Maja J. Mataric,et al.  Learning social behavior , 1997, Robotics Auton. Syst..

[14]  Shlomo Zilberstein,et al.  A Value-Driven System for Autonomous Information Gathering , 2004, Journal of Intelligent Information Systems.

[15]  Craig Boutilier,et al.  The Dynamics of Reinforcement Learning in Cooperative Multiagent Systems , 1998, AAAI/IAAI.

[16]  Victor R. Lesser,et al.  Generalizing the Partial Global Planning Algorithm , 1992, Int. J. Cooperative Inf. Syst..

[17]  Milind Tambe,et al.  The Communicative Multiagent Team Decision Problem: Analyzing Teamwork Theories and Models , 2011, J. Artif. Intell. Res..

[18]  Stuart J. Russell,et al.  Principles of Metareasoning , 1989, Artif. Intell..