Handling Communication Restrictions and Team Formation in Congestion Games

There are many domains in which a multi-agent system needs to maximize a “system utility” function which rates the performance of the entire system, while subject to communication restrictions among the agents. Such communication restrictions make it difficult for agents that take actions to optimize their own “private” utilities to also help optimize the system utility. In this article we show how previously introduced utilities that promote coordination among agents can be modified to be effective in domains with communication restrictions. The modified utilities provide performance improvements of up to 75 over previously used utilities in congestion games (i.e., games where the system utility depends solely on the number of agents choosing a particular action). In addition, we show that in the presence of severe communication restrictions, team formation for the purpose of information sharing among agents leads to an additional 25 improvement in system utility. Finally, we show that agents’ private utilities and team sizes can be manipulated to form the best compromise between how “aligned” an agent’s utility is with the system utility and how easily an agent can learn that utility.

[1]  Kagan Tumer,et al.  Optimal Payoff Functions for Members of Collectives , 2001, Adv. Complex Syst..

[2]  W. Arthur.  Complexity in economic theory: inductive reasoning and bounded rationality , 1994 .

[3]  W. Arthur.  Inductive Reasoning and Bounded Rationality , 1994 .

[4]  Milind Tambe,et al.  The Communicative Multiagent Team Decision Problem: Analyzing Teamwork Theories and Models , 2011, J. Artif. Intell. Res..

[5]  Frank Dignum,et al.  Communication for Goal Directed Agents , 2003, Communication in Multiagent Systems.

[6]  Andrew G. Barto,et al.  Improving Elevator Performance Using Reinforcement Learning , 1995, NIPS.

[7]  Tucker R. Balch,et al.  Distributed sensor fusion for object position estimation by multi-robot systems , 2001, Proceedings 2001 ICRA. IEEE International Conference on Robotics and Automation (Cat. No.01CH37164).

[8]  Kagan Tumer,et al.  Using Collective Intelligence to Route Internet Traffic , 1998, NIPS.

[9]  Theodore Groves,et al.  Incentives in Teams , 1973 .

[10]  Milind Tambe,et al.  Toward Team-Oriented Programming , 1999, ATAL.

[11]  Craig Boutilier,et al.  Planning, Learning and Coordination in Multiagent Decision Processes , 1996, TARK.

[12]  Victor R. Lesser,et al.  Coalitions Among Computationally Bounded Agents , 1997, Artif. Intell..

[13]  Sarit Kraus,et al.  Negotiation and Cooperation in Multi-Agent Environments , 1997, Artif. Intell..

[14]  Wolfram Burgard,et al.  A Probabilistic Approach to Collaborative Multi-Robot Localization , 2000, Auton. Robots.

[15]  Kagan Tumer,et al.  Multi-agent reward analysis for learning in noisy domains , 2005, AAMAS '05.

[16]  Philip R. Cohen,et al.  Toward a semantics for a speech act based agent communications language , 1995 .

[17]  Kagan Tumer,et al.  Collective Intelligence for Control of Distributed Dynamical Systems , 1999, ArXiv.

[18]  William Vickrey,et al.  Counterspeculation, Auctions, And Competitive Sealed Tenders , 1961 .

[19]  Douglas W. Gage,et al.  How to communicate with zillions of robots , 1994, Other Conferences.

[20]  Kagan Tumer,et al.  Coordinating multi-rover systems: evaluation functions for dynamic and noisy environments , 2005, GECCO '05.

[21]  Kagan Tumer,et al.  Collectives and Design Complex Systems , 2004 .

[22]  Barbara Dunin-Keplicz,et al.  Agent Theory for Team Formation by Dialogue , 2000, ATAL.

[23]  Noam Nisan,et al.  Auctions with Severely Bounded Communication , 2007, J. Artif. Intell. Res..

[24]  Kagan Tumer,et al.  Reinforcement Learning in Large Multi-agent Systems , 2005 .

[25]  Monica Divitini,et al.  Using Agents to Support the Selection of Virtual Enterprise Teams , 2002, AOIS@AAMAS.

[26]  Maja J. Matarić,et al.  Robots in Formation Using Local Information , 2002 .

[27]  Maja J. Mataric,et al.  Reward Functions for Accelerated Learning , 1994, ICML.

[28]  Kagan Tumer,et al.  Learning sequences of actions in collectives of autonomous agents , 2002, AAMAS '02.

[29]  Edmund H. Durfee,et al.  Congregation Formation in Multiagent Systems , 2003, Autonomous Agents and Multi-Agent Systems.

[30]  Kagan Tumer,et al.  Efficient Evaluation Functions for Multi-rover Systems , 2004, GECCO.

[31]  Tucker R. Balch,et al.  Communication in reactive multiagent robotic systems , 1995, Auton. Robots.

[32]  David H. Wolpert,et al.  Designing agent collectives for systems with markovian dynamics , 2002, AAMAS '02.

[33]  Michael P. Wellman,et al.  Multiagent Reinforcement Learning: Theoretical Framework and an Algorithm , 1998, ICML.

[34]  Victor R. Lesser,et al.  Communication decisions in multi-agent cooperation: model and experiments , 2001, AGENTS '01.

[35]  Steven P. Ketchpel.  Forming Coalitions in the Face of Uncertain Rewards , 1994, AAAI.

[36]  Pedro S. de Souza,et al.  Asynchronous Teams: Cooperation Schemes for Autonomous Agents , 1998, J. Heuristics.

[37]  Sandip Sen,et al.  Learning to Coordinate without Sharing Information , 1994, AAAI.

[38]  Paolo Busetta,et al.  Channeled multicast for group communications , 2002, AAMAS '02.

[39]  Sarit Kraus,et al.  Collaborative Plans for Complex Group Action , 1996, Artif. Intell..