Analysing Congestion Problems in Multi-agent Reinforcement Learning

We extend the study of congestion problems to a more realistic scenario, the Road Network Domain (RND), where the resources are no longer independent, but rather part of a network, thus choosing one path will also impact the load of another one having common road segments. We demonstrate the application of state-of-the-art multi-agent reinforcement learning methods for this new congestion model and analyse their performance. RND allows us to highlight an important limitation of resource abstraction and show that the difference rewards approach manages to better capture and inform the agents about the dynamics of the environment.

[1]  Kagan Tumer,et al.  Aligning social welfare and agent preferences to alleviate traffic congestion , 2008, AAMAS.

[2]  Ben J. A. Kröse,et al.  Learning from delayed rewards , 1995, Robotics Auton. Syst..

[3]  Richard S. Sutton,et al.  Introduction to Reinforcement Learning , 1998 .

[4]  Chieh-Yih Wan,et al.  CODA: congestion detection and avoidance in sensor networks , 2003, SenSys '03.

[5]  Kagan Tumer,et al.  Analyzing and visualizing multiagent rewards in dynamic and stochastic domains , 2008, Autonomous Agents and Multi-Agent Systems.

[6]  Sam Devlin,et al.  Potential-based difference rewards for multiagent reinforcement learning , 2014, AAMAS.

[7]  Kagan Tumer,et al.  Optimal Payoff Functions for Members of Collectives , 2001, Adv. Complex Syst..

[8]  Sam Devlin,et al.  Resource Abstraction for Reinforcement Learning in Multiagent Congestion Problems , 2016, AAMAS.

[9]  Jon Crowcroft,et al.  TCP-like congestion control for layered multicast data transfer , 1998, Proceedings. IEEE INFOCOM '98, the Conference on Computer Communications. Seventeenth Annual Joint Conference of the IEEE Computer and Communications Societies. Gateway to the 21st Century (Cat. No.98.

[10]  Jen Jen Chung,et al.  Local Approximation of Difference Evaluation Functions , 2016, AAMAS.

[11]  Kagan Tumer,et al.  Multiagent reinforcement learning in a distributed sensor network with indirect feedback , 2013, AAMAS.

[12]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[13]  Marco Wiering,et al.  Multi-Agent Reinforcement Learning for Traffic Light control , 2000 .

[14]  Kagan Tumer,et al.  A multiagent approach to managing air traffic flow , 2010, Autonomous Agents and Multi-Agent Systems.

[15]  G. Hardin,et al.  The Tragedy of the Commons , 1968, Green Planet Blues.

[16]  Kagan Tumer,et al.  Coordinating actions in congestion games: impact of top–down and bottom–up utilities , 2012, Autonomous Agents and Multi-Agent Systems.

[17]  Kagan Tumer,et al.  Traffic Congestion Management as a Learning Agent Coordination Problem , 2009, Multi-Agent Systems for Traffic and Transportation Engineering.