Extended Variable Speed Limit control using Multi-agent Reinforcement Learning

Variable Speed Limit (VSL) is a traffic control approach that optimises the mainstream traffic on motorways. Reinforcement Learning approach to VSL has been shown to achieve improvements in controlling the mainstream traffic bottleneck on motorways. However, single-agent VSL, applied to a shorter motorway segment, can produce a discontinuity in traffic flow by causing the significant differences in speeds between the uncontrolled upstream flow and the flow affected by VSL. A multi-agent control strategy can be used to overcome these problems by assigning speed limits in multiple upstream motorway sections enabling smoother speed transition. In this paper, we proposed a novel approach to set up multi-agent RLbased VSL by using the W-Learning algorithm (WL-VSL), in which two agents control two segments in the lead up to the congested area. The reward function for each agent is based on the agent’s local performance as well as the downstream bottleneck. WL-VSL is evaluated in a microscopic simulation on two traffic scenarios using dynamic and static traffic demand. We show that WL-VSL outperforms base cases (no control, single agent, and two independent agents) with the improvement of traffic parameters up to 18 %.