Collaborative Multiagent Decision Making for Lane-Free Autonomous Driving

This paper addresses the problem of collaborative multi-agent autonomous driving of connected and automated vehicles (CAVs) in lane-free highway scenarios. We eliminate the lane-changing task, i.e., CAVs may be located in any arbitrary lateral position within the road boundaries, hence allowing for better utilization of the available road capacity. As a consequence, vehicles operate in a much more complex environment, and the need for the individual CAVs to select actions that are efficient for the group as a whole is highly desired. We formulate this environment as a multiagent collaboration problem represented via a coordination graph, thus decomposing the problem with local utility functions, based on the interactions between vehicles. We produce a tractable and scalable solution by estimating the joint action of all vehicles via the anytime max-plus algorithm, with local utility functions provided by potential fields, designed to promote collision avoidance. Specifically, the fields have an ellipsoid form that is most suitable for lane-free highway environments. This novel use of max-plus with potential fields gives rise to a coordinated control policy that exploits only local information specific to each CAV. Our experimental evaluation confirms the effectiveness of our approach: lane-free movement allows for increased traffic flow rates, and vehicles are able to achieve speeds that are both high and close to their desired ones, even in demanding environments with high traffic flow.

[1]  Iasson Karafyllis,et al.  Lane-free Artificial-Fluid Concept for Vehicular Traffic , 2019, Proc. IEEE.

[2]  Xuan Di,et al.  A Survey on Autonomous Vehicle Control in the Era of Mixed-Autonomy: From Physics-Based to AI-Guided Driving Policy Learning , 2020, Transportation Research Part C: Emerging Technologies.

[3]  Mykel J. Kochenderfer,et al.  A Taxonomy and Review of Algorithms for Modeling and Predicting Human Driver Behavior. , 2020 .

[4]  Yougang Bian,et al.  A Survey on Cooperative Longitudinal Motion Control of Multiple Connected and Automated Vehicles , 2020, IEEE Intelligent Transportation Systems Magazine.

[5]  Daniel Guo,et al.  Agent57: Outperforming the Atari Human Benchmark , 2020, ICML.

[6]  Xin Wang,et al.  Distributed Multiagent Coordinated Learning for Autonomous Driving in Highways Based on Dynamic Coordination Graphs , 2020, IEEE Transactions on Intelligent Transportation Systems.

[7]  S. Whiteson,et al.  Deep Coordination Graphs , 2019, ICML.

[8]  Ouns Bouachir,et al.  Performance Evaluation of Data Dissemination Protocols for Connected Autonomous Vehicles , 2020, IEEE Access.

[9]  David Elliott,et al.  Recent advances in connected and automated vehicles , 2019, Journal of Traffic and Transportation Engineering (English Edition).

[10]  Shimon Whiteson,et al.  The Representational Capacity of Action-Value Networks for Multi-Agent Reinforcement Learning , 2019, AAMAS.

[11]  I. Nourbakhsh,et al.  On the future of transportation in an era of automated and autonomous vehicles , 2019, Proceedings of the National Academy of Sciences.

[12]  Yun-Pang Flötteröd,et al.  Microscopic Traffic Simulation using SUMO , 2018, 2018 21st International Conference on Intelligent Transportation Systems (ITSC).

[13]  Tom Schaul,et al.  Rainbow: Combining Improvements in Deep Reinforcement Learning , 2017, AAAI.

[14]  Gao Fei,et al.  An Improved Artificial Potential Field Model Considering Vehicle Velocity for Autonomous Driving , 2018 .

[15]  Eugene Vinitsky,et al.  Flow: A Modular Learning Framework for Autonomy in Traffic. , 2017 .

[16]  Pablo Hernandez-Leal,et al.  A Survey of Learning in Multiagent Environments: Dealing with Non-Stationarity , 2017, ArXiv.

[17]  Dominique Gruyer,et al.  Modified artificial potential field method for online path planning applications , 2017, 2017 IEEE Intelligent Vehicles Symposium (IV).

[18]  Joshua B. Tenenbaum,et al.  Human Learning in Atari , 2017, AAAI Spring Symposia.

[19]  Markos Papageorgiou,et al.  Optimal vehicle trajectory planning in the context of cooperative merging on highways , 2016 .

[20]  Tom Schaul,et al.  Dueling Network Architectures for Deep Reinforcement Learning , 2015, ICML.

[21]  Tom Schaul,et al.  Prioritized Experience Replay , 2015, ICLR.

[22]  Frans A. Oliehoek,et al.  Coordinated Deep Reinforcement Learners for Traffic Light Control , 2016 .

[23]  Alex Graves,et al.  Playing Atari with Deep Reinforcement Learning , 2013, ArXiv.

[24]  B. Wee,et al.  The Transport System and Transport Policy: An Introduction , 2013 .

[25]  Serge P. Hoogendoorn,et al.  Traffic flow theory and modelling , 2012 .

[26]  Zhi-yi Huang,et al.  Car-following theory of steady-state traffic flow using time-to-collision , 2011 .

[27]  Shimon Whiteson,et al.  Multiagent Reinforcement Learning for Urban Traffic Control Using Coordination Graphs , 2008, ECML/PKDD.

[28]  N B Taylor,et al.  Speed, flow and density of motorway traffic , 2008 .

[29]  Peter Stone,et al.  A Multiagent Approach to Autonomous Intersection Management , 2008, J. Artif. Intell. Res..

[30]  Nikos A. Vlassis,et al.  Collaborative Multiagent Reinforcement Learning by Payoff Propagation , 2006, J. Mach. Learn. Res..

[31]  Nikos A. Vlassis,et al.  Using the Max-Plus Algorithm for Multiagent Decision Making in Coordination Graphs , 2005, BNAIC.

[32]  Nikos A. Vlassis,et al.  Sparse cooperative Q-learning , 2004, ICML.

[33]  Carlos Guestrin,et al.  Multiagent Planning with Factored MDPs , 2001, NIPS.

[34]  O. Khatib,et al.  Real-Time Obstacle Avoidance for Manipulators and Mobile Robots , 1985, Proceedings. 1985 IEEE International Conference on Robotics and Automation.