A Survey on Multi-Agent Reinforcement Learning Methods for Vehicular Networks

Under the rapid development of the Internet of Things (IoT), vehicles can be recognized as mobile smart agents that communicating, cooperating, and competing for resources and information. The task between vehicles is to learn and make decisions depending on the policy to improve the effectiveness of the multi-agent system (MAS) that deals with the continually changing environment. The multi-agent reinforcement learning (MARL) is considered as one of the learning frameworks for finding reliable solutions in a highly dynamic vehicular MAS. In this paper, we provide a survey on research issues related to vehicular networks such as resource allocation, data offloading, cache placement, ultra-reliable low latency communication (URLLC), and high mobility. Furthermore, we show the potential applications of MARL that enables decentralized and scalable decision making in vehicle-to-everything (V2X) scenarios.

[1]  Xuemin Shen,et al.  Self-Sustaining Caching Stations: Toward Cost-Effective 5G-Enabled Vehicular Networks , 2017, IEEE Communications Magazine.

[2]  Antonio Liotta,et al.  Self-Learning Power Control in Wireless Sensor Networks , 2018, Sensors.

[3]  Leandro N. Balico,et al.  Localization Prediction in Vehicular Ad Hoc Networks , 2018, IEEE Communications Surveys & Tutorials.

[4]  Samad Baseer,et al.  Mobile Agent as an Approach to Improve QoS in Vehicular Ad Hoc Network , 2010 .

[5]  Sunil Agrawal,et al.  Comparative Analysis of Various Routing Protocols in VANET , 2015, 2015 Fifth International Conference on Advanced Computing & Communication Technologies.

[6]  Lin Yao,et al.  A Cooperative Caching Scheme Based on Mobility Prediction in Vehicular Content Centric Networks , 2018, IEEE Transactions on Vehicular Technology.

[7]  Pingzhi Fan,et al.  On the Connectivity of Vehicular Ad Hoc Network Under Various Mobility Scenarios , 2017, IEEE Access.

[8]  Geoffrey Ye Li,et al.  Deep Reinforcement Learning Based Resource Allocation for V2V Communications , 2018, IEEE Transactions on Vehicular Technology.

[9]  Petar Popovski,et al.  A Statistical Learning Approach to Ultra-Reliable Low Latency Communication , 2018, IEEE Transactions on Communications.

[10]  Montserrat Ros,et al.  A Comparative Survey of VANET Clustering Techniques , 2017, IEEE Communications Surveys & Tutorials.

[11]  Sergio Ilarri,et al.  An approach driven by mobile agents for data management in vehicular networks , 2017, Inf. Sci..

[12]  Yusheng Ji,et al.  Resource Allocation for SVC Streaming Over Cooperative Vehicular Networks , 2018, IEEE Transactions on Vehicular Technology.

[13]  Li Fan,et al.  Web caching and Zipf-like distributions: evidence and implications , 1999, IEEE INFOCOM '99. Conference on Computer Communications. Proceedings. Eighteenth Annual Joint Conference of the IEEE Computer and Communications Societies. The Future is Now (Cat. No.99CH36320).

[14]  Geoffrey Ye Li,et al.  Machine Learning for Vehicular Networks , 2017, ArXiv.

[15]  Joel J. P. C. Rodrigues,et al.  Clustering in vehicular ad hoc networks: Taxonomy, challenges and solutions , 2014, Veh. Commun..

[16]  Shahrokh Valaee,et al.  Congestion Control for Vehicular Networks With Safety-Awareness , 2016, IEEE/ACM Transactions on Networking.

[17]  Gang Qu,et al.  A Survey on Recent Advances in Vehicular Network Security, Trust, and Privacy , 2019, IEEE Transactions on Intelligent Transportation Systems.

[18]  Joel J. P. C. Rodrigues,et al.  Data Offloading in 5G-Enabled Software-Defined Vehicular Networks: A Stackelberg-Game-Based Approach , 2017, IEEE Communications Magazine.

[19]  Tamer Basar,et al.  Fully Decentralized Multi-Agent Reinforcement Learning with Networked Agents , 2018, ICML.

[20]  Yanmin Zhu,et al.  Improving Throughput and Fairness of Convergecast in Vehicular Networks , 2017, IEEE Transactions on Mobile Computing.

[21]  Xuemin Shen,et al.  An SMDP-Based Resource Allocation in Vehicular Cloud Computing Systems , 2015, IEEE Transactions on Industrial Electronics.

[22]  João Barros,et al.  Neighbor-Aided Localization in Vehicular Networks , 2017, IEEE Transactions on Intelligent Transportation Systems.

[23]  Victor C. M. Leung,et al.  Deep-Reinforcement-Learning-Based Optimization for Cache-Enabled Opportunistic Interference Alignment Wireless Networks , 2017, IEEE Transactions on Vehicular Technology.

[24]  Khaled Ghédira,et al.  A Multi-agent Approach for Routing on Vehicular Ad-Hoc Networks , 2013, ANT/SEIT.

[25]  Azhar Hussain,et al.  Artificial Intelligence for Vehicle-to-Everything: A Survey , 2019, IEEE Access.

[26]  Janez Bester,et al.  A survey on clustering algorithms for vehicular ad-hoc networks , 2012, 2012 35th International Conference on Telecommunications and Signal Processing (TSP).

[27]  Periklis Chatzimisios,et al.  High Reliability and Low Latency for Vehicular Networks: Challenges and Solutions , 2017, ArXiv.

[28]  C. Siva Ram Murthy,et al.  Improving Delay and Energy Efficiency of Vehicular Networks Using Mobile Femto Access Points , 2017, IEEE Transactions on Vehicular Technology.

[29]  Walid Saad,et al.  Echo State Networks for Proactive Caching in Cloud-Based Radio Access Networks With Mobile Users , 2016, IEEE Transactions on Wireless Communications.

[30]  Geoffrey Ye Li,et al.  Toward Intelligent Vehicular Networks: A Machine Learning Framework , 2018, IEEE Internet of Things Journal.

[31]  Zhou Su,et al.  An Edge Caching Scheme to Distribute Content in Vehicular Networks , 2018, IEEE Transactions on Vehicular Technology.

[32]  Narayan B. Mandayam,et al.  Joint Caching and Pricing Strategies for Popular Content in Information Centric Networks , 2016, IEEE Journal on Selected Areas in Communications.

[33]  Eylem Ekici,et al.  Vehicular Networking: A Survey and Tutorial on Requirements, Architectures, Challenges, Standards and Solutions , 2011, IEEE Communications Surveys & Tutorials.

[34]  Ke Zhang,et al.  Mobile-Edge Computing for Vehicular Networks: A Promising Network Paradigm with Predictive Off-Loading , 2017, IEEE Veh. Technol. Mag..

[35]  Guoqiang Mao,et al.  New Multi-Hop Clustering Algorithm for Vehicular Ad Hoc Networks , 2019, IEEE Transactions on Intelligent Transportation Systems.

[36]  Saeid Nahavandi,et al.  Deep Reinforcement Learning for Multiagent Systems: A Review of Challenges, Solutions, and Applications , 2018, IEEE Transactions on Cybernetics.

[37]  T. Urbanik,et al.  Reinforcement learning-based multi-agent system for network traffic signal control , 2010 .