Intelligent Resource Allocation in Joint Radar-Communication With Graph Neural Networks

Autonomous vehicles produce high data rates of sensory information from sensing systems. To achieve the advantages of sensor fusion among different vehicles in a cooperative driving scenario, high data-rate communication becomes essential. Current strategies for joint radar-communication (JRC) often rely on specialized hardware, prior knowledge of the system model, and entail diminished capability in either radar or communication functions. In this paper, we propose a framework for intelligent vehicles to conduct JRC, with minimal prior knowledge of the system model and a tunable performance balance, in an environment where surrounding vehicles execute radar detection periodically, which is typical in contemporary protocols. We introduce a metric on the usefulness of data to help an intelligent vehicle decide what, and to whom, data should be transmitted. The problem framework is cast as a generalized form of the Markov Decision Process (MDP). We identify deep reinforcement learning algorithms (DRL) and algorithmic extensions suitable for solving our JRC problem. For multi-agent scenarios, we introduce a Graph Neural Network (GNN) framework via a control channel. This framework enables modular and fair comparisons of various algorithmic extensions. Our experiments show that DRL results in superior performance compared to non-learning algorithms. Learning of inter-agent coordination in the GNN framework, based only on the Markov task reward, further improves performance.

[1]  Dong In Kim,et al.  Learning to Schedule Joint Radar-Communication Requests for Optimal Information Freshness , 2021, 2021 IEEE Intelligent Vehicles Symposium (IV).

[2]  Klaus Diepold,et al.  Multi-agent deep reinforcement learning: a survey , 2021, Artificial Intelligence Review.

[3]  Xuming Fang,et al.  Adaptive Scheduling for Joint CommRadar: Optimizing Tradeoff Among Data Throughput, Queueing Delay, and Detection Opportunities , 2021, 2021 IEEE 93rd Vehicular Technology Conference (VTC2021-Spring).

[4]  Heng Yang,et al.  Queue-Aware Dynamic Resource Allocation for the Joint Communication-Radar System , 2021, IEEE Transactions on Vehicular Technology.

[5]  Ning Cao,et al.  Joint Bi-Static Radar and Communications Designs for Intelligent Transportation , 2020, IEEE Transactions on Vehicular Technology.

[6]  Volker Markl,et al.  A survey of adaptive sampling and filtering algorithms for the internet of things , 2020, DEBS.

[7]  Dinh Thai Hoang,et al.  iRDRC: An Intelligent Real-Time Dual-Functional Radar-Communication System for Automotive Vehicles , 2020, IEEE Wireless Communications Letters.

[8]  Kaishun Wu,et al.  Space-Domain Index Modulation for mmWave Cloud Radio Access Networks , 2020, IEEE Transactions on Vehicular Technology.

[9]  M. Inomata,et al.  Transparent Glass Antenna for 28 GHz and its Signal Reception Characteristics in Urban Environment , 2020, 2020 14th European Conference on Antennas and Propagation (EuCAP).

[10]  Jun Zhao,et al.  Artificial-Intelligence-Enabled Intelligent 6G Networks , 2019, IEEE Network.

[11]  Yujing Hu,et al.  Multi-Agent Game Abstraction via Graph Attention Neural Network , 2019, AAAI.

[12]  Xinyi Wang,et al.  V2X-communication assisted interference minimization for automotive radars , 2019, China Communications.

[13]  Wayne Stark,et al.  Interference in Automotive Radar Systems: Characteristics, mitigation techniques, and current and future research , 2019, IEEE Signal Processing Magazine.

[14]  Yonina C. Eldar,et al.  Joint Radar-Communication Strategies for Autonomous Vehicles: Combining Two Key Automotive Technologies , 2019, IEEE Signal Processing Magazine.

[15]  Geoffrey Ye Li,et al.  Deep-Learning-Based Wireless Resource Allocation With Application to Vehicular Networks , 2019, Proceedings of the IEEE.

[16]  Ahmed Alkhateeb,et al.  Deep Reinforcement Learning for 5G Networks: Joint Beamforming, Power Control, and Interference Coordination , 2019, IEEE Transactions on Communications.

[17]  Lajos Hanzo,et al.  Joint Radar and Communication Design: Applications, State-of-the-Art, and the Road Ahead , 2019, IEEE Transactions on Communications.

[18]  Dusit Niyato,et al.  Deep Reinforcement Learning for Mobile 5G and Beyond: Fundamentals, Applications, and Challenges , 2019, IEEE Vehicular Technology Magazine.

[19]  Jan Eric Lenssen,et al.  Fast Graph Representation Learning with PyTorch Geometric , 2019, ArXiv.

[20]  Nicolas Le Roux,et al.  Understanding the impact of entropy on policy optimization , 2018, ICML.

[21]  Tiejun Huang,et al.  Graph Convolutional Reinforcement Learning , 2018, ICLR.

[22]  Ying-Chang Liang,et al.  Applications of Deep Reinforcement Learning in Communications and Networking: A Survey , 2018, IEEE Communications Surveys & Tutorials.

[23]  Joelle Pineau,et al.  TarMAC: Targeted Multi-Agent Communication , 2018, ICML.

[24]  Amanpreet Singh,et al.  Learning when to Communicate at Scale in Multiagent Cooperative and Competitive Tasks , 2018, ICLR.

[25]  Geoffrey Ye Li,et al.  Deep Reinforcement Learning Based Resource Allocation for V2V Communications , 2018, IEEE Transactions on Vehicular Technology.

[26]  Walid Saad,et al.  Federated Learning for Ultra-Reliable Low-Latency V2V Communications , 2018, 2018 IEEE Global Communications Conference (GLOBECOM).

[27]  Ying Li,et al.  Deep Learning Coordinated Beamforming for Highly-Mobile Millimeter Wave Systems , 2018, IEEE Access.

[28]  Geoffrey Ye Li,et al.  Toward Intelligent Vehicular Networks: A Machine Learning Framework , 2018, IEEE Internet of Things Journal.

[29]  Eytan Modiano,et al.  Optimizing Information Freshness in Wireless Networks Under General Interference Constraints , 2018, IEEE/ACM Transactions on Networking.

[30]  Albin Severinson,et al.  Design and Experimental Validation of a Cooperative Driving Control Architecture for the Grand Cooperative Driving Challenge 2016 , 2018, IEEE Transactions on Intelligent Transportation Systems.

[31]  E. Modiano,et al.  Scheduling Algorithms for Minimizing Age of Information in Wireless Broadcast Networks with Random Arrivals , 2017, IEEE Transactions on Mobile Computing.

[32]  Christian Steger,et al.  Next generation radar sensors in automotive sensor fusion systems , 2017, 2017 Sensor Data Fusion: Trends, Solutions, Applications (SDF).

[33]  Raphaël Couturier,et al.  Real-time sampling rate adaptation based on continuous risk level evaluation in wireless body sensor networks , 2017, 2017 IEEE 13th International Conference on Wireless and Mobile Computing, Networking and Communications (WiMob).

[34]  Alec Radford,et al.  Proximal Policy Optimization Algorithms , 2017, ArXiv.

[35]  Li Li,et al.  Vehicle headway modeling and its inferences in macroscopic/microscopic traffic flow theory: A survey , 2017 .

[36]  Theodore S. Rappaport,et al.  Directional Radio Propagation Path Loss Models for Millimeter-Wave Wireless Networks in the 28-, 60-, and 73-GHz Bands , 2016, IEEE Transactions on Wireless Communications.

[37]  Max Welling,et al.  Semi-Supervised Classification with Graph Convolutional Networks , 2016, ICLR.

[38]  Amitangshu Pal,et al.  On the Feasibility of Distributed Sampling Rate Adaptation in Heterogeneous and Collaborative Wireless Sensor Networks , 2016, 2016 25th International Conference on Computer Communication and Networks (ICCCN).

[39]  Rob Fergus,et al.  Learning Multiagent Communication with Backpropagation , 2016, NIPS.

[40]  Shimon Whiteson,et al.  Learning to Communicate with Deep Multi-Agent Reinforcement Learning , 2016, NIPS.

[41]  Robert W. Heath,et al.  Millimeter-Wave Vehicular Communication to Support Massive Automotive Sensing , 2016, IEEE Communications Magazine.

[42]  Alex Graves,et al.  Asynchronous Methods for Deep Reinforcement Learning , 2016, ICML.

[43]  Klaus C. J. Dietmayer,et al.  Autonomous driving at Ulm University: A modular, robust, and sensor-independent fusion approach , 2015, 2015 IEEE Intelligent Vehicles Symposium (IV).

[44]  Jürgen Dickmann,et al.  Making Bertha See Even More: Radar Contribution , 2015, IEEE Access.

[45]  Eytan Modiano,et al.  Optimizing age-of-information in a multi-class queueing system , 2015, 2015 IEEE International Symposium on Information Theory (ISIT).

[46]  Julius Ziegler,et al.  Making Bertha Drive—An Autonomous Journey on a Historic Route , 2014, IEEE Intelligent Transportation Systems Magazine.

[47]  Theodore S. Rappaport,et al.  Millimeter Wave Mobile Communications for 5G Cellular: It Will Work! , 2013, IEEE Access.

[48]  Theodore S. Rappaport,et al.  38 GHz and 60 GHz angle-dependent propagation for cellular & peer-to-peer wireless communications , 2012, 2012 IEEE International Conference on Communications (ICC).

[49]  Roy D. Yates,et al.  Real-time status: How often should one update? , 2012, 2012 Proceedings IEEE INFOCOM.

[50]  Sanjit Krishnan Kaul,et al.  Minimizing age of information in vehicular networks , 2011, 2011 8th Annual IEEE Communications Society Conference on Sensor, Mesh and Ad Hoc Communications and Networks.

[51]  Zhaohui Wu,et al.  Intelligent Transportation Systems , 2006, IEEE Pervasive Computing.

[52]  J. Bartram,et al.  Digital Communication Systems , 1987, IEEE Journal of Oceanic Engineering.

[53]  Dong In Kim,et al.  Learning to Schedule Joint Radar-Communication with Deep Multi-Agent Reinforcement Learning , 2021, IEEE Transactions on Vehicular Technology.

[54]  Ben-Zion Bobrovsky,et al.  Kalman Filtering With Adaptive Step Size Using a Covariance-Based Criterion , 2021, IEEE Transactions on Instrumentation and Measurement.

[55]  A. Gleave,et al.  Stable-Baselines3: Reliable Reinforcement Learning Implementations , 2021, J. Mach. Learn. Res..

[56]  Xiang Zhang,et al.  A Survey on Platoon-Based Vehicular Cyber-Physical Systems , 2016, IEEE Communications Surveys & Tutorials.