Learn to Compress CSI and Allocate Resources in Vehicular Networks

Resource allocation has a direct and profound impact on the performance of vehicle-to-everything (V2X) networks. In this paper, we develop a hybrid architecture consisting of centralized decision making and distributed resource sharing (the C-Decision scheme) to maximize the long-term sum rate of all vehicles. To reduce the network signaling overhead, each vehicle uses a deep neural network to compress its observed information that is thereafter fed back to the centralized decision making unit. The centralized decision unit employs a deep Q-network to allocate resources and then sends the decision results to all vehicles. We further adopt a quantization layer for each vehicle that learns to quantize the continuous feedback. In addition, we devise a mechanism to balance the transmission of vehicle-to-vehicle (V2V) links and vehicle-to-infrastructure (V2I) links. To further facilitate distributed spectrum sharing, we also propose a distributed decision making and spectrum sharing architecture (the D-Decision scheme) for each V2V link. Through extensive simulation results, we demonstrate that the proposed C-Decision and D-Decision schemes can both achieve near-optimal performance and are robust to feedback interval variations, input noise, and feedback noise.

[1]  Mugen Peng,et al.  Deep Reinforcement Learning-Based Mode Selection and Resource Management for Green Fog Radio Access Networks , 2018, IEEE Internet of Things Journal.

[2]  Shane Legg,et al.  Human-level control through deep reinforcement learning , 2015, Nature.

[3]  David Minnen,et al.  Variable Rate Image Compression with Recurrent Neural Networks , 2015, ICLR.

[4]  Zhu Han,et al.  Machine Learning Paradigms for Next-Generation Wireless Networks , 2017, IEEE Wireless Communications.

[5]  Zhi Ding,et al.  Graph-Based Resource Sharing in Vehicular Communication , 2018, IEEE Transactions on Wireless Communications.

[6]  Geoffrey Ye Li,et al.  Resource Allocation for D2D-Enabled Vehicular Communications , 2017, IEEE Transactions on Communications.

[7]  Philippe J. Sartori,et al.  LTE evolution for vehicle-to-everything services , 2016, IEEE Communications Magazine.

[8]  Xiaoming Wang,et al.  Incentive Mechanisms for Crowdblocking Rumors in Mobile Social Networks , 2019, IEEE Transactions on Vehicular Technology.

[9]  David Silver,et al.  Deep Reinforcement Learning with Double Q-Learning , 2015, AAAI.

[10]  Bhaskar Krishnamachari,et al.  Deep Reinforcement Learning for Dynamic Multichannel Access in Wireless Networks , 2018, IEEE Transactions on Cognitive Communications and Networking.

[11]  Yan Chen,et al.  Intelligent 5G: When Cellular Networks Meet Artificial Intelligence , 2017, IEEE Wireless Communications.

[12]  Geoffrey Ye Li,et al.  Machine Learning for Vehicular Networks: Recent Advances and Application Examples , 2018, IEEE Vehicular Technology Magazine.

[13]  Peter Dayan,et al.  Q-learning , 1992, Machine Learning.

[14]  Geoffrey Ye Li,et al.  Vehicular Communications: A Network Layer Perspective , 2017, IEEE Transactions on Vehicular Technology.

[15]  Geoffrey Ye Li,et al.  Deep-Learning-Based Wireless Resource Allocation With Application to Vehicular Networks , 2019, Proceedings of the IEEE.

[16]  Geoffrey Ye Li,et al.  Deep Reinforcement Learning Based Resource Allocation for V2V Communications , 2018, IEEE Transactions on Vehicular Technology.

[17]  Demis Hassabis,et al.  Mastering the game of Go with deep neural networks and tree search , 2016, Nature.

[18]  Yoshua Bengio,et al.  Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation , 2013, ArXiv.

[19]  Song Guo,et al.  Traffic and Computation Co-Offloading With Reinforcement Learning in Fog Computing for Industrial Applications , 2019, IEEE Transactions on Industrial Informatics.

[20]  Geoffrey Ye Li,et al.  Power of Deep Learning for Channel Estimation and Signal Detection in OFDM Systems , 2017, IEEE Wireless Communications Letters.

[21]  Yue Cao,et al.  Adaptive Network Segmentation and Channel Allocation in Large-Scale V2X Communication Networks , 2019, IEEE Transactions on Communications.

[22]  Biing-Hwang Juang,et al.  Deep Learning in Physical Layer Communications , 2018, IEEE Wireless Communications.

[23]  Li Zhao,et al.  Vehicle-to-Everything (v2x) Services Supported by LTE-Based Systems and 5G , 2017, IEEE Communications Standards Magazine.

[24]  Geoffrey Ye Li,et al.  Spectrum Sharing in Vehicular Networks Based on Multi-Agent Reinforcement Learning , 2019, IEEE Journal on Selected Areas in Communications.

[25]  Baoji Wang,et al.  Interference Hypergraph-Based Resource Allocation (IHG-RA) for NOMA-Integrated V2X Networks , 2019, IEEE Internet of Things Journal.

[26]  Geoffrey Ye Li,et al.  Resource Allocation for Low-Latency Vehicular Communications: An Effective Capacity Perspective , 2019, IEEE Journal on Selected Areas in Communications.

[27]  Lassi Hentila,et al.  WINNER II Channel Models , 2009 .

[28]  D. Ruppert.  The Elements of Statistical Learning: Data Mining, Inference, and Prediction , 2004 .

[29]  Zhigang Cao,et al.  Low Complexity Outage Optimal Distributed Channel Allocation for Vehicle-to-Vehicle Communications , 2011, IEEE Journal on Selected Areas in Communications.

[30]  Jakob Hoydis,et al.  End-to-End Learning of Communications Systems Without a Channel Model , 2018, 2018 52nd Asilomar Conference on Signals, Systems, and Computers.

[31]  Erik G. Ström,et al.  Radio Resource Management for D2D-Based V2V Communication , 2016, IEEE Transactions on Vehicular Technology.

[32]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[33]  Geoffrey Ye Li,et al.  Toward Intelligent Vehicular Networks: A Machine Learning Framework , 2018, IEEE Internet of Things Journal.

[34]  Dongning Guo,et al.  Multi-Agent Deep Reinforcement Learning for Dynamic Power Allocation in Wireless Networks , 2018, IEEE Journal on Selected Areas in Communications.

[35]  Geoffrey Ye Li,et al.  Vehicular Communications: A Physical Layer Perspective , 2017, IEEE Transactions on Vehicular Technology.

[36]  Walid Saad,et al.  Dynamic Proximity-Aware Resource Allocation in Vehicle-to-Vehicle (V2V) Communications , 2016, 2016 IEEE Globecom Workshops (GC Wkshps).

[37]  Dongning Guo,et al.  Deep Reinforcement Learning for Distributed Dynamic Power Allocation in Wireless Networks , 2018, ArXiv.

[38]  Sebastian Ruder,et al.  An overview of gradient descent optimization algorithms , 2016, Vestnik komp'iuternykh i informatsionnykh tekhnologii.

[39]  Jakob Hoydis,et al.  An Introduction to Deep Learning for the Physical Layer , 2017, IEEE Transactions on Cognitive Communications and Networking.