Deep Reinforcement Learning for Channel Selection and Power Control in D2D Networks

As a promising candidate to alleviate the mobile traffic explosion, device-to-device (D2D) technology enables the direct communications between proximal devices. To mitigate the mutual interference among D2D pairs and improve the spectrum efficiency, this paper investigates a weighted-sum-rate (WSR) maximization problem in multi-channel D2D networks. Particularly, we propose a deep reinforcement learning based scheme for each D2D pair to make decisions independently on the channel selection and power control. In contrast to the conventional methods that require instantaneous global network information, the proposed scheme only needs local information and outdated feedbacks. Simulation results demonstrate that, in terms of the WSR, the proposed scheme outperforms the conventional suboptimal fractional programming algorithm that requires the instantaneous global network information.

[1]  Qing Wang,et al.  A Survey on Device-to-Device Communication in Cellular Networks , 2013, IEEE Communications Surveys & Tutorials.

[2]  Andrea Abrardo,et al.  Distributed Power Allocation for D2D Communications Underlaying/Overlaying OFDMA Cellular Networks , 2016, IEEE Transactions on Wireless Communications.

[3]  Wei Yu,et al.  Fractional Programming for Communication Systems—Part I: Power Control and Beamforming , 2018, IEEE Transactions on Signal Processing.

[4]  Soung Chang Liew,et al.  Deep-Reinforcement Learning Multiple Access for Heterogeneous Wireless Networks , 2017, 2018 IEEE International Conference on Communications (ICC).

[5]  Lawrence Wai-Choong Wong,et al.  A Stackelberg Game Model for Overlay D2D Transmission With Heterogeneous Rate Requirements , 2015, IEEE Transactions on Vehicular Technology.

[6]  Zi-Yang Yang,et al.  Efficient resource allocation algorithm for overlay D2D communication , 2017, Comput. Networks.

[7]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[8]  Ying-Chang Liang,et al.  Applications of Deep Reinforcement Learning in Communications and Networking: A Survey , 2018, IEEE Communications Surveys & Tutorials.

[9]  Zhi-Quan Luo,et al.  Dynamic Spectrum Management: Complexity and Duality , 2008, IEEE Journal of Selected Topics in Signal Processing.

[10]  Ying-Chang Liang,et al.  Deep Reinforcement Learning-Based Modulation and Coding Scheme Selection in Cognitive Heterogeneous Networks , 2018, IEEE Transactions on Wireless Communications.

[11]  Li Wang,et al.  Device-to-Device Communications in Cellular Networks , 2016, SpringerBriefs in Electrical and Computer Engineering.

[12]  Stefan Parkvall,et al.  Design aspects of network assisted device-to-device communications , 2012, IEEE Communications Magazine.

[13]  C. Siva Ram Murthy,et al.  A novel spectrum reuse scheme for interference mitigation in a dense overlay D2D network , 2015, 2015 IEEE 26th Annual International Symposium on Personal, Indoor, and Mobile Radio Communications (PIMRC).