Energy-Efficient Power Allocation and Q-Learning-Based Relay Selection for Relay-Aided D2D Communication

Device-to-device (D2D) communication is a promising paradigm to meet the requirement of ultra-dense, low-latency and high-rate in the fifth-generation networks. However, energy consumption is a critical issue for the D2D communication, especially for D2D relay networks. To make the best use of D2D communication, the problem of optimizing energy efficiency (EE) must be addressed. In this paper, we propose a joint power allocation and relay selection (JPARS) scheme for the improvement of energy efficiency in relay-aided D2D communications underlaying cellular network. A mixed integer nonlinear fractional programming (MINLP) problem of the total EE for D2D pairs is formulated. While ensuring the quality of service (QoS) of cellular users and D2D links, we solve the power allocation problem by Dinkelbach method and Lagrange dual decomposition. After that, Q-learning, one of the reinforcement learning algorithms, is employed to solve the relay selection problem. Finally, we provide in-depth theoretical analysis of the proposed scheme in terms of complexity and signaling overhead. Simulation results verify that the proposed scheme not only overcomes the bottleneck effect, but also nearly reaches the theoretical maximum in terms of the total EE of D2D pairs.