RSS Threshold Optimization for D2D-Aided HTC/MTC in Ultra-Dense NOMA System

With the rapid development of ultra-dense networks (UDNs) and random access technologies, device-to-device (D2D) and non-orthogonal multiple access (NOMA) techniques will incorporated into future UDNs supporting both human-type communications (HTC) and machine-type communications (MTC) to fulfill the stringent requirements brought by various potential Internet of Everything (IoE) applications. Nevertheless, the combination of D2D and NOMA will make the network management more complicated. In view of this, we optimize the received signal strength (RSS) threshold value of each small base stations (SBSs) in the UDN where HTC and MTC coexist. Considering the computational complexity, we employ a multi-agent reinforcement learning based RSS threshold value selection scheme, in which each SBS acts as an agent and chose the optimal RSS threshold value to achieve maximum system throughput performance by interacting with the environment. Extensive numerical results show our proposed scheme can greatly improve the system throughput by enhancing the connectivity of massive HTC users and MTC devices via D2D and NOMA techniques.

[1]  Nei Kato,et al.  Distributed Q-Learning Aided Uplink Grant-Free NOMA for Massive Machine-Type Communications , 2021, IEEE Journal on Selected Areas in Communications.

[2]  H. Poor,et al.  A Tutorial on Ultrareliable and Low-Latency Communications in 6G: Integrating Domain Knowledge Into Deep Learning , 2021, Proceedings of the IEEE.

[3]  A. Youssef,et al.  A Hybrid NOMA/OMA Scheme for MTC in Ultra-Dense Networks , 2020, GLOBECOM 2020 - 2020 IEEE Global Communications Conference.

[4]  Amr M. Youssef,et al.  Uplink Performance of NOMA-Based Combined HTC and MTC in Ultradense Networks , 2020, IEEE Internet of Things Journal.

[5]  Ying-Chang Liang,et al.  Deep Reinforcement Learning for Multi-Agent Power Control in Heterogeneous Networks , 2020, IEEE Transactions on Wireless Communications.

[6]  Jia Shi,et al.  A Model-Driven Deep Reinforcement Learning Heuristic Algorithm for Resource Allocation in Ultra-Dense Cellular Networks , 2020, IEEE Transactions on Vehicular Technology.

[7]  Zilong Liu,et al.  Energy-Efficient Short Packet Communications for Uplink NOMA-Based Massive MTC Networks , 2019, IEEE Transactions on Vehicular Technology.

[8]  Daniela Panno,et al.  Joint Congestion Control and Resource Allocation for Massive MTC in 5G Networks Based on SCMA , 2019, 2019 15th International Conference on Telecommunications (ConTEL).

[9]  Walaa Hamouda,et al.  Capacity Analysis of Downlink NOMA-Based Coexistent HTC/MTC in UDN , 2019, 2019 International Conference on Communications, Signal Processing, and their Applications (ICCSPA).

[10]  Arumugam Nallanathan,et al.  Multi-Agent Reinforcement Learning-Based Resource Allocation for UAV Networks , 2018, IEEE Transactions on Wireless Communications.

[11]  Xianbin Wang,et al.  Toward Massive Machine Type Communications in Ultra-Dense Cellular IoT Networks: Current Issues and Machine Learning-Assisted Solutions , 2018, IEEE Communications Surveys & Tutorials.

[12]  Jeffrey G. Andrews,et al.  SINR and Throughput of Dense Cellular Networks With Stretched Exponential Path Loss , 2017, IEEE Transactions on Wireless Communications.

[13]  Nei Kato,et al.  A stochastic geometry analysis of D2D overlaying multi-channel downlink cellular networks , 2015, 2015 IEEE Conference on Computer Communications (INFOCOM).

[14]  Qing Wang,et al.  A Survey on Device-to-Device Communication in Cellular Networks , 2013, IEEE Communications Surveys & Tutorials.

[15]  Baris Eker,et al.  A finite horizon DEC-POMDP approach to multi-robot task learning , 2011, 2011 5th International Conference on Application of Information and Communication Technologies (AICT).

[16]  Nei Kato,et al.  On the Outage Probability of Device-to-Device-Communication-Enabled Multichannel Cellular Networks: An RSS-Threshold-Based Perspective , 2016, IEEE Journal on Selected Areas in Communications.