Reinforcement learning-based joint self-optimisation method for the fuzzy logic handover algorithm in 5G HetNets

5G heterogeneous networks (HetNets) can provide higher network coverage and system capacity to the user by deploying massive small base stations (BSs) within the 4G macro system. However, the large-scale deployment of small BSs significantly increases the complexity and workload of network maintenance and optimisation. The current handover (HO) triggering mechanism A3 event was designed only for mobility management in the macro system. Directly implementing A3 in 5G-HetNets may degrade the user mobility robustness. Motivated by the concept of self-organisation networks (SON), this study developed a self-optimised triggering mechanism to enable automated network maintenance and enhance user mobility robustness in 5G-HetNets. The proposed method integrates the advantages of subtractive clustering and Q-learning frameworks into the conventional fuzzy logic-based HO algorithm (FLHA). Subtractive clustering is first adopted to generate a membership function (MF) for the FLHA to enable FLHA with the self-configuration feature. Subsequently, Q-learning is utilised to learn the optimal HO policy from the environment as fuzzy rules that empower the FLHA with a self-optimisation function. The FLHA with SON functionality also overcomes the limitations of the conventional FLHA that must rely heavily on professional experience to design. The simulation results show that the proposed self-optimised FLHA can effectively generate MF and fuzzy rules for the FLHA. By comparing with conventional triggering mechanisms, the proposed approach can decrease the HO, ping-pong HO, and HO failure ratios by approximately 91%, 49%, and 97.5% while improving network throughput and latency by 8% and 35%, respectively.

[1]  Rajbir Kaur,et al.  Fuzzy Logic Based Spectrum Handover Approach in Cognitive Radio Network: A Survey , 2018, 2018 Second International Conference on Electronics, Communication and Aerospace Technology (ICECA).

[2]  Cheng-Xiang Wang,et al.  5G Ultra-Dense Cellular Networks , 2015, IEEE Wireless Communications.

[3]  Sangchul Oh,et al.  Frequent-Handover Mitigation in Ultra-Dense Heterogeneous Networks , 2019, IEEE Transactions on Vehicular Technology.

[4]  Suhaidi Hassan,et al.  Context-Aware Radio Access Technology Selection in 5G Ultra Dense Networks , 2017, IEEE Access.

[5]  Jenq-Shiou Leu,et al.  Using Long-Short-Term Memory Based Convolutional Neural Networks for Network Intrusion Detection , 2018, WICON.

[6]  Sungoh Kwon,et al.  Mobility Robustness Optimization for Handover Failure Reduction in LTE Small-Cell Networks , 2018, IEEE Transactions on Vehicular Technology.

[7]  Jenq-Shiou Leu,et al.  Automatic news-roundup generation using clustering, extraction, and presentation , 2019, Multimedia Systems.

[8]  Emna Fakhfakh,et al.  Optimised Q-learning for WiFi offloading in dense cellular networks , 2017, IET Commun..

[9]  Sakshi Kaushal,et al.  Handover optimization scheme for LTE-Advance networks based on AHP-TOPSIS and Q-learning , 2019, Comput. Commun..

[10]  Mohamed Saeed,et al.  An enhanced fuzzy logic optimization technique based on user mobility for LTE handover , 2017, 2017 34th National Radio Science Conference (NRSC).

[11]  Abolfazl Mehbodniya,et al.  Fuzzy-Based Game Theoretic Mobility Management for Energy Efficient Operation in HetNets , 2017, IEEE Access.

[12]  Jenq-Shiou Leu,et al.  Energy Efficient Clustering Scheme for Prolonging the Lifetime of Wireless Sensor Network With Isolated Nodes , 2015, IEEE Communications Letters.

[13]  Changyin Sun,et al.  Deep Q-Learning-Based Content Caching With Update Strategy for Fog Radio Access Networks , 2019, IEEE Access.

[14]  Debabrata Das,et al.  Self organizing method for handover performance optimization in LTE-advanced network , 2017, Comput. Commun..

[15]  Ibraheem Shayea,et al.  Auto Tuning Self-Optimization Algorithm for Mobility Management in LTE-A and 5G HetNets , 2020, IEEE Access.

[16]  Laurence Tianruo Yang,et al.  Fuzzy Logic with Engineering Applications , 1999 .

[17]  Momoh Jimoh Eyiomika Salami,et al.  Development of hybrid artificial intelligent based handover decision algorithm , 2017 .

[18]  Zdenek Becvar,et al.  Adaptive Hysteresis Margin Based on Fuzzy Logic for Handover in Mobile Networks With Dense Small Cells , 2018, IEEE Access.

[19]  Mu-Song Chen,et al.  Fuzzy clustering analysis for optimizing fuzzy membership functions , 1999, Fuzzy Sets Syst..

[20]  Fredrik Tufvesson,et al.  5G: A Tutorial Overview of Standards, Trials, Challenges, Deployment, and Practice , 2017, IEEE Journal on Selected Areas in Communications.

[21]  Jenq-Shiou Leu,et al.  Predicting Solar Performance Ratio Based on Encoder-Decoder Neural Network Model , 2019, 2019 11th International Congress on Ultra Modern Telecommunications and Control Systems and Workshops (ICUMT).

[22]  Andreas Mitschele-Thiel,et al.  Cognitive Cellular Networks: A Q-Learning Framework for Self-Organizing Networks , 2016, IEEE Transactions on Network and Service Management.

[23]  Lorenza Giupponi,et al.  From 4G to 5G: Self-organized Network Management meets Machine Learning , 2017, Comput. Commun..