Standing on the Shoulders of Giants: Cross-Slice Federated Meta Learning for Resource Orchestration to Cold-Start Slice

Network slicing is a key technology in 6G communication systems to support numerous vertical applications for all scenes while providing resources on demand. Due to more time-varying and dynamic traffic flows, it is difficult for traditional methods to manage complex and highly dynamic 6G networks. Therefore, intelligent method such as Deep Reinforcement Learning (DRL) is employed into network management since DRL is a model-free and experience-driven approach. However, it is difficult to leverage one DRL model to provide customized intra-slice orchestration for various applications because of their diverse flow characteristics and service requirements. Moreover, training a DRL model is notoriously time-consuming so that it is not realistic for network operator to individually orchestrate customized network slice for each application with the DRL algorithm. Additionally, the data privacy of each application should be considered into the training process. In this paper, we propose a Federated Meta Reinforcement Learning (FedMRL) approach to tackle the cold-start problem in network slice orchestration, while reserving the data privacy. The Meta Reinforcement Learning (MRL) is leveraged to train a meta policy for rapidly learn a local policy for a specific slice orchestration task by finding a common initialization that allows for a quick adaptation towards each optimal solution. With the help of federated learning setting, the training process of meta policy is not required to collect raw data of applications to the centralized server. Experimental results show that FedMRL outperforms three baselines in terms of overall costs, end-to-end latency and convergence speed.