Dual Attention-Based Federated Learning for Wireless Traffic Prediction

Wireless traffic prediction is essential for cellular networks to realize intelligent network operations, such as load-aware resource management and predictive control. Existing prediction approaches usually adopt centralized training architectures and require the transferring of huge amounts of traffic data, which may raise delay and privacy concerns for certain scenarios. In this work, we propose a novel wireless traffic prediction framework named Dual Attention-Based Federated Learning (FedDA), by which a high-quality prediction model is trained collaboratively by multiple edge clients. To simultaneously capture the various wireless traffic patterns and keep raw data locally, FedDA first groups the clients into different clusters by using a small augmentation dataset. Then, a quasi-global model is trained and shared among clients as prior knowledge, aiming to solve the statistical heterogeneity challenge confronted with federated learning. To construct the global model, a dual attention scheme is further proposed by aggregating the intra-and inter-cluster models, instead of simply averaging the weights of local models. We conduct extensive experiments on two real-world wireless traffic datasets and results show that FedDA outperforms state-of-the-art methods. The average mean squared error performance gains on the two datasets are up to 10% and 30%, respectively.

[1]  Dongfeng Yuan,et al.  Citywide Cellular Traffic Prediction Based on Densely Connected Convolutional Neural Networks , 2018, IEEE Communications Letters.

[2]  Richard Nock,et al.  Advances and Open Problems in Federated Learning , 2021, Found. Trends Mach. Learn..

[3]  Shui Yu,et al.  Network Traffic Prediction Based on Deep Belief Network in Wireless Mesh Backbone Networks , 2017, 2017 IEEE Wireless Communications and Networking Conference (WCNC).

[4]  Jacques Palicot,et al.  The prediction analysis of cellular radio access network traffic: From entropy theory to networking practice , 2014, IEEE Communications Magazine.

[5]  Marco De Nadai,et al.  A multi-source dataset of urban life in the city of Milan and the Province of Trentino , 2015, Scientific Data.

[6]  Maode Ma,et al.  SVM-Based Models for Predicting WLAN Traffic , 2006, 2006 IEEE International Conference on Communications.

[7]  Klaus David,et al.  6G Vision and Requirements: Is There Any Need for Beyond 5G? , 2018, IEEE Vehicular Technology Magazine.

[8]  Hong Liu,et al.  A path planning approach for crowd evacuation in buildings based on improved artificial bee colony algorithm , 2018, Appl. Soft Comput..

[9]  Hong Liu,et al.  Crowd evacuation simulation approach based on navigation knowledge and two-layer control mechanism , 2018, Inf. Sci..

[10]  Anit Kumar Sahu,et al.  Federated Learning: Challenges, Methods, and Future Directions , 2019, IEEE Signal Processing Magazine.

[11]  Xiaomin Song,et al.  Time Series Data Augmentation for Deep Learning: A Survey , 2020, IJCAI.

[12]  Blaise Agüera y Arcas,et al.  Communication-Efficient Learning of Deep Networks from Decentralized Data , 2016, AISTATS.

[13]  Zhifeng Zhao,et al.  The Learning and Prediction of Application-Level Traffic Data in Cellular Networks , 2016, IEEE Transactions on Wireless Communications.

[14]  F. Windmeijer,et al.  R-Squared Measures for Count Data Regression Models With Applications to Health-Care Utilization , 1996 .

[15]  Shuguang Cui,et al.  Spatio-Temporal Wireless Traffic Prediction With Recurrent Neural Network , 2018, IEEE Wireless Communications Letters.

[16]  George Athanasopoulos,et al.  Forecasting: principles and practice , 2013 .

[17]  Z. Sun,et al.  Traffic predictability based on ARIMA/GARCH model , 2006, 2006 2nd Conference on Next Generation Internet Design and Engineering, 2006. NGI '06..

[18]  Jianwei Wang,et al.  6G Technologies: Key Drivers, Core Requirements, System Architectures, and Enabling Technologies , 2019, IEEE Vehicular Technology Magazine.

[19]  Dongfeng Yuan,et al.  Deep Transfer Learning for Intelligent Cellular Traffic Prediction Based on Cross-Domain Big Data , 2019, IEEE Journal on Selected Areas in Communications.

[20]  Yue Zhao,et al.  Federated Learning with Non-IID Data , 2018, ArXiv.

[21]  Nei Kato,et al.  Ten Challenges in Advancing Machine Learning Technologies toward 6G , 2020, IEEE Wireless Communications.

[22]  Zi Huang,et al.  Learning Private Neural Language Modeling with Attentive Aggregation , 2018, 2019 International Joint Conference on Neural Networks (IJCNN).

[23]  Shuguang Cui,et al.  Wireless Traffic Prediction With Scalable Gaussian Process: Framework, Algorithms, and Verification , 2019, IEEE Journal on Selected Areas in Communications.

[24]  Jeffrey H. Reed,et al.  Artificial Intelligence Defined 5G Radio Access Networks , 2018, IEEE Communications Magazine.

[25]  Yu Zheng,et al.  Deep Spatio-Temporal Residual Networks for Citywide Crowd Flows Prediction , 2016, AAAI.

[26]  Tianjian Chen,et al.  Federated Machine Learning: Concept and Applications , 2019 .

[27]  Yaohui Jin,et al.  Analyzing and modeling spatio-temporal dependence of cellular traffic at city scale , 2015, 2015 IEEE International Conference on Communications (ICC).

[28]  Hubert Eichner,et al.  Towards Federated Learning at Scale: System Design , 2019, SysML.

[29]  Yong Li,et al.  Big Data Driven Mobile Traffic Understanding and Forecasting: A Time Series Approach , 2016, IEEE Transactions on Services Computing.

[30]  B. Shihada,et al.  What should 6G be? , 2019, Nature Electronics.

[31]  Albert Y. Zomaya,et al.  Federated Learning over Wireless Networks: Optimization Model Design and Analysis , 2019, IEEE INFOCOM 2019 - IEEE Conference on Computer Communications.

[32]  Wei Chen,et al.  The Roadmap to 6G: AI Empowered Wireless Networks , 2019, IEEE Communications Magazine.

[33]  Oliver W. W. Yang,et al.  Wireless traffic modeling and prediction using seasonal ARIMA models , 2003, IEEE International Conference on Communications, 2003. ICC '03..

[34]  Jing Wang,et al.  Spatiotemporal modeling and prediction in cellular networks: A big data enabled deep learning approach , 2017, IEEE INFOCOM 2017 - IEEE Conference on Computer Communications.

[35]  Anit Kumar Sahu,et al.  Federated Optimization in Heterogeneous Networks , 2018, MLSys.