Joint Demand Forecasting and DQN-Based Control for Energy-Aware Mobile Traffic Offloading

With the explosive growth in demand for mobile traffic, one of the promising solutions is to offload cellular traffic to small base stations for better system efficiency. Due to increasing system complexity, network operators are facing severe challenges and looking for machine learning-based solutions. In this work, we propose an energy-aware mobile traffic offloading scheme in the heterogeneous network jointly apply deep Q network (DQN) decision making and advanced traffic demand forecasting. The base station control model is trained and verified on an open dataset from a major telecom operator. The performance evaluation shows that DQN with traffic forecasting outperforms others at all levels of mobile traffic demands. Also, the advantage of accurate traffic prediction is more significant under higher traffic loads.

[1]  Xuemin Shen,et al.  Energy-Aware Traffic Offloading for Green Heterogeneous Networks , 2016, IEEE Journal on Selected Areas in Communications.

[2]  Muhammad Ali Imran,et al.  Challenges in 5G: how to empower SON with big data for enabling 5G , 2014, IEEE Network.

[3]  Yong Li,et al.  Big Data Driven Mobile Traffic Understanding and Forecasting: A Time Series Approach , 2016, IEEE Transactions on Services Computing.

[4]  Marco Pavone,et al.  Cellular Network Traffic Scheduling With Deep Reinforcement Learning , 2018, AAAI.

[5]  Chih-Wei Huang,et al.  A study of deep learning networks on mobile traffic forecasting , 2017, 2017 IEEE 28th Annual International Symposium on Personal, Indoor, and Mobile Radio Communications (PIMRC).

[6]  A. Liu,et al.  Characterizing and modeling internet traffic dynamics of cellular devices , 2011, PERV.

[7]  W. Marsden.  I and J , 2012 .

[8]  Hamid Aghvami,et al.  A survey on mobile data offloading: technical and business perspectives , 2013, IEEE Wireless Communications.

[9]  Xuemin Shen,et al.  Cloud assisted HetNets toward 5G wireless networks , 2015, IEEE Communications Magazine.

[10]  Qiang Liu,et al.  On Designing Energy-Efficient Heterogeneous Cloud Radio Access Networks , 2018, IEEE Transactions on Green Communications and Networking.

[11]  Jinsong Wu,et al.  Survey of Strategies for Switching Off Base Stations in Heterogeneous Networks for Greener 5G Systems , 2016, IEEE Access.

[12]  Shuguang Cui,et al.  Spatio-Temporal Wireless Traffic Prediction With Recurrent Neural Network , 2018, IEEE Wireless Communications Letters.

[13]  Balasubramaniam Natarajan,et al.  Small Cell Base Station Sleep Strategies for Energy Efficiency , 2016, IEEE Transactions on Vehicular Technology.

[14]  Zhu Han,et al.  Self-Organization in Small Cell Networks: A Reinforcement Learning Approach , 2013, IEEE Transactions on Wireless Communications.

[15]  Zhu Han,et al.  User Scheduling and Resource Allocation in HetNets With Hybrid Energy Supply: An Actor-Critic Reinforcement Learning Approach , 2018, IEEE Transactions on Wireless Communications.

[16]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[17]  Tijani Chahed,et al.  Optimal Control of Wake Up Mechanisms of Femtocells in Heterogeneous Networks , 2012, IEEE Journal on Selected Areas in Communications.

[18]  Dongfeng Yuan,et al.  Deep Transfer Learning for Intelligent Cellular Traffic Prediction Based on Cross-Domain Big Data , 2019, IEEE Journal on Selected Areas in Communications.

[19]  Marco De Nadai,et al.  A multi-source dataset of urban life in the city of Milan and the Province of Trentino , 2015, Scientific Data.

[20]  Zhu Han,et al.  Machine Learning Paradigms for Next-Generation Wireless Networks , 2017, IEEE Wireless Communications.

[21]  Zhifeng Zhao,et al.  The Learning and Prediction of Application-Level Traffic Data in Cellular Networks , 2016, IEEE Transactions on Wireless Communications.

[22]  Hyundong Shin,et al.  Energy Efficient Heterogeneous Cellular Networks , 2013, IEEE Journal on Selected Areas in Communications.

[23]  Vasilios A. Siris,et al.  Performance and energy efficiency of mobile data offloading with mobility prediction and prefetching , 2013, 2013 IEEE 14th International Symposium on "A World of Wireless, Mobile and Multimedia Networks" (WoWMoM).

[24]  Wenhao Huang,et al.  Deep Architecture for Traffic Flow Prediction: Deep Belief Networks With Multitask Learning , 2014, IEEE Transactions on Intelligent Transportation Systems.

[25]  Peter Dayan,et al.  Q-learning , 1992, Machine Learning.

[26]  Shane Legg,et al.  Human-level control through deep reinforcement learning , 2015, Nature.

[27]  Ismail Güvenç,et al.  Context-aware mobility management in HetNets: A reinforcement learning approach , 2015, 2015 IEEE Wireless Communications and Networking Conference (WCNC).

[28]  Jingjing Yao,et al.  QoS-Aware Joint BBU-RRH Mapping and User Association in Cloud-RANs , 2018, IEEE Transactions on Green Communications and Networking.

[29]  Fei-Yue Wang,et al.  Traffic Flow Prediction With Big Data: A Deep Learning Approach , 2015, IEEE Transactions on Intelligent Transportation Systems.

[30]  Jian Zhang,et al.  Traffic Offloading in Two-Tier Multi-Mode Small Cell Networks over Unlicensed Bands: A Hierarchical Learning Framework , 2015, KSII Trans. Internet Inf. Syst..

[31]  Yuval Tassa,et al.  Continuous control with deep reinforcement learning , 2015, ICLR.

[32]  Di Yuan,et al.  Data Offloading in Load Coupled Networks: A Utility Maximization Framework , 2014, IEEE Transactions on Wireless Communications.

[33]  Marco Ajmone Marsan,et al.  Multiple daily base station switch-offs in cellular networks , 2012, 2012 Fourth International Conference on Communications and Electronics (ICCE).

[34]  Zhifeng Zhao,et al.  The predictability of cellular networks traffic , 2012, 2012 International Symposium on Communications and Information Technologies (ISCIT).

[35]  Xianfu Chen,et al.  Energy-Efficiency Oriented Traffic Offloading in Wireless Networks: A Brief Survey and a Learning Approach for Heterogeneous Cellular Networks , 2015, IEEE Journal on Selected Areas in Communications.

[36]  Andreas Mitschele-Thiel,et al.  Cooperative Fuzzy Q-Learning for self-organized coverage and capacity optimization , 2012, 2012 IEEE 23rd International Symposium on Personal, Indoor and Mobile Radio Communications - (PIMRC).

[37]  Jamil Salem Barbar,et al.  Computer network traffic prediction: a comparison between traditional and deep learning neural networks , 2015, Int. J. Big Data Intell..