Deep Reinforcement Learning for Router Selection in Network With Heavy Traffic

The rapid development of wireless communications brings a tremendous increase in the amount number of data streams and poses significant challenges to the traditional routing protocols. In this paper, we leverage deep reinforcement learning (DRL) for router selection in the network with heavy traffic, aiming at reducing the network congestion and the length of the data transmission path. We first illustrate the challenges of the existing routing protocols when the amount of the data explodes. We then utilize the Markov decision process (RSMDP) to formulate the routing problem. Two novel deep Q network (DQN)-based algorithms are designed to reduce the network congestion probability with a short transmission path: one focusing on reducing the congestion probability; while the other focuses on shortening the transmission path. The simulation results demonstrate that the proposed algorithms can achieve higher network throughput comparing to existing routing algorithms in heavy network traffic scenarios.

[1]  Gordon T. Wilfong,et al.  The stable paths problem and interdomain routing , 2002, TNET.

[2]  Geoffrey Ye Li,et al.  Deep Learning-Based Channel Estimation for Beamspace mmWave Massive MIMO Systems , 2018, IEEE Wireless Communications Letters.

[3]  Tobias Weber,et al.  Multi-Agent Reinforcement Learning for Energy Harvesting Two-Hop Communications with Full Cooperation , 2017, ArXiv.

[4]  Jeffrey G. Andrews,et al.  What Will 5G Be? , 2014, IEEE Journal on Selected Areas in Communications.

[5]  Shi Jin,et al.  Deep Learning for Massive MIMO CSI Feedback , 2017, IEEE Wireless Communications Letters.

[6]  Ah Chung Tsoi,et al.  Face recognition: a convolutional neural-network approach , 1997, IEEE Trans. Neural Networks.

[7]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[8]  Shane Legg,et al.  Human-level control through deep reinforcement learning , 2015, Nature.

[9]  Timo Aila,et al.  Pruning Convolutional Neural Networks for Resource Efficient Inference , 2016, ICLR.

[10]  Mikkel Thorup,et al.  Optimizing OSPF/IS-IS weights in a changing world , 2002, IEEE J. Sel. Areas Commun..

[11]  Ali Farhadi,et al.  You Only Look Once: Unified, Real-Time Object Detection , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[12]  Yujing Hu,et al.  Reinforcement Learning to Rank in E-Commerce Search Engine: Formalization, Analysis, and Application , 2018, KDD.

[13]  Hamed Haddadi,et al.  Deep Learning in Mobile and Wireless Networking: A Survey , 2018, IEEE Communications Surveys & Tutorials.

[14]  Stephan ten Brink,et al.  On deep learning-based channel decoding , 2017, 2017 51st Annual Conference on Information Sciences and Systems (CISS).

[15]  Kilian Q. Weinberger,et al.  Densely Connected Convolutional Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[16]  Ross B. Girshick,et al.  Fast R-CNN , 2015, 1504.08083.

[17]  Ali Farhadi,et al.  YOLO9000: Better, Faster, Stronger , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[18]  Robert W. Heath,et al.  Five disruptive technology directions for 5G , 2013, IEEE Communications Magazine.

[19]  Jakob Hoydis,et al.  An Introduction to Deep Learning for the Physical Layer , 2017, IEEE Transactions on Cognitive Communications and Networking.

[20]  Nei Kato,et al.  The Deep Learning Vision for Heterogeneous Network Traffic Control: Proposal, Challenges, and Future Perspective , 2017, IEEE Wireless Communications.

[21]  Alex Graves,et al.  Playing Atari with Deep Reinforcement Learning , 2013, ArXiv.

[22]  Trevor Darrell,et al.  Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation , 2013, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[23]  Xiaohu You,et al.  AI for 5G: research directions and paradigms , 2018, Science China Information Sciences.

[24]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[25]  Shuguang Cui,et al.  Reinforcement Learning Based Multi-Access Control with Energy Harvesting , 2018, 2018 IEEE Global Communications Conference (GLOBECOM).

[26]  Theodore S. Rappaport,et al.  Millimeter Wave Mobile Communications for 5G Cellular: It Will Work! , 2013, IEEE Access.

[27]  Nei Kato,et al.  On Removing Routing Protocol from Future Wireless Networks: A Real-time Deep Learning Approach for Intelligent Traffic Control , 2018, IEEE Wireless Communications.

[28]  Tom Schaul,et al.  Dueling Network Architectures for Deep Reinforcement Learning , 2015, ICML.

[29]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[30]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[31]  Alex Graves,et al.  Asynchronous Methods for Deep Reinforcement Learning , 2016, ICML.

[32]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[33]  Charles L. Hedrick,et al.  Routing Information Protocol , 1988, RFC.

[34]  Xiqi Gao,et al.  Cellular architecture and key technologies for 5G wireless communication networks , 2014, IEEE Communications Magazine.