AI Routers & Network Mind: A Hybrid Machine Learning Paradigm for Packet Routing

With the increasing complexity of network topologies and architectures, adding intelligence to the network control plane through Artificial Intelligence and Machine Learning (AI&ML) is becoming a trend in network development. For large-scale geo-distributed systems, determining how to appropriately introduce intelligence in networking is the key to high-efficiency operation. In this treatise, we explore two deployment paradigms (centralized vs. distributed) for AI-based networking. To achieve the best results, we propose a hybrid ML paradigm that combines a distributed intelligence, based on units called "AI routers," with a centralized intelligence, called the "network mind", to support different network services. In the proposed paradigm, we deploy centralized AI control for connection-oriented tunneling-based routing protocols (such as multiprotocol label switching and segment routing) to guarantee a high QoS, whereas for hop-by-hop IP routing, we shift the intelligent control responsibility to each AI router to ease the overhead imposed by centralized control and use the network mind to improve the global convergence.

[1]  Mr. Jamal Mhawesh Challab.  Adaptive Opportunistic Routing For Wireless AD HOC Networks , 2016 .

[2]  Michael L. Littman,et al.  Packet Routing in Dynamically Changing Networks: A Reinforcement Learning Approach , 1993, NIPS.

[3]  Pin-Han Ho,et al.  ARBR: Adaptive reinforcement-based routing for DTN , 2010, 2010 IEEE 6th International Conference on Wireless and Mobile Computing, Networking and Communications.

[4]  Martin Weiß,et al.  An improvement of the convergence proof of the ADAM-Optimizer , 2018, ArXiv.

[5]  Peter Stone.  TPOT-RL Applied to Network Routing , 2000, ICML.

[6]  Jim Dowling,et al.  Using feedback in collaborative reinforcement learning to adaptively optimize MANET routing , 2005, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[7]  Ting Wang,et al.  Adaptive Routing for Sensor Networks using Reinforcement Learning , 2006, The Sixth IEEE International Conference on Computer and Information Technology (CIT'06).

[8]  Jean C. Walrand,et al.  Knowledge-Defined Networking: Modelització de la xarxa a través de l’aprenentatge automàtic i la inferència , 2016 .

[9]  Shailesh Kumar and Risto Miikkulainen.  Dual Reinforcement Q-Routing: An On-Line Adaptive Routing Algorithm , 1997 .

[10]  Ian F. Akyildiz,et al.  QoS-Aware Adaptive Routing in Multi-layer Hierarchical Software Defined Networks: A Reinforcement Learning Approach , 2016, 2016 IEEE International Conference on Services Computing (SCC).

[11]  Zhu Han,et al.  Machine Learning Paradigms for Next-Generation Wireless Networks , 2017, IEEE Wireless Communications.

[12]  Kagan Tumer,et al.  Using Collective Intelligence to Route Internet Traffic , 1998, NIPS.

[13]  Manuela M. Veloso,et al.  Team-partitioned, opaque-transition reinforcement learning , 1999, AGENTS '99.

[14]  J. Cid-Sueiro,et al.  Q-Probabilistic Routing in Wireless Sensor Networks , 2007, 2007 3rd International Conference on Intelligent Sensors, Sensor Networks and Information.

[15]  David D. Clark,et al.  A knowledge plane for the internet , 2003, SIGCOMM '03.

[16]  H. Vincent Poor,et al.  A Secure Mobile Crowdsensing Game With Deep Reinforcement Learning , 2018, IEEE Transactions on Information Forensics and Security.

[17]  KyoungSoo Park,et al.  APUNet: Revitalizing GPU as Packet Processing Accelerator , 2017, NSDI.

[18]  Albert Cabellos-Aparicio,et al.  A Deep-Reinforcement Learning Approach for Software-Defined Networking Routing Optimization , 2017, ArXiv.

[19]  Xu Chen,et al.  A novel reinforcement learning algorithm for virtual network embedding , 2018, Neurocomputing.

[20]  Sam Devlin,et al.  Resource Abstraction for Reinforcement Learning in Multiagent Congestion Problems , 2016, AAMAS.

[21]  Dit-Yan Yeung,et al.  Predictive Q-Routing: A Memory-based Reinforcement Learning Approach to Adaptive Traffic Control , 1995, NIPS.

[22]  Haipeng Yao,et al.  NetworkAI: An Intelligent Network Architecture for Self-Learning Control Strategies in Software Defined Networks , 2018, IEEE Internet of Things Journal.

[23]  Manuela M. Veloso,et al.  Team-Partitioned, Opaque-Transition Reinforced Learning , 1998, RoboCup.

[24]  Shahid Mumtaz,et al.  An Efficient Edge Artificial Intelligence MultiPedestrian Tracking Method With Rank Constraint , 2019, IEEE Transactions on Industrial Informatics.

[25]  Xin Wang,et al.  Machine Learning for Networking: Workflow, Advances and Opportunities , 2017, IEEE Network.

[26]  Sherali Zeadally,et al.  Data collection using unmanned aerial vehicles for Internet of Things platforms , 2019, Comput. Electr. Eng..

[27]  Yuval Tassa,et al.  Continuous control with deep reinforcement learning , 2015, ICLR.