Neural Net Based Approach for Adaptive Routing Policy in Telecommunication Networks

This paper deals with Quality of Service-Routing (QoS-routing) in telecommunication networks with irregular traffics. The prediction of congestion’s problems in the networks in real time is quite difficult, making the effectiveness of ”traditional” methodologies based on analytical models questionable. An adaptive routing algorithm called Q-Neural Routing is proposed here. Compared to standard Q-Routing, the Q-value is approximated by a reinforcement learning based neural network of a fixed size, allowing the learner to incorporate various parameters such as local queue size and time of day, into its distance estimation. Moreover, each router uses an on line learning module to optimize the path in terms of average packet delivery time, by taking into account the waiting queue states of neighboring routers. The performance of the proposed algorithm is evaluated experimentally with OPNET simulator for different levels of load and compared to standard RIP and Q-Routing algorithms.

[1]  John Moy,et al.  OSPF Version 2 , 1998, RFC.

[2]  Eric S. Crawley,et al.  A Framework for QoS-based Routing in the Internet , 1998, RFC.

[3]  Esther Levin.  Hidden control neural architecture modeling of nonlinear time varying systems and its applications , 1993, IEEE Trans. Neural Networks.

[4]  Craig Partridge,et al.  A Proposed Flow Specification , 1992, RFC.

[5]  Eric C. Rosen,et al.  Multiprotocol Label Switching Architecture , 2001, RFC.

[6]  I. W. Habib.  Applications of neurocomputing in traffic management of ATM networks , 1996 .

[7]  Francis Reichmeyer,et al.  Requirements of Diff-serv Boundary Routers , 1998 .

[8]  Erol Gelenbe,et al.  Towards Networks with Cognitive Packets , 2001 .

[9]  Peter Dayan,et al.  Technical Note: Q-Learning , 2004, Machine Learning.

[10]  Kang G. Shin,et al.  Understanding TCP dynamics in an integrated services Internet , 1997, Proceedings of 7th International Workshop on Network and Operating System Support for Digital Audio and Video (NOSSDAV '97).

[11]  Michael L. Littman,et al.  Packet Routing in Dynamically Changing Networks: A Reinforcement Learning Approach , 1993, NIPS.

[12]  Serge Fdida,et al.  A scalable algorithm for link-state QoS-based routing with three metrics , 2001, ICC 2001. IEEE International Conference on Communications. Conference Record (Cat. No.01CH37240).

[13]  Kang G. Shin,et al.  Adaptive-weighted packet scheduling for premium service , 2001, ICC.

[14]  Devika Subramanian,et al.  Ants and Reinforcement Learning: A Case Study in Routing in Dynamic Networks , 1997, IJCAI.

[15]  Alex Aussem,et al.  Neural-based queueing system modelling for service quality estimation in B-ISDN networks , 1999 .

[16]  Richard Bellman,et al.  ON A ROUTING PROBLEM , 1958 .

[17]  J. Turner,et al.  New directions in communications (or which way to the information age?) , 1986, IEEE Communications Magazine.

[18]  P. Gallinari,et al.  Discriminative training for improved neural prediction systems , 1994, Proceedings of ICASSP '94. IEEE International Conference on Acoustics, Speech and Signal Processing.

[19]  Deborah Estrin,et al.  RSVP: a new resource ReSerVation Protocol , 1993 .

[20]  Scott Shenker,et al.  Specification of Guaranteed Quality of Service , 1997, RFC.

[21]  Risto Miikkulainen,et al.  On-Line Adaptation of a Signal Predistorter through Dual Reinforcement Learning , 1996, ICML.

[22]  Luis B. Almeida,et al.  Neural networks in B-ISDN flow control: ATM traffic prediction or network modeling? , 1995 .

[23]  Andrew G. Barto,et al.  Reinforcement learning , 1998 .

[24]  Abdelhamid Mellouk,et al.  A discriminative neural prediction system for speech recognition , 1993, 1993 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[25]  Kathleen Nichols,et al.  Differentiated Services Operational Model and Definitions , 1998 .