Assured end-to-end QoS through adaptive marking in multi-domain differentiated services networks

The issue of resource management in multi-domain Differentiated Services (DiffServ) networks has attracted a lot of attention from researchers who have proposed various provisioning, adaptive marking and admission control schemes. In this paper, we propose a Reinforcement Learning-based Adaptive Marking (RLAM) approach for providing assured end-to-end quality of service (QoS) in the form of end-to-end delay and throughput assurances, while minimizing packet transmission cost since 'expensive' Per Hop Behaviors like Expedited Forwarding (EF) are used only when necessary. The proposed scheme tries to satisfy per flow end-to-end QoS through control action,s which act on flow aggregates in the core of the network. Using an ns2 simulation of a multi-domain DiffServ network with multimedia traffic, the RLAM scheme is shown to be effective in significantly lowering packet transmission costs without sacrificing end-to-end QoS, when compared to the commonly used static marking scheme.

[1]  Van Jacobson,et al.  An Expedited Forwarding PHB , 1999, RFC.

[2]  Dilip D. Kandlur,et al.  Adaptive packet marking for maintaining end-to-end throughput in a differentiated-services internet , 1999, TNET.

[3]  Prasant Mohapatra,et al.  A random early demotion and promotion marker for assured services , 2000, IEEE Journal on Selected Areas in Communications.

[4]  Scott Shenker,et al.  Specification of Guaranteed Quality of Service , 1997, RFC.

[5]  Chen K. Tham,et al.  Reinforcement learning of multiple tasks using a hierarchical CMAC architecture , 1995, Robotics Auton. Syst..

[6]  John N. Tsitsiklis,et al.  Call admission control and routing in integrated services networks using neuro-dynamic programming , 2000, IEEE Journal on Selected Areas in Communications.

[7]  Timothy X. Brown,et al.  Adaptive call admission control under quality of service constraints: a reinforcement learning solution , 2000, IEEE Journal on Selected Areas in Communications.

[8]  David L. Black,et al.  The Addition of Explicit Congestion Notification (ECN) to IP , 2001, RFC.

[9]  Richard S. Sutton,et al.  Learning to predict by the methods of temporal differences , 1988, Machine Learning.

[10]  Ibrahim Khalil,et al.  Implementation of a bandwidth broker for dynamic end-to-end resource reservation in outsourced virtual private networks , 2000, Proceedings 25th Annual IEEE Conference on Local Computer Networks. LCN 2000.

[11]  Randy H. Katz,et al.  QoS provisioning using a clearing house architecture , 2000, 2000 Eighth International Workshop on Quality of Service. IWQoS 2000 (Cat. No.00EX400).

[12]  Fred Baker,et al.  Assured Forwarding PHB Group , 1999, RFC.

[13]  Philip Almquist,et al.  Type of Service in the Internet Protocol Suite , 1992, RFC.

[14]  Yong Liu,et al.  Minimizing Transmission Costs through Adaptive Marking in Differentiated Services Networks , 2002, MMNS.

[15]  David L. Black,et al.  Definition of the Differentiated Services Field (DS Field) in the IPv4 and IPv6 Headers , 1998, RFC.

[16]  Peter Dayan,et al.  Technical Note: Q-Learning , 2004, Machine Learning.

[17]  David L. Black,et al.  An Architecture for Differentiated Service , 1998 .

[18]  Peter Dayan,et al.  Q-learning , 1992, Machine Learning.

[19]  Simon Haykin,et al.  A dynamic channel assignment policy through Q-learning , 1999, IEEE Trans. Neural Networks.

[20]  John N. Tsitsiklis,et al.  Neuro-Dynamic Programming , 1996, Encyclopedia of Machine Learning.

[21]  Richard S. Sutton,et al.  Neuronlike adaptive elements that can solve difficult learning control problems , 1983, IEEE Transactions on Systems, Man, and Cybernetics.