An adaptive AQM algorithm based on neuron reinforcement learning

In recent years, it has become an active research direction to develop adaptive and robust active queue management (AQM) scheme for congestion control of complex time-varying network. A novel adaptive AQM scheme based on Neuron Reinforcement Learning (NRL) is presented in this paper. This scheme uses queue length and link rate as congestion notification to determine an appropriate drop/mark probability, and the parameters of neuron can be adjusted online according to the time-varying network environment so that the stability of queue dynamics and robustness for fluctuation of TCP loads are guaranteed. This scheme is easy to implement with simple structure, and it is independent of the model of plant to be controlled. Simulation results show that this proposed algorithm is especially suitable for solving the complex network congestion control problem, and also has better stability and robustness.

[1]  Vishal Misra,et al.  Fluid-based analysis of a network of AQM routers supporting TCP flows with an application to RED , 2000, SIGCOMM.

[2]  P. Dayan.  The Convergence of TD(λ) for General λ , 2004, Machine Learning.

[3]  QUTdN QeO,et al.  Random early detection gateways for congestion avoidance , 1993, TNET.

[4]  Sally Floyd,et al.  Promoting the use of end-to-end congestion control in the Internet , 1999, TNET.

[5]  Steven H. Low,et al.  REM: active queue management , 2001, IEEE Netw..

[6]  Bo Li,et al.  LRED: A Robust and Responsive AQM Algorithm Using Packet Loss Ratio Measurement , 2007 .

[7]  Donald F. Towsley,et al.  On designing improved controllers for AQM routers supporting TCP flows , 2001, Proceedings IEEE INFOCOM 2001. Conference on Computer Communications. Twentieth Annual Joint Conference of the IEEE Computer and Communications Society (Cat. No.01CH37213).

[8]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[9]  Ahmed Mehaoua,et al.  A fuzzy logic-based AQM for real-time traffic over internet , 2007, Comput. Networks.