Infinite horizon optimal impulsive control with applications to Internet congestion control

We investigate infinite-horizon deterministic optimal control problems with both gradual and impulsive controls, where any finitely many impulses are allowed simultaneously. Both discounted and long-run time-average criteria are considered. We establish very general and at the same time natural conditions, under which the dynamic programming approach results in an optimal feedback policy. The established theoretical results are applied to the Internet congestion control, and by solving analytically and nontrivially the underlying optimal control problems, we obtain a simple threshold-based active queue management scheme, which takes into account the main parameters of the transmission control protocols, and improves the fairness among the connections in a given network.

[1]  K. H. Wong,et al.  Optimal Impulsive Control Problem with Application to Human Immunodeficiency Virus Treatment , 2011, J. Optim. Theory Appl..

[2]  Van Jacobson,et al.  Random early detection gateways for congestion avoidance , 1993, TNET.

[3]  A. Hordijk,et al.  Average optimal policies in Markov decision drift processes with applications to a queueing and a replacement model , 1983, Advances in Applied Probability.

[4]  Jean C. Walrand,et al.  Fair end-to-end window-based congestion control , 2000, TNET.

[5]  Arnaud Dupas,et al.  Semantic networking: Flow-based, traffic-aware, and self-managed networking , 2009, Bell Labs Technical Journal.

[6]  Boris M. Miller,et al.  Impulsive Control in Continuous and Discrete-Continuous Systems , 2003 .

[7]  Alexander B. Miller,et al.  Control of connected Markov chains. Application to congestion avoidance in the Internet , 2011, IEEE Conference on Decision and Control and European Control Conference.

[8]  Konstantin Avrachenkov,et al.  Convergence of trajectories and optimal buffer sizing for MIMD congestion control , 2010, Comput. Commun..

[9]  Frank Kelly,et al.  Rate control for communication networks: shadow prices, proportional fairness and stability , 1998, J. Oper. Res. Soc..

[10]  I. M. Pyshik,et al.  Table of integrals, series, and products , 1965 .

[11]  Cheng Jin,et al.  FAST TCP: Motivation, Architecture, Algorithms, Performance , 2006, IEEE/ACM Transactions on Networking.

[12]  A. A. Yushkevich,et al.  Verification Theorems for Markov Decision Processes with Controlled Deterministic Drift and Gradual and Impulsive Controls , 1990 .

[13]  Peter E. Caines,et al.  On the Optimal Control of Impulsive Hybrid Systems on Riemannian Manifolds , 2012, SIAM J. Control. Optim..

[14]  L. Stettner,et al.  Impulsive Control of Portfolios , 2007 .

[15]  N. Moller,et al.  Inter-protocol fairness between TCP New Reno and TCP Westwood , 2007, 2007 Next Generation Internet Networks.

[16]  M. Bardi,et al.  Optimal Control and Viscosity Solutions of Hamilton-Jacobi-Bellman Equations , 1997 .

[17]  X. Xu,et al.  Impulsive control in continuous and discrete-continuous systems [Book Reviews] , 2003, IEEE Transactions on Automatic Control.

[18]  Denis Weaire,et al.  The differential equation , 1985 .

[19]  F. Rampazzo,et al.  Dynamic Programming for Nonlinear Systems Driven by Ordinaryand Impulsive Controls , 1996 .

[20]  Alexei B. Piunovskiy,et al.  Multicriteria impulsive control of jump Markov processes , 2004, Math. Methods Oper. Res..

[21]  Richelle V. Adams,et al.  Active Queue Management: A Survey , 2013, IEEE Communications Surveys & Tutorials.

[22]  João Pedro Hespanha,et al.  Hybrid Modeling of TCP Congestion Control , 2001, HSCC.

[23]  Eitan Altman,et al.  c ○ 2005 Springer Science + Business Media, Inc. Manufactured in The Netherlands. Fairness in MIMD Congestion Control Algorithms , 2022 .

[24]  Konstantin Avrachenkov,et al.  Convergence of trajectories and optimal buffer sizing for AIMD congestion control , 2010, Perform. Evaluation.

[25]  Sally Floyd,et al.  HighSpeed TCP for Large Congestion Windows , 2003, RFC.

[26]  Felix Schlenk,et al.  Proof of Theorem 3 , 2005 .

[27]  Robert Shorten,et al.  Experimental Evaluation of TCP Protocols for High-Speed Networks , 2007, IEEE/ACM Transactions on Networking.

[28]  R. Srikant,et al.  End-to-end congestion control schemes: utility functions, random losses and ECN marks , 2000, Proceedings IEEE INFOCOM 2000. Conference on Computer Communications. Nineteenth Annual Joint Conference of the IEEE Computer and Communications Societies (Cat. No.00CH37064).

[29]  Kerstin Vogler,et al.  Table Of Integrals Series And Products , 2016 .

[30]  Steven H. Low,et al.  Optimization flow control—I: basic algorithm and convergence , 1999, TNET.

[31]  Mark H. Davis.  Markov Models and Optimization , 1995 .

[32]  Boris M. Miller,et al.  Necessary conditions for optimal singular stochastic control problems , 2007 .

[33]  Daizhan Cheng,et al.  Optimal impulsive control in periodic ecosystem , 2006, Syst. Control. Lett..

[34]  Ralf Korn,et al.  Some applications of impulse control in mathematical finance , 1999, Math. Methods Oper. Res..

[35]  Alain Bensoussan,et al.  Impulse Control and Quasi-Variational Inequalities , 1984 .

[36]  Tom Kelly,et al.  Scalable TCP: improving performance in highspeed wide area networks , 2003, CCRV.

[37]  Soren Christensen,et al.  On the solution of general impulse control problems using superharmonic functions , 2013, 1304.3578.