Optimization of queuing system via stochastic control

The problem of access and service rate control in queuing systems as a general optimization problem for controlled Markov process with finite state space is considered. By using the dynamic programming approach we obtain the explicit form of the optimal control in the case of minimizing cost given as a mixture of an average queue length, number of lost jobs, and service resources. The problem is considered on a finite time interval in the case of nonstationary input flow. In this case we suggest the general procedure of the numerical solution which can be applied to a problems with constraints.

[1]  E. Coddington,et al.  Theory of Ordinary Differential Equations , 1955 .

[2]  Konstantin Avrachenkov,et al.  Flow Control as a Stochastic Optimal Control Problem with Incomplete Information , 2005, Probl. Inf. Transm..

[3]  Robert J. Elliott,et al.  A partially observed control problem for Markov chains , 1992 .

[4]  E. Wong,et al.  Stochastic Processes in Engineering Systems , 1984 .

[5]  Alʹbert Nikolaevich Shiri︠a︡ev,et al.  Statistics of random processes , 1977 .

[6]  Fernando Paganini,et al.  Internet congestion control , 2002 .

[7]  V. A. Vasenin,et al.  Mathematical Models of Traffic Control in Internet: New Approaches Based of TCP/AQM Schemes , 2005 .

[8]  Mark H. Davis.  Markov Models and Optimization , 1995 .

[9]  B. L. Miller.  Finite state continuous time Markov decision processes with an infinite planning horizon , 1968 .

[10]  B. L. Miller.  Finite State Continuous Time Markov Decision Processes with a Finite Planning Horizon , 1968 .

[11]  A. Hordijk,et al.  Constrained admission control to a queueing system , 1989, Advances in Applied Probability.

[12]  John B. Moore,et al.  Hidden Markov Models: Estimation and Control , 1994 .

[13]  R. Serfozo.  Optimal control of random walks, birth and death processes, and queues , 1981, Advances in Applied Probability.

[14]  Alexei B. Piunovskiy.  Bicriteria Optimization of a Queue with a Controlled Input Stream , 2004, Queueing Syst. Theory Appl..

[15]  Mark H. A. Davis,et al.  Existence of Optimal Controls for Stochastic Jump Processes , 1979 .

[16]  P. Brémaud,et al.  Optimal Thinning of a Point Process , 1979 .

[17]  P. Varaiya,et al.  Optimal Control of Jump Processes , 1977 .

[18]  Frank Kelly,et al.  Rate control for communication networks: shadow prices, proportional fairness and stability , 1998, J. Oper. Res. Soc..

[19]  Robert J. Elliott,et al.  Optimal control of a jump process , 1977 .

[20]  P. Brémaud.  Point processes and queues, martingale dynamics , 1983 .