Optimal control of queueing networks: an approach via fluid models

We consider a general control problem for networks with linear dynamics which includes the special cases of scheduling in multiclass queueing networks and routeing problems. The fluid approximation of the network is used to derive new results about the optimal control for the stochastic network. The main emphasis lies on the average-cost criterion; however, the β-discounted as well as the finite-cost problems are also investigated. One of our main results states that the fluid problem provides a lower bound to the stochastic network problem. For scheduling problems in multiclass queueing networks we show the existence of an average-cost optimal decision rule, if the usual traffic conditions are satisfied. Moreover, we give under the same conditions a simple stabilizing scheduling policy. Another important issue that we address is the construction of simple asymptotically optimal decision rules. Asymptotic optimality is here seen with respect to fluid scaling. We show that every minimizer of the optimality equation is asymptotically optimal and, what is more important for practical purposes, we outline a general way to identify fluid optimal feedback rules as asymptotically optimal. Last, but not least, for routeing problems an asymptotically optimal decision rule is given explicitly, namely a so-called least-loaded-routeing rule.

[1]  Linn I. Sennott,et al.  Average Cost Optimal Stationary Policies in Infinite State Markov Decision Processes with Unbounded Costs , 1989, Oper. Res..

[2]  Sean P. Meyn Stability and optimization of queueing networks and their fluid models , 2003 .

[3]  Arie Hordijk,et al.  Fluid approximation of a controlled multiclass tandem network , 2000, Queueing Syst. Theory Appl..

[4]  Sean P. Meyn The policy iteration algorithm for average reward Markov decision processes with general state space , 1997, IEEE Trans. Autom. Control..

[5]  C. Maglaras Discrete-review policies for scheduling stochastic networks: trajectory tracking and fluid-scale asymptotic optimality , 2000 .

[6]  Michael H. Veatch,et al.  Fluid analysis of arrival routing , 2001, IEEE Trans. Autom. Control..

[7]  Nicole Bäuerle,et al.  Discounted Stochastic Fluid Programs , 2001, Math. Oper. Res..

[8]  Sean P. Meyn Sequencing and Routing in Multiclass Queueing Networks Part I: Feedback Regulation , 2001, SIAM J. Control. Optim..

[9]  Constantinos Maglaras,et al.  Dynamic scheduling in multiclass queueing networks: Stability under discrete-review policies , 1999, Queueing Syst. Theory Appl..

[10]  Knut Sydsæter,et al.  Optimal control theory with economic applications , 1987 .

[11]  Bruce E. Hajek,et al.  Analysis of Simple Algorithms for Dynamic Load Balancing , 1997, Math. Oper. Res..

[12]  J. Dai On Positive Harris Recurrence of Multiclass Queueing Networks: A Unified Approach Via Fluid Limit Models , 1995 .

[13]  Upendra Dave,et al.  Applied Probability and Queues , 1987 .

[14]  I SennottLinn Average Cost Optimal Stationary Policies in Infinite State Markov Decision Processes with Unbounded Costs , 1989 .

[15]  Hong Chen Fluid Approximations and Stability of Multiclass Queueing Networks: Work-Conserving Disciplines , 1995 .

[16]  B. Hajek Optimal control of two interacting service stations , 1982, 1982 21st IEEE Conference on Decision and Control.

[17]  Nicole Bäuerle,et al.  How to improve the performance of ATM multiplexers , 1999, Oper. Res. Lett..

[18]  Hong Chen,et al.  On the asymptote of the optimal routing policy for two service stations , 1993, IEEE Trans. Autom. Control..

[19]  D. Bertsimas,et al.  A New Algorithm for State-Constrained Separated Continuous Linear Programs , 1999 .

[20]  J. George Shanthikumar,et al.  Discrete storage processes and their Poisson flow and fluid flow approximations , 1996, Queueing Syst. Theory Appl..

[21]  L. Sennott Stochastic Dynamic Programming and the Control of Queueing Systems , 1998 .

[22]  Ger Koole,et al.  On the Assignment of Customers to Parallel Queues , 1992, Probability in the Engineering and Informational Sciences.

[23]  Ruth J. Williams,et al.  Some Recent Developments for Queueing Networks , 1998 .

[24]  Shaler Stidham,et al.  A survey of Markov decision models for control of networks of queues , 1993, Queueing Syst. Theory Appl..

[25]  Florin Avram,et al.  Fluid models of sequencing problems in open queueing networks; an optimal control approach , 1995 .

[26]  Nicole Bäuerle,et al.  Optimal control of single-server fluid networks , 2000, Queueing Syst. Theory Appl..

[27]  N. Bäuerle Asymptotic optimality of tracking policies in stochastic networks , 2001 .

[28]  Hong Chen,et al.  Performance evaluation of scheduling control of queueing networks: Fluid model heuristics , 1995, Queueing Syst. Theory Appl..

[29]  V. Rykov,et al.  Controlled Queueing Systems , 1995 .

[30]  Eitan Altman,et al.  ON THE COMPARISON OF QUEUEING SYSTEMS WITH THEIR FLUID LIMITS , 2001, Probability in the Engineering and Informational Sciences.

[31]  A. W. Kemp,et al.  Applied Probability and Queues , 1989 .

[32]  M. Pullan Forms of Optimal Solutions for Separated Continuous Linear Programs , 1995 .

[33]  Ruth J. Williams,et al.  Existence and Uniqueness of Semimartingale Reflecting Brownian Motions in Convex Polyhedrons , 1996 .