Diffusion approximations for controlled stochastic networks: An asymptotic bound for the value function

We consider the scheduling control problem for a family of unitary networks under heavy traffic, with general interarrival and service times, probabilistic routing and infinite horizon discounted linear holding cost. A natural nonanticipativity condition for admissibility of control policies is introduced. The condition is seen to hold for a broad class of problems. Using this formulation of admissible controls and a time-transformation technique, we establish that the infimum of the cost for the network control problem over all admissible sequencing control policies is asymptotically bounded below by the value function of an associated diffusion control problem (the Brownian control problem). This result provides a useful bound on the best achievable performance for any admissible control policy for a wide class of networks.

[1]  Maury Bramson,et al.  Two Workload Properties for Brownian Networks , 2003, Queueing Syst. Theory Appl..

[2]  Volker Böhm.  On the continuity of the optimal policy set for linear programs , 1975 .

[3]  Ruth J. Williams,et al.  Existence and Uniqueness of Semimartingale Reflecting Brownian Motions in Convex Polyhedrons , 1996 .

[4]  H. Kushner,et al.  Optimal and approximately optimal control policies for queues in heavy traffic , 1989 .

[5]  R. Durrett.  Probability: Theory and Examples , 1993 .

[6]  Xiongzhi Chen.  Brownian Motion and Stochastic Calculus , 2008 .

[7]  J. Michael Harrison,et al.  Brownian Models of Queueing Networks with Heterogeneous Customer Populations , 1988 .

[8]  Jan A. Van Mieghem,et al.  Dynamic Control of Brownian Networks: State Space Collapse and Equivalent Workload Formulations , 1997 .

[9]  S. Sushanth Kumar,et al.  Heavy traffic analysis of open processing networks with complete resource pooling: Asymptotic optimality of discrete review policies , 2005, math/0503477.

[10]  J. Harrison.  Brownian models of open processing networks: canonical representation of workload , 2000 .

[11]  P. Protter.  Stochastic integration and differential equations , 1990 .

[12]  Sean P. Meyn.  Sequencing and Routing in Multiclass Queueing Networks Part II: Workload Relaxations , 2003, SIAM J. Control. Optim..

[13]  Amarjit Budhiraja,et al.  A large deviations approach to asymptotically optimal control of crisscross network in heavy traffic , 2005 .

[14]  Ruth J. Williams,et al.  On dynamic scheduling of stochastic networks in heavy traffic and some new results for the workload process , 2000, Proceedings of the 39th IEEE Conference on Decision and Control (Cat. No.00CH37187).

[15]  S. Ethier,et al.  Markov Processes: Characterization and Convergence , 2005 .

[16]  L. F. Martins,et al.  Heavy Traffic Analysis of a Controlled Multiclass Queueing Network via Weak Convergence Methods , 1996 .

[17]  Richard H. Stockbridge,et al.  Martingale problems and linear programs for singular control ∗ , 1999 .

[18]  Thomas G. Kurtz,et al.  Random Time Changes and Convergence in Distribution Under the Meyer-Zheng Conditions , 1991 .

[19]  Ronald J. Williams,et al.  Dynamic scheduling of a system with two parallel servers in heavy traffic with resource pooling: asymptotic optimality of a threshold policy , 2001 .

[20]  P. Billingsley,et al.  Convergence of Probability Measures , 1970, The Mathematical Gazette.

[21]  H. Thorisson.  Coupling, stationarity, and regeneration , 2000 .