The achievable region approach to the optimal control of stochastic systems

The achievable region approach seeks solutions to stochastic optimization problems by characterizing the space of all possible performances (the achievable region) of the system of interest and optimizing the overall system‐wide performance objective over this space. This is radically different from conventional formulations based on dynamic programming. The approach is explained with reference to a simple two‐class queueing system. Powerful new methodologies due to the authors and co‐workers are deployed to analyse a general multi‐class queuing system with parallel servers and then to develop an approach to optimal load distribution across a network of interconnected stations. Finally, the approach is used for the first time to analyse a class of intensity control problems.

[1]  A. S. Manne.  Linear Programming and Sequential Decisions , 1960 .

[2]  Dong-Wan Tcha,et al.  Optimal Control of Single-Server Queuing Networks and Multi-Class M/G/1 Queues with Feedback , 1977, Oper. Res..

[3]  David M. Kreps,et al.  Martingales and arbitrage in multiperiod securities markets , 1979 .

[4]  J. Gittins.  Bandit processes and dynamic allocation indices , 1979 .

[5]  B. Heimann,et al.  Fleming, W. H./Rishel, R. W., Deterministic and Stochastic Optimal Control. New York‐Heidelberg‐Berlin. Springer‐Verlag. 1975. XIII, 222 S, DM 60,60 , 1979 .

[6]  Edward G. Coffman,et al.  A Characterization of Waiting Time Performance Realizable by Single-Server Queues , 1980, Oper. Res..

[7]  P. Whittle.  Multi‐Armed Bandits and the Gittins Index , 1980 .

[8]  Erol Gelenbe,et al.  Analysis and Synthesis of Computer Systems , 1980 .

[9]  R. Weber.  Scheduling jobs by stochastic processing requirements on parallel machines to minimize makespan or flowtime , 1982, Journal of Applied Probability.

[10]  K. Glazebrook.  On a sufficient condition for superprocesses due to whittle , 1982, Journal of Applied Probability.

[11]  D. R. Robinson.  Algorithms for evaluating the dynamic allocation index , 1982, Oper. Res. Lett..

[12]  Asser N. Tantawi,et al.  A General Model for Optimal Static Load Balancing in Star Network Configurations , 1984, Performance.

[13]  Asser N. Tantawi,et al.  Optimal static load balancing in distributed computer systems , 1985, JACM.

[14]  Stanley R. Pliska,et al.  A Stochastic Calculus Model of Continuous Trading: Optimal Portfolios , 1986, Math. Oper. Res..

[15]  Guy Pujolle,et al.  Introduction to queueing networks , 1987 .

[16]  References to discussion , 1988 .

[17]  P. Whittle.  Restless bandits: activity allocation in a changing world , 1988, Journal of Applied Probability.

[18]  J. Ben Atkinson,et al.  An Introduction to Queueing Networks , 1988 .

[19]  P. Whittle.  Restless Bandits: Activity Allocation in a Changing World , 1988 .

[20]  Gideon Weiss,et al.  Branching Bandit Processes , 1988, Probability in the Engineering and Informational Sciences.

[21]  Lawrence M. Wein,et al.  Scheduling networks of queues: Heavy traffic analysis of a simple open network , 1989, Queueing Syst. Theory Appl..

[22]  Christian M. Ernst,et al.  Multi-armed Bandit Allocation Indices , 1989 .

[23]  David D. Yao,et al.  Decentralized control of service rates in a closed Jackson network , 1989 .

[24]  Ruth J. Williams,et al.  Brownian models of multiclass queueing networks , 1990, 29th IEEE Conference on Decision and Control.

[25]  J. Bather,et al.  Multi‐Armed Bandit Allocation Indices , 1990 .

[26]  David D. Yao,et al.  Optimal intensity control of a queueing system with state-dependent capacity limit , 1990 .

[27]  R. Weber,et al.  ON AN INDEX POLICY FOR RESTLESS BANDITS , 1990 .

[28]  David D. Yao,et al.  Optimal load balancing and scheduling in a distributed computer system , 1991, JACM.

[29]  S. Tijs,et al.  LP-games and combinatorial optimization games , 1992 .

[30]  Z. Liu,et al.  Optimality of the round-robin routing policy , 1992, Journal of Applied Probability.

[31]  Gideon Weiss,et al.  Turnpike Optimality of Smith's Rule in Parallel Machines Stochastic Scheduling , 1992, Math. Oper. Res..

[32]  David D. Yao,et al.  Multiclass Queueing Systems: Polymatroidal Structure and Optimal Scheduling Control , 1992, Oper. Res..

[33]  David D. Yao,et al.  Rearrangement, Majorization and Stochastic Scheduling , 1993, Math. Oper. Res..

[34]  J. Michael Harrison,et al.  Brownian models of multiclass queueing networks: Current status and open problems , 1993, Queueing Syst. Theory Appl..

[35]  Maurice Queyranne,et al.  Structure of a simple scheduling polyhedron , 1993, Math. Program..

[36]  C. N. Laws,et al.  Asymptotically Optimal Loss Network Control , 1993, Math. Oper. Res..

[37]  Erol Gelenbe,et al.  Load Balancing Pragmatics , 1993 .

[38]  David D. Yao,et al.  Monotone Optimal Control of Permutable GSMPs , 1994, Math. Oper. Res..

[39]  Sean P. Meyn,et al.  Duality and linear programs for stability and performance analysis of queueing networks and scheduling policies , 1994, Proceedings of 1994 33rd IEEE Conference on Decision and Control.

[40]  Richard J. Gibbens,et al.  Network Programming Methods for Loss Networks (Invited Paper) , 1995, IEEE J. Sel. Areas Commun..

[41]  Dimitri P. Bertsekas,et al.  Nonlinear Programming , 1997 .

[42]  G. Weiss,et al.  On almost optimal priority rules for preemptive scheduling of stochastic jobs on parallel machines , 1995, Advances in Applied Probability.

[43]  Dimitris Bertsimas,et al.  The achievable region method in the optimal control of queueing systems; formulations, bounds and policies , 1995, Queueing Syst. Theory Appl..

[44]  Vincent Feltkamp.  Cooperation in controlled network structures , 1995 .

[45]  Hong Chen,et al.  Performance evaluation of scheduling control of queueing networks: Fluid model heuristics , 1995, Queueing Syst. Theory Appl..

[46]  J. Tsitsiklis,et al.  Branching bandits and Klimov's problem: achievable region and side constraints , 1995, IEEE Trans. Autom. Control..

[47]  John M. McNamara,et al.  Dynamic optimization in fluctuating environments , 1995, Proceedings of the Royal Society of London. Series B: Biological Sciences.

[48]  Leonidas Georgiadis,et al.  Problems of Adaptive Optimization In Multiclass M/GI/1 Queues with Bernoulli Feedback , 1995, Math. Oper. Res..

[49]  Sean P. Meyn,et al.  Duality and linear programs for stability and performance analysis of queuing networks and scheduling policies , 1996, IEEE Trans. Autom. Control..

[50]  Dimitris Bertsimas,et al.  Conservation Laws, Extended Polymatroids and Multiarmed Bandit Problems; A Polyhedral Approach to Indexable Systems , 1996, Math. Oper. Res..

[51]  David B. Shmoys,et al.  Scheduling to Minimize Average Completion Time: Off-Line and On-Line Approximation Algorithms , 1997, Math. Oper. Res..

[52]  Costis Maglaras.  Design of dynamic control policies for stochastic processing networks via fluid models , 1997, Proceedings of the 36th IEEE Conference on Decision and Control.

[53]  WeinJoel,et al.  Scheduling to Minimize Average Completion Time , 1997 .

[54]  Kevin D. Glazebrook,et al.  Scheduling Multiclass Queueing Networks on Parallel Servers: Approximate and Heavy-Traffic Optimality of Klimov's Priority Rule , 1997, ESA.

[55]  K. D. Glazebrook,et al.  Stochastic Scheduling with Priority Classes , 1998, Math. Oper. Res..

[56]  Kevin D. Glazebrook,et al.  Submodular Returns and Greedy Heuristics for Queueing Scheduling Problems , 1998, Oper. Res..

[57]  K. D. Glazebrook,et al.  On a new approach to the analysis of complex multi-armed bandits , 1998, Math. Methods Oper. Res..

[58]  E. J. Collins,et al.  Finite-horizon dynamic optimisation when the terminal reward is a concave functional of the distribution of the final state , 1998, Advances in Applied Probability.

[59]  Andreas S. Schulz,et al.  Approximation in Stochastic Scheduling: the Power of Lp-based Priority Rules Approximation in Stochastic Scheduling: the Power of Lp-based Priority Rules , 1998 .

[60]  José Niño-Mora.  On the throughput-WIP trade-off in queueing systems, diminishing returns and the threshold property: A linear programming approach , 1998 .

[61]  Kevin D. Glazebrook,et al.  Almost optimal policies for stochastic systemswhich almost satisfy conservation laws , 1999, Ann. Oper. Res..

[62]  Shaler Stidham,et al.  Sample-path conservation laws, with applications to scheduling queues and fluid systems , 2000, Queueing Syst. Theory Appl..

[63]  José Niño Mora.  Beyond Smith's Rule: An Optimal Dynamic Index, Rule For Single Machine Stochastic Scheduling With Convex Holding Costs , 2000 .

[64]  Dimitris Bertsimas,et al.  Restless Bandits, Linear Programming Relaxations, and a Primal-Dual Index Heuristic , 2000, Oper. Res..

[65]  José Niño-Mora.  On certain greedoid polyhedra, partially indexable scheduling problems and extended restless bandit allocation indices , 2000 .

[66]  J. Niño-Mora.  RESTLESS BANDITS, PARTIAL CONSERVATION LAWS AND INDEXABILITY , 2001 .