Control Techniques for Complex Networks: Workload

Power grids, flexible manufacturing, cellular communications: interconnectedness has consequences. This remarkable book gives the tools and philosophy you need to build network models detailed enough to capture essential dynamics but simple enough to expose the structure of effective control solutions and to clarify analysis. Core chapters assume only exposure to stochastic processes and linear algebra at the undergraduate level; later chapters are for advanced graduate students and researchers/practitioners. This gradual development bridges classical theory with the state-of-the-art. The workload model that is the basis of traditional analysis of the single queue becomes a foundation for workload relaxations used in the treatment of complex networks. Lyapunov functions and dynamic programming equations lead to the celebrated MaxWeight policy along with many generalizations. Other topics include methods for synthesizing hedging and safety stocks, stability theory for networks, and techniques for accelerated simulation. Examples and figures throughout make ideas concrete. Solutions to end-of-chapter exercises available on a companion website.

[1]  G. G. Stokes "J." , 1890, The New Yale Book of Quotations.

[2]  T. Thorpe My Life and Work , 1916, Nature.

[3]  Kai Lai Chung,et al.  A Course in Probability Theory , 1949 .

[4]  D. Kendall Some Problems in the Theory of Queues , 1951 .

[5]  K. Arrow,et al.  Optimal Inventory Policy. , 1951 .

[6]  J. Kiefer,et al.  The Inventory Problem: I. Case of Known Distributions of Demand , 1952 .

[7]  J. Kiefer,et al.  Stochastic Estimation of the Maximum of a Regression Function , 1952 .

[8]  D. Kendall Stochastic Processes Occurring in the Theory of Queues and their Analysis by the Method of the Imbedded Markov Chain , 1953 .

[9]  F. G. Foster On the Stochastic Matrices Associated with Certain Queuing Processes , 1953 .

[10]  Oliver Alfred Gross,et al.  On the Optimal Inventory Equation , 1955 .

[11]  D. R. Fulkerson,et al.  A Simple Algorithm for Finding Maximal Network Flows and an Application to the Hitchcock Problem , 1957, Canadian Journal of Mathematics.

[12]  Wayne E. Smith Various optimizers for single‐stage production , 1956 .

[13]  D. R. Fulkerson,et al.  Maximal Flow Through a Network , 1956 .

[14]  S. Vaida,et al.  Studies in the Mathematical Theory of Inventory and Production , 1958 .

[15]  H. Scarf THE OPTIMALITY OF (S,S) POLICIES IN THE DYNAMIC INVENTORY PROBLEM , 1959 .

[16]  S. Karlin,et al.  Studies in the Mathematical Theory of Inventory and Production, by K.J. Arrow, S. Karlin, H. Scarf with contributions by M.J. Beckmann, J. Gessford, R.F. Muth. Stanford, California, Stanford University Press, 1958, X p.340p., $ 8.75. , 1959, Bulletin de l'Institut de recherches économiques et sociales.

[17]  S. Karlin,et al.  Studies in the Mathematical Theory of Inventory and Production, by K.J. Arrow, S. Karlin, H. Scarf with contributions by M.J. Beckmann, J. Gessford, R.F. Muth. Stanford, California, Stanford University Press, 1958, X p.340p., $ 8.75. , 1959, Bulletin de l'Institut de recherches économiques et sociales.

[18]  J. E. Kelley,et al.  The Cutting-Plane Method for Solving Convex Programs , 1960 .

[19]  A. S. Manne Linear Programming and Sequential Decisions , 1960 .

[20]  Ronald A. Howard,et al.  Dynamic Programming and Markov Processes , 1960 .

[21]  John Frank Charles Kingman,et al.  The single server queue in heavy traffic , 1961, Mathematical Proceedings of the Cambridge Philosophical Society.

[22]  J. Beránek RONALD A. HOWARD “Dynamic Programming and Markov Processes,” , 1961 .

[23]  D. R. Fulkerson,et al.  Flows in Networks. , 1964 .

[24]  G. Bennett Probability Inequalities for the Sum of Independent Random Variables , 1962 .

[25]  J. Kingman On Queues in Heavy Traffic , 1962 .

[26]  D. White Dynamic programming, Markov chains, and the method of successive approximations , 1963 .

[27]  W. Hoeffding Probability Inequalities for sums of Bounded Random Variables , 1963 .

[28]  Onésimo Hernández-Lerma,et al.  Controlled Markov Processes , 1965 .

[29]  H. Mori Transport, Collective Motion, and Brownian Motion , 1965 .

[30]  Kazuoki Azuma WEIGHTED SUMS OF CERTAIN DEPENDENT RANDOM VARIABLES , 1967 .

[31]  G. Dantzig,et al.  On the continuity of the minimum set of a continuous function , 1967 .

[32]  R. Faure,et al.  Introduction to operations research , 1968 .

[33]  E. Denardo,et al.  Multichain Markov Renewal Programs , 1968 .

[34]  A. F. Veinott Discrete Dynamic Programming with Sensitive Discount Optimality Criteria , 1969 .

[35]  R. LePage Review: Kai Lai Chung, A Course in Probability Theory , 1970 .

[36]  G. P. Szegö,et al.  Stability theory of dynamical systems , 1970 .

[37]  D. Iglehart,et al.  Multiple channel queues in heavy traffic. I , 1970, Advances in Applied Probability.

[38]  Richard W. Cottle,et al.  On classes of copositive matrices , 1970 .

[39]  Vincent Hodgson,et al.  The Single Server Queue. , 1972 .

[40]  Allen J. Scott,et al.  Applications of Queueing Theory , 1972 .

[41]  Huibert Kwakernaak,et al.  Linear Optimal Control Systems , 1972 .

[42]  B. Mitchell Optimal service-rate selection in an $M| G |\hat 1$Queue , 1973 .

[43]  Rudolf Ahlswede,et al.  Multi-way communication channels , 1973 .

[44]  F. Black,et al.  The Pricing of Options and Corporate Liabilities , 1973, Journal of Political Economy.

[45]  Carlos S. Kubrusly,et al.  Stochastic approximation algorithms and applications , 1973, CDC 1973.

[46]  Alan F. Blackwell,et al.  Programming , 1973, CSC '73.

[47]  W. Pranger Extreme points of convex sets , 1973 .

[48]  Harold Joseph Highland,et al.  An inventory problem , 1973, SIML.

[49]  Stability , 1973 .

[50]  J. Harrison A limit theorem for priority queues in heavy traffic , 1973, Journal of Applied Probability.

[51]  Arie Hordijk,et al.  Dynamic programming and Markov potential theory , 1974 .

[52]  Erhan Çinlar,et al.  Introduction to stochastic processes , 1974 .

[53]  Steven A. Lippman,et al.  Applying a New Device in the Optimization of Exponential Queuing Systems , 1975, Oper. Res..

[54]  J. Michael Harrison,et al.  Dynamic Scheduling of a Multiclass Queue: Discount Optimality , 1975, Oper. Res..

[55]  K. Mani Chandy,et al.  Open, Closed, and Mixed Networks of Queues with Different Classes of Customers , 1975, JACM.

[56]  D. Iglehart,et al.  Discrete time methods for simulating continuous time Markov chains , 1976, Advances in Applied Probability.

[57]  Lennart Ljung,et al.  On positive real transfer functions and the convergence of some recursive schemes , 1977 .

[58]  Jonathan Halpern,et al.  Shortest route with time dependent length of edges and limited delay possibilities in nodes , 1977, Math. Methods Oper. Res..

[59]  Moshe Ben-Horim,et al.  A linear programming approach , 1977 .

[60]  Philip Heidelberger Variance reduction techniques for simulating Markov chains , 1977, WSC '77.

[61]  C. H. Sauer,et al.  Simultaneous resource possession in queueing models of computers , 1978, PERV.

[62]  A. Federgruen,et al.  Denumerable state semi-markov decision processes with unbounded costs, average cost criterion : (preprint) , 1979 .

[63]  Donald L. Iglehart,et al.  Regenerative Simulation of Response Times in Networks of Queues , 1980, J. ACM.

[64]  Martin L. Puterman,et al.  THE ANALYTIC THEORY OF POLICY ITERATION , 1978 .

[65]  B. Doshi Optimal control of the service rate in an M/G/1 queueing system , 1978, Advances in Applied Probability.

[66]  R. Weber On the optimal assignment of customers to parallel servers , 1978, Journal of Applied Probability.

[67]  J. Halpern Finding Minimal Center-Median Convex Combination Cent-Dian of a Graph , 1978 .

[68]  G. J. Foschini,et al.  A Basic Dynamic Routing Problem and Diffusion , 1978, IEEE Trans. Commun..

[69]  Jacob Wolfowitz,et al.  Multiple Access Channels , 1978 .

[70]  Jonathan Halpern,et al.  A generalized dynamic flows problem , 1979, Networks.

[71]  M. David HARRISON, J. Michael, and KREPS, . Martingales and Arbitrage in Multiperiod Securities Markets, Journal of Economic Theory, , . , 1979 .

[72]  L. Rogers,et al.  Diffusions, Markov processes, and martingales , 1979 .

[73]  S. Ross,et al.  Option pricing: A simplified approach☆ , 1979 .

[74]  J. Gittins Bandit processes and dynamic allocation indices , 1979 .

[75]  Martin L. Puterman,et al.  On the Convergence of Policy Iteration in Stationary Dynamic Programming , 1979, Math. Oper. Res..

[76]  P. Whittle Multi‐Armed Bandits and the Gittins Index , 1980 .

[77]  Erol Gelenbe,et al.  Analysis and Synthesis of Computer Systems , 1980 .

[78]  Ward Whitt,et al.  Some Useful Functions for Functional Limit Theorems , 1980, Math. Oper. Res..

[79]  Anthony Unwin,et al.  Reversibility and Stochastic Networks , 1980 .

[80]  P. Hall,et al.  Martingale Limit Theory and Its Application , 1980 .

[81]  Anthony Ephremides,et al.  A simple dynamic routing problem , 1980 .

[82]  R. Khasminskii Stochastic Stability of Differential Equations , 1980 .

[83]  J. Harrison,et al.  On the distribution of m ultidimen-sional reflected Brownian motion , 1981 .

[84]  E. Anderson A new continuous model for job-shop scheduling , 1981 .

[85]  J. MacGregor Smith,et al.  The Use of Queuing Networks in the Evaluation of Egress from Buildings , 1981 .

[86]  Marcel F. Neuts,et al.  Matrix-Geometric Solutions in Stochastic Models , 1981 .

[87]  Stephen S. Lavenberg,et al.  A Perspective on the Use of Control Variables to Increase the Efficiency of Monte Carlo Simulations , 1981 .

[88]  Zvi Rosberg,et al.  Optimal control of service in tandem queues , 1982 .

[89]  B. Hajek Optimal control of two interacting service stations , 1982, 1982 21st IEEE Conference on Decision and Control.

[90]  Edward D. Lazowska,et al.  Analyzing queueing networks with simultaneous resource possession , 1982, CACM.

[91]  T. Başar,et al.  Dynamic Noncooperative Game Theory , 1982 .

[92]  Sheldon M. Ross,et al.  Stochastic Processes , 2018, Gauge Integral Structures for Stochastic Calculus and Quantum Electrodynamics.

[93]  Daniel P. Heyman,et al.  Stochastic models in operations research , 1982 .

[94]  Bruce E. Hajek,et al.  Optimal dynamic routing in communication networks with continuous traffic , 1982, 1982 21st IEEE Conference on Decision and Control.

[95]  Peter Whittle,et al.  Optimization Over Time , 1982 .

[96]  Jan Telgen,et al.  Stochastic Dynamic Programming , 1982 .

[97]  Marcel F. Neuts,et al.  Matrix-geometric solutions in stochastic models - an algorithmic approach , 1982 .

[98]  Averill M. Law,et al.  Simulation Modeling and Analysis , 1982 .

[99]  Paul J. Schweitzer,et al.  Denumerable Undiscounted Semi-Markov Decision Processes with Unbounded Rewards , 1983, Math. Oper. Res..

[100]  James S. Vandergraft A Fluid Flow Model of Networks of Queues , 1983 .

[101]  Paul J. Schweitzer,et al.  Aggregation Methods for Large Markov Chains , 1983, Computer Performance and Reliability.

[102]  Vivek S. Borkar,et al.  Controlled Markov Chains and Stochastic Networks , 1983 .

[103]  Edward G. Coffman,et al.  Diffusion approximations for storage processes in computer systems , 1983, SIGMETRICS '83.

[104]  V. Borkar On Minimum Cost Per Unit Time Control of Markov Chains , 1984 .

[105]  J. Tsitsiklis Periodic Review Inventory Systems with Continuous Demand and Discrete Order Sizes , 1984 .

[106]  P. R. Kumar,et al.  Optimal control of a queueing system with two heterogeneous servers , 1984 .

[107]  Martin I. Reiman,et al.  Open Queueing Networks in Heavy Traffic , 1984, Math. Oper. Res..

[108]  Tapani Lehtonen,et al.  On the optimality of the shortest line discipline , 1984 .

[109]  P. Glynn Some asymptotic formulas for markov chains with applications to simulation , 1984 .

[110]  Harold J. Kushner,et al.  Approximation and Weak Convergence Methods for Random Processes , 1984 .

[111]  Martin I. Reiman,et al.  Some diffusion approximations with state space collapse , 1984 .

[112]  Graham C. Goodwin,et al.  Adaptive filtering prediction and control , 1984 .

[113]  Jean Walrand,et al.  The c# rule revisited , 1985 .

[114]  Bruce E. Hajek,et al.  Extremal Splittings of Point Processes , 1985, Math. Oper. Res..

[115]  Paul H. Zipkin,et al.  Computing optimal (s, S) policies in inventory models with continuous demands , 1985, Advances in Applied Probability.

[116]  J. Harrison,et al.  Brownian motion and stochastic flow systems , 1986 .

[117]  John F. MacGregor,et al.  Proceedings of the American Control Conference , 1985 .

[118]  J. Walrand,et al.  The cμ rule revisited , 1985, Advances in Applied Probability.

[119]  Christos Papadimitriou,et al.  Intractable problems in control theory , 1985, 1985 24th IEEE Conference on Decision and Control.

[120]  Editors , 1986, Brain Research Bulletin.

[121]  Awi Federgruen,et al.  An Inventory Model with Limited Production Capacity and Uncertain Demands II. The Discounted-Cost Criterion , 1986, Math. Oper. Res..

[122]  N. Kartashov Inequalities in Theorems of Ergodicity and Stability for Markov Chains with Common Phase Space. I , 1986 .

[123]  Hassan K. Khalil,et al.  Singular perturbation methods in control : analysis and design , 1986 .

[124]  Awi Federgruen,et al.  An Inventory Model with Limited Production Capacity and Uncertain Demands I. The Average-Cost Criterion , 1986, Math. Oper. Res..

[125]  Dimitri P. Bertsekas,et al.  Data Networks , 1986 .

[126]  R. Dekker Counter examples for compact action Markov decision chains with average reward criteria , 1987 .

[127]  Katta G. Murty,et al.  Some NP-complete problems in quadratic and nonlinear programming , 1987, Math. Program..

[128]  Ruth J. Williams,et al.  Multidimensional Reflected Brownian Motions Having Exponential Stationary Distributions , 1987 .

[129]  R. Weber,et al.  Optimal control of service rates in networks of queues , 1987, Advances in Applied Probability.

[130]  A. Borovkov Limit Theorems for Queueing Networks. I , 1987 .

[131]  P. Glynn,et al.  A joint central limit theorem for the sample mean and regenerative variance estimator , 1987 .

[132]  Lessi Oliviero,et al.  Variance Reduction Techniques in the Simulation of Markov Processes , 1987 .

[133]  Ruth J. Williams,et al.  Brownian Models of Open Queueing Networks with Homogeneous Customer Populations , 1987 .

[134]  Upendra Dave,et al.  Applied Probability and Queues , 1987 .

[135]  John N. Tsitsiklis,et al.  The Complexity of Markov Decision Processes , 1987, Math. Oper. Res..

[136]  Ruth J. Williams,et al.  A boundary property of semimartingale reflecting Brownian motions , 1988 .

[137]  Ping Yang,et al.  Pathwise solutions for a class of linear stochastic systems , 1988 .

[138]  M. Reiman A multiclass feedback queue in heavy traffic , 1988 .

[139]  Karl Sigman,et al.  Queues as Harris recurrent Markov chains , 1988, Queueing Syst. Theory Appl..

[140]  Anthony Ephremides,et al.  On the stability of interacting queues in a multiple-access system , 1988, IEEE Trans. Inf. Theory.

[141]  Donald L. Iglehart,et al.  Simulation methods for queues: An overview , 1988, Queueing Syst. Theory Appl..

[142]  P. R. Kumar,et al.  Stable distributed real-time scheduling of flexible manufacturing/assembly/disassembly systems , 1988, Proceedings of the 27th IEEE Conference on Decision and Control.

[143]  Stanley B. Gershwin,et al.  A hierarchical framework for discrete event scheduling in manufacturing systems , 1988 .

[144]  J. Michael Harrison,et al.  Brownian Models of Queueing Networks with Heterogeneous Customer Populations , 1988 .

[145]  A. W. Kemp,et al.  Applied Probability and Queues , 1989 .

[146]  Lawrence M. Wein,et al.  Scheduling networks of queues: Heavy traffic analysis of a simple open network , 1989, Queueing Syst. Theory Appl..

[147]  P. R. Kumar,et al.  Dynamic instabilities and stabilization methods in distributed real-time scheduling of manufacturing systems , 1989, Proceedings of the 28th IEEE Conference on Decision and Control,.

[148]  Linn I. Sennott,et al.  Average Cost Optimal Stationary Policies in Infinite State Markov Decision Processes with Unbounded Costs , 1989, Oper. Res..

[149]  W. Whitt Simulation run length planning , 1989, WSC '89.

[150]  Guy Fayolle,et al.  On random walks arising in queueing systems: ergodicity and transience via quadratic forms as lyapounov functions — Part I , 1989, Queueing Syst. Theory Appl..

[151]  D. Yao,et al.  Stochastic monotonicity in general queueing networks , 1989, Journal of Applied Probability.

[152]  John N. Tsitsiklis,et al.  Parallel and distributed computation , 1989 .

[153]  J. Walrand,et al.  Monotonicity of Throughput in Non-Markovian Networks. , 1989 .

[154]  H. Kushner,et al.  Optimal and approximately optimal control policies for queues in heavy traffic , 1989 .

[155]  V. Borkar Control of Markov chains with long-run average cost criterion: the dynamic programming equations , 1989 .

[156]  W. Whitt Planning queueing simulations , 1989 .

[157]  Gautam Appa,et al.  Linear Programming in Infinite-Dimensional Spaces , 1989 .

[158]  H. Soner,et al.  Regularity of the value function for a two-dimensional singular stochastic control problem , 1989 .

[159]  G. Weiss,et al.  Approximation results in parallel machnies stochastic scheduling , 1991 .

[160]  B. Anderson,et al.  Optimal control: linear quadratic methods , 1990 .

[161]  K. Sigman The stability of open queueing networks , 1990 .

[162]  Richard J. Gibbens,et al.  Dynamic Routing in Fully Connected Networks , 1990 .

[163]  David L. Woodruff,et al.  CONWIP: a pull alternative to kanban , 1990 .

[164]  Lawrence M. Wein,et al.  Scheduling Networks of Queues: Heavy Traffic Analysis of a Two-Station Closed Network , 1990, Oper. Res..

[165]  Pierre Priouret,et al.  Adaptive Algorithms and Stochastic Approximations , 1990, Applications of Mathematics.

[166]  P. Tseng Solving H-horizon, stationary Markov decision problems in time proportional to log(H) , 1990 .

[167]  Zhang Hanqin,et al.  MULTIPLE CHANNEL QUEUES IN HEAVY TRAFFIC , 1990 .

[168]  C. N. Laws,et al.  Dynamic Scheduling of a Four-Station Queueing Network , 1990, Probability in the Engineering and Informational Sciences.

[169]  J. M. Harrison,et al.  On the Quasireversibility of a Multiclass Brownian Service Station , 1990 .

[170]  Barry L. Nelson,et al.  Control Variate Remedies , 1990, Oper. Res..

[171]  R. Weber,et al.  STABILITY OF ON-LINE BIN PACKING WITH RANDOM ARRIVALS AND LONG-RUN-AVERAGE CONSTRAINTS , 1990 .

[172]  Hong Chen,et al.  Discrete Flow Networks: Bottleneck Analysis and Fluid Approximations , 1991, Math. Oper. Res..

[173]  Liliane Pintelon,et al.  Handbooks in operations research and management science volume 2: Stochastic models , 1991 .

[174]  P. R. Kumar,et al.  Distributed scheduling based on due dates and buffer priorities , 1991 .

[175]  V. Borkar Topics in controlled Markov chains , 1991 .

[176]  John N. Tsitsiklis,et al.  An Analysis of Stochastic Shortest Path Problems , 1991, Math. Oper. Res..

[177]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[178]  K. Sigman A Note on a Sample-Path Rate Conservation Law and its Relationship with H = λG , 1991, Advances in Applied Probability.

[179]  L. F. Martins,et al.  Numerical Methods for Stochastic Singular Control Problems , 1991 .

[180]  Venkat Anantharam The stability region of the finite-user slotted ALOHA protocol , 1991, IEEE Trans. Inf. Theory.

[181]  P. Dupuis,et al.  On Lipschitz continuity of the solution mapping to the Skorokhod problem , 1991 .

[182]  Yves De Serres,et al.  Simultaneous optimization of flow control and scheduling in a single server queue with two job classes , 1991, Oper. Res. Lett..

[183]  F. Kelly Network routing , 1991, Philosophical Transactions of the Royal Society of London. Series A: Physical and Engineering Sciences.

[184]  Rene L. Cruz,et al.  A calculus for network delay, Part I: Network elements in isolation , 1991, IEEE Trans. Inf. Theory.

[185]  V. Anantharam On fast simulation of the time to saturation of slotted ALOHA , 1992 .

[186]  Lawrence M. Wein,et al.  Dynamic Scheduling of a Multiclass Make-to-Stock Queue , 2015, Oper. Res..

[187]  Ward Whitt,et al.  The Asymptotic Efficiency of Simulation Estimators , 1992, Oper. Res..

[188]  A. Hordijk,et al.  On ergodicity and recurrence properties of a Markov chain by an application to an open jackson network , 1992, Advances in Applied Probability.

[189]  Christos G. Cassandras,et al.  Optimal routing and buffer allocation for a class of finite capacity queueing systems , 1992 .

[190]  Cheng-Shang Chang,et al.  On the optimality of LEPT and cµ rules for machines in parallel , 1992, Journal of Applied Probability.

[191]  J. Harrison,et al.  Reflected Brownian Motion in an Orthant: Numerical Methods for Steady-State Analysis , 1992 .

[192]  A. Mandelbaum,et al.  Regenerative closed queueing networks , 1992 .

[193]  John N. Tsitsiklis,et al.  Optimization of multiclass queuing networks: polyhedral and nonlinear characterizations of achievable performance , 1994 .

[194]  Leandros Tassiulas,et al.  Stability properties of constrained queueing systems and scheduling policies for maximum throughput in multihop radio networks , 1992 .

[195]  Linn I. Sennott,et al.  Optimal Stationary Policies in General State Space Markov Decision Chains with Finite Action Sets , 1992, Math. Oper. Res..

[196]  Sean P. Meyn,et al.  Generalized Resolvents and Harris Recurrence of Markov Processes , 1992 .

[197]  Lawrence M. Wein,et al.  A broader view of the job-shop scheduling problem , 1992 .

[198]  David D. Yao,et al.  Multiclass Queueing Systems: Polymatroidal Structure and Optimal Scheduling Control , 1992, Oper. Res..

[199]  Yves Dallery,et al.  Manufacturing flow line systems: a review of models and analytical results , 1992, Queueing Syst. Theory Appl..

[200]  J. M. Harrison,et al.  Brownian Models of Feedforward Queueing Networks: Quasireversibility and Product Form Solutions , 1992 .

[201]  Søren Asmussen,et al.  Queueing Simulation in Heavy Traffic , 1992, Math. Oper. Res..

[202]  J. Ou,et al.  Performance Bounds for Scheduling Queueing Networks , 1992 .

[203]  Thomas G. Robertazzi,et al.  Input Versus Output Queueing on a SpaceDivision Packet Switch , 1993 .

[204]  QUTdN QeO,et al.  Random Early Detection Gateways for Congestion Avoidance , 1993 .

[205]  F. P. Kelly,et al.  Dynamic routing in open queueing networks: Brownian models, cut constraints and resource pooling , 1993, Queueing Syst. Theory Appl..

[206]  James Randolph Perkins Control of push and pull manufacturing systems , 1993 .

[207]  Hideaki Takagi,et al.  Queueing analysis: a foundation of performance evaluation , 1993 .

[208]  S. Meyn,et al.  Stability of Markovian processes III: Foster–Lyapunov criteria for continuous-time processes , 1993, Advances in Applied Probability.

[209]  C. Cassandras,et al.  On the duality between routing and scheduling systems with finite buffer space , 1993, IEEE Trans. Autom. Control..

[210]  Guy Fayolle,et al.  Lyapounov Functions for Jackson Networks , 1993, Math. Oper. Res..

[211]  Shaler Stidham,et al.  A survey of Markov decision models for control of networks of queues , 1993, Queueing Syst. Theory Appl..

[212]  Richard L. Tweedie,et al.  Markov Chains and Stochastic Stability , 1993, Communications and Control Engineering Series.

[213]  R. J. Williams,et al.  Existence and uniqueness of semimartingale reflecting Brownian motions in an orthant , 1993 .

[214]  Yang Wang,et al.  Nonexistence of Brownian models for certain multiclass queueing networks , 1993, Queueing Syst. Theory Appl..

[215]  M. Pullan An algorithm for a class of continuous linear programs , 1993 .

[216]  Hong Chen,et al.  Dynamic Scheduling of a Multiclass Fluid Network , 1993, Oper. Res..

[217]  Ioannis Ch. Paschalidis Scheduling of multiclass queueing networks : bounds on achievable performance , 1993 .

[218]  Sigrún Andradóttir,et al.  Variance reduction through smoothing and control variates for Markov chain simulations , 1993, TOMC.

[219]  Edward G. Coffman,et al.  Markov chains, computer proofs, and average-case analysis of best fit bin packing , 1993, STOC '93.

[220]  V. Jacobson,et al.  Random early detection gateways for congestion avoidance , 1993, TNET.

[221]  James R. Wilson,et al.  A splitting scheme for control variates , 1993, Oper. Res. Lett..

[222]  Toshikazu Kimura,et al.  A Bibliography of Research on Heavy Traffic Limit Theorems for Queues , 1993 .

[223]  J. Tsitsiklis A short proof of the Gittins index theorem , 1993, Proceedings of 32nd IEEE Conference on Decision and Control.

[224]  M. K. Ghosh,et al.  Discrete-time controlled Markov processes with average cost criterion: a survey , 1993 .

[225]  M. Bramson Correction: Instability of FIFO Queueing Networks , 1994 .

[226]  John N. Tsitsiklis,et al.  The complexity of optimal queueing network control , 1994, Proceedings of IEEE 9th Annual Conference on Structure in Complexity Theory.

[227]  Ruth J. Williams,et al.  Lyapunov Functions for Semimartingale Reflecting Brownian Motions , 1994 .

[228]  Joy A. Thomas,et al.  On the stability of open networks: A unified approach by stochastic dominance , 1994, Queueing Syst. Theory Appl..

[229]  P. Glynn,et al.  Logarithmic asymptotics for steady-state tail probabilities in a single-server queue , 1994, Journal of Applied Probability.

[230]  J. Rosenthal Minorization Conditions and Convergence Rates for Markov Chain Monte Carlo , 1995 .

[231]  R. Tweedie,et al.  Subgeometric Rates of Convergence of f-Ergodic Markov Chains , 1994, Advances in Applied Probability.

[232]  V. Borkar Ergodic Control of Markov Chains with Constraints---The General Case , 1994 .

[233]  Thomas I. Seidman,et al.  "First come, first served" can be unstable! , 1994, IEEE Trans. Autom. Control..

[234]  Martin L. Puterman,et al.  Markov Decision Processes: Discrete Stochastic Dynamic Programming , 1994 .

[235]  S.C.H. Lu,et al.  Efficient scheduling policies to reduce mean and variance of cycle-time in semiconductor manufacturing plants , 1994 .

[236]  P. R. Kumar,et al.  Performance bounds for queueing networks and scheduling policies , 1994, IEEE Trans. Autom. Control..

[237]  Robert G. Gallager,et al.  An Inequality on the Capacity Region of Multiaccess Multipath Channels , 1994 .

[238]  P. R. Kumar,et al.  Optimal control of pull manufacturing systems , 1994, Proceedings of 1994 American Control Conference - ACC '94.

[239]  Moshe Shaked,et al.  Stochastic orders and their applications , 1994 .

[240]  François Baccelli,et al.  Ergodicity of Jackson-type queueing networks , 1994, Queueing Syst. Theory Appl..

[241]  Izak Duenyas A Simple Release Policy for Networks of Queues with Controllable Inputs , 1994, Oper. Res..

[242]  Sean P. Meyn,et al.  Stability of Generalized Jackson Networks , 1994 .

[243]  Javed Mostafa,et al.  The Easy Internet Handbook , 1994 .

[244]  D. Botvich,et al.  Ergodicity of conservative communication networks , 1994 .

[245]  M. Bramson Instability of FIFO Queueing Networks , 1994 .

[246]  Sean P. Meyn,et al.  State-Dependent Criteria for Convergence of Markov Chains , 1994 .

[247]  David D. Yao,et al.  Scheduling semiconductor lines using a fluid network model , 1994, IEEE Trans. Robotics Autom..

[248]  Sean P. Meyn,et al.  Duality and linear programs for stability and performance analysis of queueing networks and scheduling policies , 1994, Proceedings of 1994 33rd IEEE Conference on Decision and Control.

[249]  Nick Duffield,et al.  Large deviations and overflow probabilities for the general single-server queue, with applications , 1995 .

[250]  Sean P. Meyn Transience of Multiclass Queueing Networks Via Fluid Limit Models , 1995 .

[251]  Leandros Tassiulas,et al.  Adaptive back-pressure congestion control based on local information , 1995, IEEE Trans. Autom. Control..

[252]  E. Fernández-Gaucherand,et al.  Value iteration in a class of controlled Markov chains with average-criterion: unbounded costs case , 1995, Proceedings of 1995 34th IEEE Conference on Decision and Control.

[253]  Gideon Weiss,et al.  On optimal draining of re-entrant fluid lines , 1995 .

[254]  Viên Nguyen,et al.  Fluid and Diffusion Approximations of a Two-Station Mixed Queueing Network , 1995, Math. Oper. Res..

[255]  C. SIAMJ. A NEW VALUE ITERATION METHOD FOR THE AVERAGE COST DYNAMIC PROGRAMMING PROBLEM∗ , 1995 .

[256]  S. Meyn,et al.  Exponential and Uniform Ergodicity of Markov Processes , 1995 .

[257]  Leandros Tassiulas,et al.  A scheduling policy with maximal stability region for ring networks with spatial reuse , 1995, Queueing Syst. Theory Appl..

[258]  Hong Chen,et al.  Performance evaluation of scheduling control of queueing networks: Fluid model heuristics , 1995, Queueing Syst. Theory Appl..

[259]  Sean P. Meyn,et al.  Stability of acyclic multiclass queueing networks , 1995, IEEE Trans. Autom. Control..

[260]  Frank Kelly Dynamic Routing in Stochastic Networks , 1995 .

[261]  G. Fayolle,et al.  Topics in the Constructive Theory of Countable Markov Chains , 1995 .

[262]  Christian Huitema,et al.  Routing in the Internet , 1995 .

[263]  J. V. Mieghem Dynamic Scheduling with Convex Delay Costs: The Generalized CU Rule , 1995 .

[264]  Dimitri P. Bertsekas,et al.  Dynamic Programming and Optimal Control, Two Volume Set , 1995 .

[265]  T. Elfving,et al.  Criteria for copositive matrices using simplices and barycentric coordinates , 1995 .

[266]  Sean P. Meyn,et al.  Stability of queueing networks and scheduling policies , 1995, IEEE Trans. Autom. Control..

[267]  Florin Avram,et al.  Fluid models of sequencing problems in open queueing networks; an optimal control approach , 1995 .

[268]  P. Glasserman,et al.  Monotone Structure in Discrete Event Systems. , 1995 .

[269]  M. Pullan Forms of Optimal Solutions for Separated Continuous Linear Programs , 1995 .

[270]  J. Dai On Positive Harris Recurrence of Multiclass Queueing Networks: A Unified Approach Via Fluid Limit Models , 1995 .

[271]  Sean P. Meyn,et al.  Stability and convergence of moments for multiclass queueing networks via fluid limit models , 1995, IEEE Trans. Autom. Control..

[272]  S. Meyn,et al.  Computable exponential convergence rates for stochastically ordered Markov processes , 1996 .

[273]  A. Borodin,et al.  Handbook of Brownian Motion - Facts and Formulae , 1996 .

[274]  Lawrence M. Wein,et al.  Scheduling a Make-To-Stock Queue: Index Policies and Hedging Points , 1996, Oper. Res..

[275]  J. Tsitsiklis,et al.  Stability conditions for multiclass fluid queueing networks , 1996, IEEE Trans. Autom. Control..

[276]  Sunil Kumar,et al.  Fluctuation smoothing policies are stable for stochastic re-entrant lines , 1996, Discret. Event Dyn. Syst..

[277]  Gideon Weiss,et al.  Stability and Instability of Fluid Models for Reentrant Lines , 1996, Math. Oper. Res..

[278]  Sean P. Meyn,et al.  Duality and linear programs for stability and performance analysis of queuing networks and scheduling policies , 1996, IEEE Trans. Autom. Control..

[279]  Linn I. Sennott,et al.  The convergence of value iteration in average cost Markov decision chains , 1996, Oper. Res. Lett..

[280]  Sunil Kumar,et al.  Closed Queueing Networks in Heavy Traffic: Fluid Limits and Efficiency , 1996 .

[281]  Sean P. Meyn,et al.  Fluid Network Models: Linear Programs for Control and Performance Bounds , 1996 .

[282]  John N. Tsitsiklis,et al.  Neuro-Dynamic Programming , 1996, Encyclopedia of Machine Learning.

[283]  Wallace J. Hopp,et al.  Factory physics : foundations of manufacturing management , 1996 .

[284]  Elizabeth Schwerer A linear programming approach to the steady-state analysis of Markov processes , 1996 .

[285]  G. Dai A Fluid-limit Model Criterion for Instability of Multiclass Queueing Networks , 1996 .

[286]  Jean C. Walrand,et al.  High-performance communication networks , 1999 .

[287]  R. Cavazos-Cadena Value Iteration in a Class of Communicating Markov Decision Chains with the Average Cost Criterion , 1996 .

[288]  John N. Tsitsiklis,et al.  Analysis of Temporal-Diffference Learning with Function Approximation , 1996, NIPS.

[289]  Sean P. Meyn,et al.  A Liapounov bound for solutions of the Poisson equation , 1996 .

[290]  X. Zhou,et al.  Hierarchical feedback controls in two-machine flowshops under uncertainty , 1996, Proceedings of 35th IEEE Conference on Decision and Control.

[291]  Satish K. Tripathi,et al.  Enhancing throughput over wireless LANs using channel state dependent packet scheduling , 1996, Proceedings of IEEE INFOCOM '96. Conference on Computer Communications.

[292]  Asbjoern M. Bonvik,et al.  A comparison of production-line control mechanisms , 1997 .

[293]  Costis Maglaras Design of dynamic control policies for stochastic processing networks via fluid models , 1997, Proceedings of the 36th IEEE Conference on Decision and Control.

[294]  Hong Chen,et al.  Stability of Multiclass Queueing Networks Under FIFO Service Discipline , 1997, Math. Oper. Res..

[295]  Sean P. Meyn The policy iteration algorithm for average reward Markov decision processes with general state space , 1997, IEEE Trans. Autom. Control..

[296]  R. Gibbens Measurement-based connection admission control , 1997 .

[297]  M. Pullan Existence and duality theory for separated continuous linear programs , 1997 .

[298]  Shlomo Shamai,et al.  Information-theoretic considerations for symmetric, cellular, multiple-access fading channels - Part II , 1997, IEEE Trans. Inf. Theory.

[299]  Jan A. Van Mieghem,et al.  Dynamic Control of Brownian Networks: State Space Collapse and Equivalent Workload Formulations , 1997 .

[300]  J. Lynch,et al.  A weak convergence approach to the theory of large deviations , 1997 .

[301]  Sean P. Meyn,et al.  Efficient simulation of multiclass queueing networks , 1997, WSC '97.

[302]  Sean P. Meyn,et al.  Piecewise linear test functions for stability and instability of queueing networks , 1997, Queueing Syst. Theory Appl..

[303]  Bruce E. Hajek,et al.  Analysis of Simple Algorithms for Dynamic Load Balancing , 1997, Math. Oper. Res..

[304]  R. Hariharan,et al.  Scheduling in a multi-class series of queues with deterministic service times , 1996, Queueing Syst. Theory Appl..

[305]  John N. Tsitsiklis,et al.  Correction to "Stability Conditions for Multiclass Fluid Queueing Networks" , 1997, IEEE Trans. Autom. Control..

[306]  Paul H. Zipkin,et al.  Dynamic Scheduling Rules for a Multiproduct Make-to-Stock Queue , 1997, Oper. Res..

[307]  Amir Dembo,et al.  Large Deviations Techniques and Applications , 1998 .

[308]  S. Shreve,et al.  Stochastic differential equations , 1955, Mathematical Proceedings of the Cambridge Philosophical Society.

[309]  David Tse,et al.  Multiaccess Fading Channels-Part I: Polymatroid Structure, Optimal Resource Allocation and Throughput Capacities , 1998, IEEE Trans. Inf. Theory.

[310]  B. Hajek,et al.  On large deviations in load sharing networks , 1998 .

[311]  S. Hanly,et al.  Multi-access fading channels: delay-limited capacities , 1998, Proceedings. 1998 IEEE International Symposium on Information Theory (Cat. No.98CH36252).

[312]  Fabrice Guillemin,et al.  On the area swept under the occupation process of an M/M/1 queue in a busy period , 1998, Queueing Syst. Theory Appl..

[313]  Mark P. Van Oyen,et al.  Beyond the cμ rule: Dynamic scheduling of a two-class loss queue , 1998, Math. Methods Oper. Res..

[314]  Thomas M. Cover,et al.  Comments on Broadcast Channels , 1998, IEEE Trans. Inf. Theory.

[315]  Ruth J. Williams,et al.  Diffusion approximations for open multiclass queueing networks: sufficient conditions involving state space collapse , 1998, Queueing Syst. Theory Appl..

[316]  Richard D. McBride,et al.  Progress Made in Solving the Multicommodity Flow Problem , 1998, SIAM J. Optim..

[317]  S. Wittevrongel,et al.  Queueing systems , 2019, Autom..

[318]  Maury Bramson,et al.  State space collapse with application to heavy traffic limits for multiclass queueing networks , 1998, Queueing Syst. Theory Appl..

[319]  Reuven Y. Rubinstein,et al.  Modern simulation and modeling , 1998 .

[320]  J. Harrison Heavy traffic analysis of a system with parallel servers: asymptotic optimality of discrete-review policies , 1998 .

[321]  L. Sennott Stochastic Dynamic Programming and the Control of Queueing Systems , 1998 .

[322]  D. Bertsimas,et al.  A New Algorithm for State-Constrained Separated Continuous Linear Programs , 1999 .

[323]  Sean P. Meyn Algorithms for optimization and stabilization of controlled Markov chains , 1999 .

[324]  O. Hernández-Lerma,et al.  Discrete-time Markov control processes , 1999 .

[325]  J. Michael Harrison,et al.  Heavy traffic resource pooling in parallel‐server systems , 1999, Queueing Syst. Theory Appl..

[326]  Feryal Erhun,et al.  An overview of design and operational issues of kanban systems , 1999 .

[327]  J. R. Morrison,et al.  New Linear Program Performance Bounds for Queueing Networks , 1999 .

[328]  Constantinos Maglaras,et al.  Dynamic scheduling in multiclass queueing networks: Stability under discrete-review policies , 1999, Queueing Syst. Theory Appl..

[329]  John N. Tsitsiklis,et al.  Actor-Critic Algorithms , 1999, NIPS.

[330]  Christos G. Cassandras,et al.  Introduction to Discrete Event Systems , 1999, The Kluwer International Series on Discrete Event Dynamic Systems.

[331]  M. Dacre,et al.  The achievable region approach to the optimal control of stochastic systems , 1999 .

[332]  R. Srikant,et al.  Scheduling Real-Time Traffic With Deadlines over a Wireless Channel , 1999, WOWMOM '99.

[333]  Andrea J. Goldsmith,et al.  Capacity of time-slotted ALOHA systems , 1999, WCNC. 1999 IEEE Wireless Communications and Networking Conference (Cat. No.99TH8466).

[334]  R. J. Williams,et al.  Dynamic Scheduling of a System with Two Parallel Servers in Heavy Traac with Complete Resource Pooling: Asymptotic Optimality of a Continuous Review Threshold Policy 1 , 1999 .

[335]  M. Benaïm Dynamics of stochastic approximation algorithms , 1999 .

[336]  Eli Upfal,et al.  Balanced Allocations , 1999, SIAM J. Comput..

[337]  O. Hernández-Lerma,et al.  Further topics on discrete-time Markov control processes , 1999 .

[338]  Dynamic Scheduling of a System with Two Parallel Servers: Asymptotic Optimality of a Continuous Review Threshold Policy in Heavy Traffic , 1999 .

[339]  P. Dupuis,et al.  Convex duality and the Skorokhod Problem. I , 1999 .

[340]  R. Srikant,et al.  Variance Reduction in Simulations of Loss Models , 1999, Oper. Res..

[341]  Rajeev Agrawal,et al.  Performance bonds for flow control protocols , 1999, TNET.

[342]  G. Kallianpur,et al.  Introduction to option pricing theory , 1999 .

[343]  Sean P. Meyn,et al.  Value iteration and optimization of multiclass queueing networks , 1999, Queueing Syst. Theory Appl..

[344]  M. Bramson A stable queueing network with unstable fluid model , 1999 .

[345]  Sean P. Meyn,et al.  Variance Reduction for Simulation in Multiclass Queueing Networks , 1999 .

[346]  H. Kushner,et al.  Optimal control of assignment of jobs to processors under heavy traffic , 2000 .

[347]  Neil Immerman,et al.  The Complexity of Decentralized Control of Markov Decision Processes , 2000, UAI.

[348]  Ward Whitt,et al.  An overview of Brownian and non-Brownian FCLTs for the single-server queue , 2000, Queueing Syst. Theory Appl..

[349]  H. Thorisson Coupling, stationarity, and regeneration , 2000 .

[350]  H. Föllmer,et al.  On Itô s formula for multidimensional Brownian motion , 2000 .

[351]  Shaler Stidham,et al.  Sample-path conservation laws, with applications to scheduling queues and fluid systems , 2000, Queueing Syst. Theory Appl..

[352]  Ruth J. Williams,et al.  On dynamic scheduling of stochastic networks in heavy traffic and some new results for the workload process , 2000, Proceedings of the 39th IEEE Conference on Decision and Control (Cat. No.00CH37187).

[353]  H. Kushner Numerical Methods for Stochastic Control Problems in Continuous Time , 2000 .

[354]  Hong Chen,et al.  Stability of Multiclass Queueing Networks Under Priority Service Disciplines , 2000, Oper. Res..

[355]  Eitan Altman,et al.  Applications of Markov Decision Processes in Communication Networks , 2000 .

[356]  G. Thompson,et al.  Optimal Control Theory: Applications to Management Science and Economics , 2000 .

[357]  Sean P. Meyn,et al.  The O.D.E. Method for Convergence of Stochastic Approximation and Reinforcement Learning , 2000, SIAM J. Control. Optim..

[358]  Eitan Altman,et al.  Multimodularity, Convexity, and Optimization Properties , 2000, Math. Oper. Res..

[359]  Yves Dallery,et al.  A unified framework for pull control mechanisms in multi‐stage manufacturing systems , 2000, Ann. Oper. Res..

[360]  David Q. Mayne,et al.  Constrained model predictive control: Stability and optimality , 2000, Autom..

[361]  Michael Kearns,et al.  Bias-Variance Error Bounds for Temporal Difference Updates , 2000, COLT.

[362]  Dimitris Bertsimas,et al.  Restless Bandits, Linear Programming Relaxations, and a Primal-Dual Index Heuristic , 2000, Oper. Res..

[363]  J. Harrison Brownian models of open processing networks: canonical representation of workload , 2000 .

[364]  John H. Vande Vate,et al.  The Stability of Two-Station Multitype Fluid Networks , 2000, Oper. Res..

[365]  C. Maglaras Discrete-review policies for scheduling stochastic networks: trajectory tracking and fluid-scale asymptotic optimality , 2000 .

[366]  N. Bäuerle Asymptotic optimality of tracking policies in stochastic networks , 2001 .

[367]  Elizabeth Schwerer A LINEAR PROGRAMMING APPROACH TO THE STEADY-STATE ANALYSIS OF REFLECTED BROWNIAN MOTION , 2001 .

[368]  Michael Mitzenmacher,et al.  The Power of Two Choices in Randomized Load Balancing , 2001, IEEE Trans. Parallel Distributed Syst..

[369]  Lisa Fleischer,et al.  Faster Algorithms for the Quickest Transshipment Problem , 2001, SIAM J. Optim..

[370]  Ioannis Ch. Paschalidis,et al.  Probabilistic Service Level Guarantees in Make-to-Stock Manufacturing Systems , 2001, Oper. Res..

[371]  Shane G. Henderson,et al.  Mathematics for simulation , 2000, Proceeding of the 2001 Winter Simulation Conference (Cat. No.01CH37304).

[372]  Sean P. Meyn Sequencing and Routing in Multiclass Queueing Networks Part I: Feedback Regulation , 2001, SIAM J. Control. Optim..

[373]  Lisa Fleischer,et al.  Universally maximum flow with piecewise‐constant capacities , 2001, Networks.

[374]  D. McDonald,et al.  Join the shortest queue: stability and exact asymptotics , 2001 .

[375]  S. Strogatz Exploring complex networks , 2001, Nature.

[376]  E. Jouini,et al.  Option pricing, interest rates and risk management , 2001 .

[377]  Eitan Altman,et al.  ON THE COMPARISON OF QUEUEING SYSTEMS WITH THEIR FLUID LIMITS , 2001, Probability in the Engineering and Informational Sciences.

[378]  Peter W. Glynn,et al.  Regenerative steady-state simulation of discrete-event systems , 2001, TOMC.

[379]  P. Dupuis,et al.  ON POSITIVE RECURRENCE OF CONSTRAINED DIFFUSION PROCESSES , 2001, math/0501018.

[380]  Walter Willinger,et al.  Long-Range Dependence and Data Network Traffic , 2001 .

[381]  Sem C. Borst,et al.  Dynamic rate control algorithms for HDR throughput optimization , 2001, Proceedings IEEE INFOCOM 2001. Conference on Computer Communications. Twentieth Annual Joint Conference of the IEEE Computer and Communications Society (Cat. No.01CH37213).

[382]  R. Zwanzig Nonequilibrium statistical mechanics , 2001, Physics Subject Headings (PhySH).

[383]  R. Srikant,et al.  Failure-prone production systems with uncertain demand , 2001, IEEE Trans. Autom. Control..

[384]  H. Kushner Heavy Traffic Analysis of Controlled Queueing and Communication Networks , 2001 .

[385]  P. Glynn,et al.  Some New Perspectives on the Method of Control Variates , 2002 .

[386]  Hong Chen,et al.  The Finite Element Method for Computing the Stationary Distribution of an SRBM in a Hypercube with Applications to Finite Buffer Queueing Networks , 2002, Queueing Syst. Theory Appl..

[387]  P. Glynn,et al.  Hoeffding's inequality for uniformly ergodic Markov chains , 2002 .

[388]  David Gamarnik On Deciding Stability of Constrained Homogeneous Random Walks and Queueing Systems , 2002, Math. Oper. Res..

[389]  Vivek S. Borkar,et al.  Convex Analytic Methods in Markov Decision Processes , 2002 .

[390]  David Tse,et al.  Opportunistic beamforming using dumb antennas , 2002, IEEE Trans. Inf. Theory.

[391]  Lawrence E. Holloway,et al.  Stability of pull production control methods for systems with significant setups , 2002, IEEE Trans. Autom. Control..

[392]  Sean P. Meyn Stability, Performance Evaluation, and Optimization , 2002 .

[393]  Sean P. Meyn,et al.  Obituary: Richard Lewis Tweedie , 2002, Journal of Applied Probability.

[394]  Sean P. Meyn,et al.  The ODE Method and Spectral Theory of Markov Operators , 2002, math/0209277.

[395]  Benjamin Van Roy Neuro-Dynamic Programming: Overview and Recent Trends , 2002 .

[396]  J. Michael Harrison,et al.  Stochastic Networks and Activity Analysis , 2002 .

[397]  Peter W. Glynn,et al.  Managing Capacity and Inventory Jointly in Manufacturing Systems , 2002, Manag. Sci..

[398]  Christos G. Cassandras,et al.  Perturbation analysis for online control and optimization of stochastic fluid models , 2002, IEEE Trans. Autom. Control..

[399]  Peter W. Glynn,et al.  Approximating Martingales for Variance Reduction in Markov Process Simulation , 2002, Math. Oper. Res..

[400]  Eugene A. Feinberg,et al.  Handbook of Markov Decision Processes , 2002 .

[401]  Dimitris Bertsimas,et al.  From fluid relaxations to practical algorithms for job shop scheduling: the makespan objective , 2002, Math. Program..

[402]  S. Meyn,et al.  Spectral theory and limit theorems for geometrically ergodic Markov processes , 2002, math/0209200.

[403]  Gideon Weiss,et al.  A Fluid Heuristic for Minimizing Makespan in Job Shops , 2002, Oper. Res..

[404]  Maury Bramson,et al.  Two Workload Properties for Brownian Networks , 2003, Queueing Syst. Theory Appl..

[405]  Eitan Altman,et al.  Discrete-Event Control of Stochastic Networks - Multimodularity and Regularity , 2004, Lecture notes in mathematics.

[406]  David Gamarnik,et al.  From Fluid Relaxations to Practical Algorithms for High-Multiplicity Job-Shop Scheduling: The Holding Cost Objective , 2003, Oper. Res..

[407]  Sean P. Meyn,et al.  Value functions and performance evaluation in stochastic network models , 2003, 42nd IEEE International Conference on Decision and Control (IEEE Cat. No.03CH37475).

[408]  Mike Chen,et al.  In Search of Sensitivity in Network Optimization , 2003, Queueing Syst. Theory Appl..

[409]  Sean P. Meyn Stability and optimization of queueing networks and their fluid models , 2003 .

[410]  A. Hordijk,et al.  LARGE DEVIATIONS BOUNDS FOR FACE-HOMOGENEOUS RANDOM WALKS IN THE QUARTER-PLANE , 2003, Probability in the Engineering and Informational Sciences.

[411]  Dimitri P. Bertsekas,et al.  Least Squares Policy Evaluation Algorithms with Linear Function Approximation , 2003, Discret. Event Dyn. Syst..

[412]  D. Gamarnik,et al.  Instability in stochastic and fluid queueing networks , 2003, math/0311206.

[413]  Henk Zijm,et al.  Analysis and Modeling of Manufacturing Systems , 2003 .

[414]  John W. Fowler,et al.  Capacity Loss Factors in Semiconductor Manufacturing , 2003 .

[415]  Benjamin Van Roy,et al.  The Linear Programming Approach to Approximate Dynamic Programming , 2003, Oper. Res..

[416]  Alexandre J. Chorin Conditional Expectations and Renormalization , 2003, Multiscale Model. Simul..

[417]  O. Hernández-Lerma,et al.  Markov chains and invariant probabilities , 2003 .

[418]  Ad Ridder Large-deviations analysis of the fluid approximation for a controllable tandem queue , 2003 .

[419]  Sean P. Meyn Sequencing and Routing in Multiclass Queueing Networks Part II: Workload Relaxations , 2003, SIAM J. Control. Optim..

[420]  A. Borovkov,et al.  On the integral of the workload process of the single server queue , 2003, Journal of Applied Probability.

[421]  Sean P. Meyn,et al.  Performance Evaluation and Policy Selection in Multiclass Networks , 2003, Discret. Event Dyn. Syst..

[422]  Milos Hauskrecht,et al.  Linear Program Approximations for Factored Continuous-State Markov Decision Processes , 2003, NIPS.

[423]  Sujin Kim,et al.  Adaptive control variates , 2004, Proceedings of the 2004 Winter Simulation Conference, 2004..

[424]  Shane G. Henderson,et al.  Call Center Staffing with Simulation and Cutting Plane Methods , 2004, Ann. Oper. Res..

[425]  P. Fairley,et al.  The unruly power grid , 2004, IEEE Spectrum.

[426]  R. Douc,et al.  Practical drift conditions for subgeometric rates of convergence , 2004, math/0407122.

[427]  Approximate Dynamic Programming for Networks : Fluid Models and Constraint Reduction , 2004 .

[428]  Stanley B. Gershwin,et al.  Production and Subcontracting Strategies for Manufacturers with Limited Capacity and Volatile Demand , 2004, Ann. Oper. Res..

[429]  Steven J. Bradtke,et al.  Linear Least-Squares algorithms for temporal difference learning , 2004, Machine Learning.

[430]  George B. Dantzig,et al.  Linear Programming Under Uncertainty , 2004, Manag. Sci..

[431]  Herbert E. Scarf,et al.  Optimal Policies for a Multi-Echelon Inventory Problem , 1960, Manag. Sci..

[432]  D. Yao,et al.  Fundamentals of Queueing Networks: Performance, Asymptotics, and Optimization , 2001, IEEE Transactions on Automatic Control.

[433]  Richard C. Chen,et al.  Dynamic programming equations for discounted constrained stochastic control , 2004, IEEE Transactions on Automatic Control.

[434]  Ioannis G. Kevrekidis,et al.  Equation-free: The computer-aided analysis of complex multiscale systems , 2004 .

[435]  G. Yin,et al.  Discrete-Time Markov Chains: Two-Time-Scale Methods and Applications , 2004 .

[436]  Daniel Adelman,et al.  A Price-Directed Approach to Stochastic Inventory/Routing , 2004, Oper. Res..

[437]  Mike Chen,et al.  Management of demand-driven production systems , 2004, IEEE Transactions on Automatic Control.

[438]  James R. Jackson,et al.  Jobshop-Like Queueing Systems , 2004, Manag. Sci..

[439]  Sean P. Meyn Dynamic Safety-Stocks for Asymptotic Optimality in Stochastic Networks , 2005, 2004 43rd IEEE Conference on Decision and Control (CDC) (IEEE Cat. No.04CH37601).

[440]  Alexander L. Stolyar,et al.  Scheduling Flexible Servers with Convex Delay Costs: Heavy-Traffic Optimality of the Generalized cµ-Rule , 2004, Oper. Res..

[441]  S. Shakkottai,et al.  Pathwise optimality of the exponential scheduling rule for wireless channels , 2004, Advances in Applied Probability.

[442]  D. E. Heyman,et al.  Diffusion Approximations , 2004 .

[443]  A. Stolyar MaxWeight scheduling in a generalized switch: State space collapse and workload minimization in heavy traffic , 2004 .

[444]  Galin L. Jones On the Markov chain central limit theorem , 2004, math/0409112.

[445]  T. Lindvall ON A ROUTING PROBLEM , 2004, Probability in the Engineering and Informational Sciences.

[446]  J. Dai,et al.  Stability and instability of a two-station queueing network , 2004 .

[447]  A. Barto,et al.  Improved Temporal Difference Methods with Linear Function Approximation , 2004 .

[448]  Sean R Eddy,et al.  What is dynamic programming? , 2004, Nature Biotechnology.

[449]  Mike Chen,et al.  Modeling and control of complex stochastic networks, with applications to manufacturing systems and electric power transmission networks , 2005 .

[450]  V. Borkar Controlled diffusion processes , 2005, math/0511077.

[451]  Graham C. Goodwin,et al.  Constrained Control and Estimation , 2005 .

[452]  R. Srikant,et al.  Stable scheduling policies for fading wireless channels , 2005, IEEE/ACM Transactions on Networking.

[453]  Shie Mannor,et al.  Basis Function Adaptation in Temporal Difference Reinforcement Learning , 2005, Ann. Oper. Res..

[454]  Sean P. Meyn,et al.  Relative entropy and exponential deviation bounds for general Markov chains , 2005, Proceedings. International Symposium on Information Theory, 2005. ISIT 2005..

[455]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[456]  M. Safari Price of Anarchy ⋆ , 2005 .

[457]  Derong Liu The Mathematics of Internet Congestion Control , 2005, IEEE Transactions on Automatic Control.

[458]  J. G. Dai,et al.  Maximum Pressure Policies in Stochastic Processing Networks , 2005, Oper. Res..

[459]  D. McDonald,et al.  Large deviations of a modified Jackson network: Stability and rough asymptotics , 2005, math/0503487.

[460]  Kazuo Murota,et al.  Note on Multimodularity and L-Convexity , 2005, Math. Oper. Res..

[461]  S. Meyn,et al.  Large Deviations Asymptotics and the Spectral Theory of Multiplicatively Regular Markov Processes , 2005, math/0509310.

[462]  Sean P. Meyn Workload models for stochastic networks: value functions and performance evaluation , 2005, IEEE Transactions on Automatic Control.

[463]  P. Baxendale Renewal theory and computable convergence rates for geometrically ergodic Markov chains , 2005, math/0503515.

[464]  S. Sushanth Kumar,et al.  Heavy traffic analysis of open processing networks with complete resource pooling: Asymptotic optimality of discrete review policies , 2005, math/0503477.

[465]  S. Ethier,et al.  Markov Processes: Characterization and Convergence , 2005 .

[466]  Claire Mathieu,et al.  On the Sum-of-Squares algorithm for bin packing , 2002, JACM.

[467]  R. Srikant,et al.  Joint congestion control, routing, and MAC for stability and fairness in wireless networks , 2006, IEEE Journal on Selected Areas in Communications.

[468]  Y. Zou,et al.  Multiscale analysis of re-entrant production lines: An equation-free approach , 2005, math/0511360.

[469]  José Niño-Mora,et al.  Restless Bandit Marginal Productivity Indices, Diminishing Returns, and Optimal Control of Make-to-Order/Make-to-Stock M/G/1 Queues , 2006, Math. Oper. Res..

[470]  Graham C. Goodwin,et al.  Constrained Control and Estimation: an Optimization Approach , 2004, IEEE Transactions on Automatic Control.

[471]  K. Ramanan Reflected Diffusions Defined via the Extended Skorokhod Map , 2006 .

[472]  J. Harrison Correction: Brownian models of open processing networks: canonical representation of workload , 2000, math/0610352.

[473]  R. Atar,et al.  Singular control with state constraints on unbounded domain , 2005, math/0501016.

[474]  A. Sapozhnikov Subgeometric rates of convergence of f-ergodic Markov chains , 2006 .

[475]  Leandros Tassiulas,et al.  Resource Allocation and Cross Layer Control in Wireless Networks (Foundations and Trends in Networking, V. 1, No. 1) , 2006 .

[476]  Mike Chen,et al.  Reliability by design in distributed power transmission networks , 2006, Autom..

[477]  S. Meyn Large deviation asymptotics and control variates for simulating large functions , 2006, math/0603328.

[478]  D. Gamarnik,et al.  Validity of heavy traffic steady-state approximations in generalized Jackson networks , 2004, math/0410066.

[479]  Gersende Fort,et al.  ODE methods for Markov chain stability with applications to MCMC , 2006, valuetools '06.

[480]  Adam Shwartz,et al.  Large Deviations For Performance Analysis , 2019 .

[481]  Peter Whittle Networks: Optimisation and Evolution , 2007 .

[482]  H. Robbins A Stochastic Approximation Method , 1951 .

[483]  Susan A. Murphy,et al.  Monographs on statistics and applied probability , 1990 .

[484]  Myron Hlynka,et al.  Queueing Networks and Markov Chains (Modeling and Performance Evaluation With Computer Science Applications) , 2007, Technometrics.

[485]  Anthony S. Acampora,et al.  On the Performance of a Two-User MIMO Downlink System in Heavy Traffic , 2007, IEEE Transactions on Information Theory.

[486]  David Clark,et al.  The Morgan Kaufmann Series in Networking , 2008 .

[487]  Xiongzhi Chen Brownian Motion and Stochastic Calculus , 2008 .

[488]  Sunil Kumar,et al.  Decision , Risk & Operations Working Papers Series Approximate and Data-Driven Dynamic Programming for Queueing Networks , 2008 .

[489]  P. Dupuis,et al.  SDEs with Oblique Reflection on Nonsmooth Domains , 2008 .

[490]  S. Meyn,et al.  COMPUTABLE EXPONENTIAL BOUNDS FOR SCREENED ESTIMATION AND SIMULATION , 2006, math/0612040.

[491]  John N. Tsitsiklis,et al.  A Single-Unit Decomposition Approach to Multiechelon Inventory Systems , 2008, Oper. Res..

[492]  Sean P. Meyn Stability and Asymptotic Optimality of Generalized MaxWeight Policies , 2008, SIAM J. Control. Optim..

[493]  Amarjit Budhiraja,et al.  Stationary Distribution Convergence for Generalized Jackson Networks in Heavy Traffic , 2009, Math. Oper. Res..

[494]  A. Stolyar On the Stability of Multiclass Queueing Networks: A Relaxed SuÆcient Condition via Limiting Fluid Processes , .