Deadline Scheduling as Restless Bandits

The problem of stochastic deadline scheduling is considered. A constrained Markov decision process model is introduced in which jobs arrive randomly at a service center with stochastic job sizes, rewards, and completion deadlines. The service provider faces random processing costs, convex noncompletion penalties, and a capacity constraint that limits the simultaneous processing of jobs. Formulated as a restless multiarmed bandit problem, the stochastic deadline scheduling problem is shown to be indexable. A closed-form expression of the Whittle's index is obtained for the case when the processing costs are constant. An upper bound on the gap-to-optimality for the Whittle's index policy is obtained, and it is shown that the bound converges to zero as the job arrival rate and the number of available processors increase simultaneously to infinity.

[1]  George B. Dantzig,et al.  Linear programming and extensions , 1965 .

[2]  D. Warner,et al.  A Mathematical Programming Model for Scheduling Nursing Personnel in a Hospital , 1972 .

[3]  Michael L. Dertouzos,et al.  Control Robotics: The Procedural Control of Physical Processes , 1974, IFIP Congress.

[4]  J. Gittins.  Bandit processes and dynamic allocation indices , 1979 .

[5]  J. Gittins,et al.  A dynamic allocation index for the discounted multiarmed bandit problem , 1979 .

[6]  Aloysius Ka-Lau Mok,et al.  Fundamental design problems of distributed systems for the hard-real-time environment , 1983 .

[7]  J. Blazewicz,et al.  Selected Topics in Scheduling Theory , 1987 .

[8]  P. Whittle.  Restless Bandits: Activity Allocation in a Changing World , 1988 .

[9]  James W. Layland,et al.  Scheduling Algorithms for Multiprogramming in a Hard-Real-Time Environment , 1989, JACM.

[10]  Christian M. Ernst,et al.  Multi-armed Bandit Allocation Indices , 1989 .

[11]  Don Towsley,et al.  Optimal scheduling policies for a class of Queues with customer deadlines to the beginning of service , 1990, PERV.

[12]  J. Bather,et al.  Multi‐Armed Bandit Allocation Indices , 1990 .

[13]  Don Towsley,et al.  On the optimality of minimum laxity and earliest deadline scheduling for real-time multiprocessors , 1990, Proceedings. EUROMICRO '90 Workshop on Real Time.

[14]  R. Weber,et al.  ON AN INDEX POLICY FOR RESTLESS BANDITS , 1990 .

[15]  Prajit K. Dutta,et al.  What do discounted optima converge to?: A theory of discount rate asymptotics in economic models , 1991 .

[16]  John P. Lehoczky.  Real-time queueing theory , 1996, 17th IEEE Real-Time Systems Symposium.

[17]  Leandros Tassiulas,et al.  Optimal scheduling with deadline constraints in tree networks , 1997 .

[18]  John N. Tsitsiklis,et al.  The Complexity of Optimal Queuing Network Control , 1999, Math. Oper. Res..

[19]  E. Altman.  Constrained Markov Decision Processes , 1999 .

[20]  R. Rockafellar,et al.  Optimization of conditional value-at risk , 2000 .

[21]  S. Shreve,et al.  Real-time queues in heavy traffic with earliest-deadline-first queue discipline , 2001 .

[22]  Sham M. Kakade,et al.  Trading in Markovian Price Models , 2005, COLT.

[23]  Pascale Vicat-Blanc Primet,et al.  Scheduling deadline-constrained bulk data transfers to minimize network congestion , 2007, Seventh IEEE International Symposium on Cluster Computing and the Grid (CCGrid '07).

[24]  Characterization and computation of restless bandit marginal productivity indices , 2007, ValueTools '07.

[25]  Vivek S. Borkar,et al.  Index Policies for Real-Time Multicast Scheduling for Wireless Broadcast Systems , 2008, IEEE INFOCOM 2008 - The 27th Conference on Computer Communications.

[26]  Diego Ruiz-Hernández,et al.  Indexable Restless Bandits: Index Policies for Some Families of Stochastic Scheduling and Dynamic Allocation Problems , 2008 .

[27]  Qing Zhao,et al.  Indexability of Restless Bandit Problems and Optimality of Whittle Index for Dynamic Multichannel Access , 2008, IEEE Transactions on Information Theory.

[28]  Kavita Ramanan,et al.  Heavy traffic analysis for EDF queues with reneging , 2011, The Annals of Applied Probability.

[29]  Alan Burns,et al.  A survey of hard real-time scheduling for multiprocessor systems , 2011, CSUR.

[30]  R. Srikant,et al.  Scheduling for Optimal Rate Allocation in Ad Hoc Networks With Heterogeneous Delay Constraints , 2011, IEEE Journal on Selected Areas in Communications.

[31]  J. Dai.  Queues in Service Systems : Customer Abandonment and Diffusion Approximations , 2011 .

[32]  Feng Pan,et al.  Scheduling for charging plug-in hybrid electric vehicles , 2012, 2012 IEEE 51st IEEE Conference on Decision and Control (CDC).

[33]  Yuting Ji,et al.  Large scale charging of Electric Vehicles , 2012, PES 2012.

[34]  Panganamala Ramana Kumar,et al.  Packets with Deadlines: A Framework for Real-Time Wireless Networks , 2013, Packets with Deadlines: A Framework for Real-Time Wireless Networks.

[35]  Peter Jacko,et al.  Generalized Restless Bandits and the Knapsack Problem for Perishable Inventories , 2014, Oper. Res..

[36]  Jordi Vilaplana,et al.  A queuing theory model for cloud computing , 2014, The Journal of Supercomputing.

[37]  Panganamala Ramana Kumar,et al.  Decentralized throughput maximizing policies for deadline-constrained wireless networks , 2015, 2015 54th IEEE Conference on Decision and Control (CDC).

[38]  Qing-Shan Jia,et al.  Matching EV Charging Load With Uncertain Wind: A Simulation-Based Policy Improvement Approach , 2015, IEEE Transactions on Smart Grid.

[39]  Lang Tong,et al.  Large scale charging of electric vehicles: A multi-armed bandit approach , 2015, 2015 53rd Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[40]  Lang Tong,et al.  Deadline scheduling as restless bandits , 2016, 2016 54th Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[41]  Zhe Yu,et al.  An intelligent energy management system for large-scale charging of electric vehicles , 2016 .

[42]  Lang Tong,et al.  Demand response via large scale charging of electric vehicles , 2016, 2016 IEEE Power and Energy Society General Meeting (PESGM).

[43]  Lang Tong,et al.  Dynamic Scheduling for Charging Electric Vehicles: A Priority Rule , 2016, IEEE Transactions on Automatic Control.

[44]  Eilyan Bitar,et al.  Deadline Differentiated Pricing of Deferrable Electric Loads , 2014, IEEE Transactions on Smart Grid.

[45]  Zhe Yu.  Large scale charging of electric vehicles: Technology and economy , 2017 .

[46]  Yunjian Xu,et al.  Meeting inelastic demand in systems with storage and renewable sources , 2014, 2014 IEEE International Conference on Smart Grid Communications (SmartGridComm).