Deadline scheduling as restless bandits

The problem of stochastic deadline scheduling is considered. A constrained Markov decision process model is introduced in which jobs arrive randomly at a service center with stochastic job sizes, rewards, and completion deadlines. The service provider faces random processing costs, convex non-completion penalties, and a capacity constraint that limits the simultaneous processing of jobs. Formulated as a restless multi-armed bandit problem, the stochastic deadline scheduling problem is shown to be indexable. A closed-form expression of the Whittle's index is obtained for the case when the processing costs are constant. An upper bound on the gap-to-optimality of the Whittle's index policy is established, and the bound is shown to converge to zero as the job arrival rate and the the number of simultaneously available processors increase simultaneously.

[1]  Yuting Ji,et al.  Large scale charging of Electric Vehicles , 2012, PES 2012.

[2]  Pascale Vicat-Blanc Primet,et al.  Scheduling deadline-constrained bulk data transfers to minimize network congestion , 2007, Seventh IEEE International Symposium on Cluster Computing and the Grid (CCGrid '07).

[3]  John N. Tsitsiklis,et al.  The Complexity of Optimal Queuing Network Control , 1999, Math. Oper. Res..

[4]  José Niño-Mora.  Characterization and computation of restless bandit marginal productivity indices , 2007, VALUETOOLS.

[5]  P. Whittle.  Restless Bandits: Activity Allocation in a Changing World , 1988 .

[6]  Don Towsley,et al.  Optimal scheduling policies for a class of Queues with customer deadlines to the beginning of service , 1990, PERV.

[7]  Lang Tong,et al.  Dynamic Scheduling for Charging Electric Vehicles: A Priority Rule , 2016, IEEE Transactions on Automatic Control.

[8]  Qing-Shan Jia,et al.  Matching EV Charging Load With Uncertain Wind: A Simulation-Based Policy Improvement Approach , 2015, IEEE Transactions on Smart Grid.

[9]  D. Warner,et al.  A Mathematical Programming Model for Scheduling Nursing Personnel in a Hospital , 1972 .

[10]  Zhe Yu,et al.  An intelligent energy management system for large-scale charging of electric vehicles , 2016 .

[11]  Eilyan Bitar,et al.  Deadline Differentiated Pricing of Deferrable Electric Loads , 2014, IEEE Transactions on Smart Grid.

[12]  Lang Tong,et al.  Large scale charging of electric vehicles: A multi-armed bandit approach , 2015, 2015 53rd Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[13]  Jordi Vilaplana,et al.  A queuing theory model for cloud computing , 2014, The Journal of Supercomputing.

[14]  J. Blaswicz.  Selected topics in scheduling theory , 1987 .

[15]  Vivek S. Borkar,et al.  Index Policies for Real-Time Multicast Scheduling for Wireless Broadcast Systems , 2008, IEEE INFOCOM 2008 - The 27th Conference on Computer Communications.

[16]  Panganamala Ramana Kumar,et al.  Packets with Deadlines: A Framework for Real-Time Wireless Networks , 2013, Packets with Deadlines: A Framework for Real-Time Wireless Networks.

[17]  Emiliano A. Valdez,et al.  Tail Conditional Expectations for Exponential Dispersion Models , 2005, ASTIN Bulletin.

[18]  Leandros Tassiulas,et al.  Optimal scheduling with deadline constraints in tree networks , 1997 .

[19]  J. Gittins.  Bandit processes and dynamic allocation indices , 1979 .

[20]  Yunjian Xu,et al.  Deadline Scheduling as Restless Bandits , 2018, IEEE Transactions on Automatic Control.

[21]  George B. Dantzig,et al.  Linear programming and extensions , 1965 .

[22]  Alan Burns,et al.  A survey of hard real-time scheduling for multiprocessor systems , 2011, CSUR.

[23]  Zhe Yu.  Large scale charging of electric vehicles: Technology and economy , 2017 .

[24]  Ronald L. Rivest,et al.  Introduction to Algorithms , 1990 .

[25]  James W. Layland,et al.  Scheduling Algorithms for Multiprogramming in a Hard-Real-Time Environment , 1989, JACM.

[26]  Kavita Ramanan,et al.  Heavy traffic analysis for EDF queues with reneging , 2011, The Annals of Applied Probability.

[27]  R. Weber,et al.  ON AN INDEX POLICY FOR RESTLESS BANDITS , 1990 .

[28]  Qing Zhao,et al.  Indexability of Restless Bandit Problems and Optimality of Whittle Index for Dynamic Multichannel Access , 2008, IEEE Transactions on Information Theory.

[29]  Michael L. Dertouzos,et al.  Control Robotics: The Procedural Control of Physical Processes , 1974, IFIP Congress.

[30]  J. Dai.  Queues in Service Systems : Customer Abandonment and Diffusion Approximations , 2011 .

[31]  J. Gittins,et al.  A dynamic allocation index for the discounted multiarmed bandit problem , 1979 .

[32]  Christian M. Ernst,et al.  Multi-armed Bandit Allocation Indices , 1989 .

[33]  Prajit K. Dutta,et al.  What do discounted optima converge to?: A theory of discount rate asymptotics in economic models , 1991 .

[34]  Peter Jacko,et al.  Generalized Restless Bandits and the Knapsack Problem for Perishable Inventories , 2014, Oper. Res..

[35]  John P. Lehoczky.  Real-time queueing theory , 1996, 17th IEEE Real-Time Systems Symposium.

[36]  Diego Ruiz-Hernández,et al.  Indexable Restless Bandits: Index Policies for Some Families of Stochastic Scheduling and Dynamic Allocation Problems , 2008 .

[37]  R. Rockafellar,et al.  Optimization of conditional value-at risk , 2000 .

[38]  J. Blazewicz,et al.  Selected Topics in Scheduling Theory , 1987 .

[39]  J. Bather,et al.  Multi‐Armed Bandit Allocation Indices , 1990 .

[40]  Feng Pan,et al.  Scheduling for charging plug-in hybrid electric vehicles , 2012, 2012 IEEE 51st IEEE Conference on Decision and Control (CDC).

[41]  Panganamala Ramana Kumar,et al.  Decentralized throughput maximizing policies for deadline-constrained wireless networks , 2015, 2015 54th IEEE Conference on Decision and Control (CDC).

[42]  Sham M. Kakade,et al.  Trading in Markovian Price Models , 2005, COLT.

[43]  S. Shreve,et al.  Real-time queues in heavy traffic with earliest-deadline-first queue discipline , 2001 .

[44]  Bernhard Klar,et al.  BOUNDS ON TAIL PROBABILITIES OF DISCRETE DISTRIBUTIONS , 2000, Probability in the Engineering and Informational Sciences.

[45]  R. Srikant,et al.  Scheduling for Optimal Rate Allocation in Ad Hoc Networks With Heterogeneous Delay Constraints , 2011, IEEE Journal on Selected Areas in Communications.

[46]  E. Altman.  Constrained Markov Decision Processes , 1999 .

[47]  Don Towsley,et al.  On the optimality of minimum laxity and earliest deadline scheduling for real-time multiprocessors , 1990, Proceedings. EUROMICRO '90 Workshop on Real Time.

[48]  Anthony Ephremides,et al.  Optimal scheduling with strict deadlines , 1989 .

[49]  Yunjian Xu,et al.  Meeting inelastic demand in systems with storage and renewable sources , 2014, 2014 IEEE International Conference on Smart Grid Communications (SmartGridComm).