On the Control of a Queueing System with Aging State Information

We investigate control of a queueing system in which a component of the state space is subject to aging. The controller can choose to forward incoming queries to the system (where it needs time for processing), or respond with a previously generated response (incurring a penalty for not providing a fresh value). Hence, the controller faces a tradeoff between data freshness and response times. We model the system as a complex Markov decision process, simplify it, and construct a control policy. This policy shows near-optimal performance and achieves lower costs than both a myopic policy and a threshold policy.

[1]  Sandjai Bhulai.  Markov Decision Processes , 2002 .

[2]  Sandjai Bhulai.  Dynamic routing policies for multiskill call centers , 2009 .

[3]  Eitan Altman,et al.  Optimal admission, routing and service assignment control: the case of single buffer queues , 1998, Proceedings of the 37th IEEE Conference on Decision and Control (Cat. No.98CH36171).

[4]  Jan-Kees C. W. van Ommeren,et al.  An Optimal Query Assignment for Wireless Sensor Networks , 2012, ArXiv.

[5]  Jan van der Wal,et al.  AN MDP DECOMPOSITION APPROACH FOR TRAFFIC CONTROL AT ISOLATED SIGNALIZED INTERSECTIONS , 2008, Probability in the Engineering and Informational Sciences.

[6]  D. White,et al.  Dynamic programming, Markov chains, and the method of successive approximations , 1963 .

[7]  Michael Mitzenmacher,et al.  How Useful Is Old Information? , 2000, IEEE Trans. Parallel Distributed Syst..

[8]  Sandjai Bhulai,et al.  On the structure of value functions for threshold policies in queueing models , 2003, Journal of Applied Probability.

[9]  Richard F. Serfozo,et al.  Technical Note - An Equivalence Between Continuous and Discrete Time Markov Decision Processes , 1979, Oper. Res..

[10]  R. Serfozo.  An Equivalence between Continuous and Discrete Time Markov Decision Processes. , 1976 .

[11]  K. R. Krishnan,et al.  Separable routing: A scheme for state-dependent routing of circuit switched telephone traffic , 1992, Ann. Oper. Res..

[12]  Henk Tijms,et al.  A heuristic rule for routing customers to parallel servers , 1997 .

[13]  Michael Dahlin.  Interpreting Stale Load Information , 2000, IEEE Trans. Parallel Distributed Syst..

[14]  Steven A. Lippman,et al.  Applying a New Device in the Optimization of Exponential Queuing Systems , 1975, Oper. Res..

[15]  Y. Ho,et al.  Ordinal Optimization: Soft Optimization for Hard Problems , 2007 .

[16]  J. M. Norman,et al.  Heuristic procedures in dynamic programming , 1972 .

[17]  J Wijngaard,et al.  Decomposition for dynamic programming in production and inventory control , 1979 .

[18]  O. Hernández-Lerma,et al.  Discrete-time Markov control processes , 1999 .

[19]  W. Fleming.  Book Review: Discrete-time Markov control processes: Basic optimality criteria , 1997 .