On truncations and perturbations of Markov decision problems with an application to queueing network overflow control

Conditions are provided to derive error bounds on the effect of truncations and perturbations in Markov decision problems. Both the average and finite horizon case are studied. As an application, an explicit error bound is obtained for a truncation of a Jacksonian queueing network with overflow control.

[1]  Peter W. Jones,et al.  Stochastic Modelling and Analysis , 1988 .

[2]  Ward Whitt,et al.  Approximations of Dynamic Programs, I , 1978, Math. Oper. Res..

[3]  Nico M. van Dijk.  PERTURBATION THEORY FOR UNBOUNDED MARKOV REWARD PROCESSES WITH APPLICATIONS TO QUEUEING , 1988 .

[4]  Henk Tijms,et al.  Stochastic modelling and analysis: a computational approach , 1986 .

[5]  Erik A. van Doorn,et al.  On the Overflow Process from a Finite Markovian Queue , 1984, Perform. Evaluation.

[6]  CARL D. MEYER,et al.  The Condition of a Finite Markov Chain and Perturbation Bounds for the Limiting Probabilities , 1980, SIAM J. Algebraic Discret. Methods.

[7]  Michael Kolonko,et al.  Bounds for the regret loss in dynamic programming under adaptive control , 1983, Z. Oper. Research.

[8]  Hans-Joachim Langen,et al.  Convergence of Dynamic Programming Models , 1981, Math. Oper. Res..

[9]  Nico M. van Dijk,et al.  Truncation of Markov Chains with Applications to Queueing , 1991, Oper. Res..

[10]  M. Kurano.  AVERAGE-OPTIMAL ADAPTIVE POLICIES IN SEMI-MARKOV DECISION PROCESSES INCLUDING AN UNKNOWN PARAMETER , 1985 .

[11]  P. Schweitzer.  Perturbation theory and finite Markov chains , 1968 .

[12]  K. Hinderer.  ON APPROXIMATE SOLUTIONS OF FINITE-STAGE DYNAMIC PROGRAMS , 1978 .

[13]  M. Puterman,et al.  Perturbation theory for Markov reward processes with applications to queueing systems , 1988, Advances in Applied Probability.

[14]  Nico M. van Dijk,et al.  Simple Bounds and Monotnicity the Call Congestion of Finite Multiserver Delay Systems , 1988, Probability in the Engineering and Informational Sciences.

[15]  Eugene Seneta,et al.  Finite approximations to infinite non-negative matrices , 1967, Mathematical Proceedings of the Cambridge Philosophical Society.

[16]  A proof of simple insensitive bounds for a pure overflow system , 1989 .