Exact finite approximations of average-cost countable Markov decision processes

For a countable-state Markov decision process we introduce an embedding which produces a finite-state Markov decision process. The finite-state embedded process has the same optimal cost, and moreover, it has the same dynamics as the original process when restricting to the approximating set. The embedded process can be used as an approximation which, being finite, is more convenient for computation and implementation.

[1]  Abdeslem Boukhtouta,et al.  Water Reservoir Applications of Markov Decision Processes , 2002 .

[2]  Martin L. Puterman,et al.  Markov Decision Processes: Discrete Stochastic Dynamic Programming , 1994 .

[3]  Jean B. Lasserre,et al.  Detecting optimal and non-optimal actions in average-cost Markov decision processes , 1994 .

[4]  A. Shwartz,et al.  The Poisson Equation for Countable Markov Chains: Probabilistic Methods and Interpretations , 2002 .

[5]  Eitan Altman,et al.  Denumerable Constrained Markov Decision Processes and Finite Approximations , 1994, Math. Oper. Res..

[6]  Eugene A. Feinberg,et al.  Handbook of Markov Decision Processes , 2002 .

[7]  J. Ben Atkinson,et al.  An Introduction to Queueing Networks , 1988 .

[8]  V. Borkar.  Topics in controlled Markov chains , 1991 .

[9]  E. Fainberg.  Sufficient Classes of Strategies in Discrete Dynamic Programming I: Decomposition of Randomized Strategies and Embedded Models , 1987 .

[10]  G. G. Stokes.  "J." , 1890, The New Yale Book of Quotations.

[11]  Armand M. Makowski,et al.  K competing queues with geometric service requirements and linear costs: The μc-rule is always optimal☆ , 1985 .

[12]  Richard L. Tweedie,et al.  Markov Chains and Stochastic Stability , 1993, Communications and Control Engineering Series.

[13]  Robert Givan,et al.  Equivalence notions and model minimization in Markov decision processes , 2003, Artif. Intell..

[14]  Guy Pujolle,et al.  Introduction to queueing networks , 1987 .

[15]  E. Altman.  Constrained Markov Decision Processes , 1999 .

[16]  R. Cavazos-Cadena.  Finite-state approximations for denumerable state discounted markov decision processes , 1986 .