论文信息 - Folding Algorithm for Policy Evaluation for Markov Decision Processes With Quasi-Birth Death Structure

Folding Algorithm for Policy Evaluation for Markov Decision Processes With Quasi-Birth Death Structure

This technical note presents a new numerical procedure for policy evaluation of Stochastic Shortest Path Markov Decision Processes (MDPs) having a level independent Quasi Birth-Death structure. The algorithm is derived using a method analogous to the folding method of Ye and Li (1994). The computational complexity is O(M3log2N) + O(M2N), where the process has N levels and M phases. A simple example involving the control of two queues is presented to illustrate the application of this efficient policy evaluation algorithm to compare and rank control policies.

Langford B. White | Yassir Yassir | Yassir A. Yassir | L. White

[1] Vaidyanathan Ramaswami,et al. Introduction to Matrix Analytic Methods in Stochastic Modeling , 1999, ASA-SIAM Series on Statistics and Applied Mathematics.

[2] Dimitri P. Bertsekas,et al. Dynamic Programming and Optimal Control, Two Volume Set , 1995 .

[3] Marcel F. Neuts,et al. Matrix-Geometric Solutions in Stochastic Models , 1981 .

[4] Christos G. Cassandras,et al. Optimal scheduling of two competing queues with blocking , 1991 .

[5] San-qi Li,et al. Folding algorithm: a computational method for finite QBD processes with level-dependent transitions , 1994, IEEE Trans. Commun..

[6] Langford B. White,et al. A New Policy Evaluation Algorithm for Markov Decision Processes with Quasi Birth-Death Structure , 2005 .