论文信息 - On polynomial cases of the unichain classification problem for Markov Decision Processes

On polynomial cases of the unichain classification problem for Markov Decision Processes

The Unichain classification problem detects whether a finite state and action MDP is unichain under all deterministic policies. This problem is NP-hard. This paper provides polynomial algorithms for this problem when there is a state that is either recurrent under all deterministic policies or absorbing under some action.

Eugene A. Feinberg | Fenghsu Yang | E. Feinberg | Fenghsu Yang

[1] John N. Tsitsiklis,et al. NP-Hardness of checking the unichain condition in average cost MDPs , 2006, Oper. Res. Lett..

[2] Paul Seymour,et al. Graph Structure Theory, Proceedings of a AMS-IMS-SIAM Joint Summer Research Conference on Graph Minors held June 22 to July 5, 1991, at the University of Washington, Seattle, USA , 1993, Graph Structure Theory.

[3] Ronald A. Howard,et al. Dynamic Programming and Markov Processes , 1960 .

[4] Martin L. Puterman,et al. Markov Decision Processes: Discrete Stochastic Dynamic Programming , 1994 .

[5] Onésimo Hernández-Lerma,et al. Controlled Markov Processes , 1965 .

[6] Cyrus Derman,et al. Finite State Markovian Decision Processes , 1970 .

[7] L. C. M. Kallenberg,et al. Classification Problems in MDPs , 2002 .

[8] J. Bather. Optimal decision procedures for finite markov chains. Part I: Examples , 1973, Advances in Applied Probability.

[9] J. Bather. Optimal decision procedures for finite Markov chains. Part III: General convex systems , 1973 .

[10] U. Rieder,et al. Markov Decision Processes , 2010 .

[11] L. Kallenberg. Finite State and Action MDPS , 2003 .

[12] Spyros A. Reveliotis,et al. Optimal Node Visitation in Stochastic Digraphs , 2008, IEEE Transactions on Automatic Control.

[13] William McCuaig,et al. Intercyclic digraphs , 1991, Graph Structure Theory.

[14] Eugene A. Feinberg,et al. Handbook of Markov Decision Processes , 2002 .