论文信息 - Approximate Linear-Programming Algorithms for Graph-Based Markov Decision Processes

Approximate Linear-Programming Algorithms for Graph-Based Markov Decision Processes

In this article, we consider a form of compact representation of MDP based on graphs, and we propose an approximate solution algorithm derived from this representation. The approach we propose belongs to the family of Approximate Linear Programming methods, but the graph-structure we assume allows it to become particularly efficient. The proposed method complexity is linear in the number of variables in the graph and only exponential in the width of a dependency graph among variables.

Régis Sabbadin | Nicklas Forsell

[1] Nathalie Peyrard,et al. Mean Field Approximation of the Policy Iteration Algorithm for Graph-Based Markov Decision Processes , 2006, ECAI.

[2] Craig Boutilier,et al. Stochastic dynamic programming with factored representations , 2000, Artif. Intell..

[3] H. Daduna,et al. Controlled Markov Elds with Nite State Space on Graphs Controlled Markov Elds with Nite State Space on Graphs , 2007 .

[4] D. Koller,et al. Planning under uncertainty in complex structured environments , 2003 .

[5] Martin L. Puterman,et al. Markov Decision Processes: Discrete Stochastic Dynamic Programming , 1994 .

[6] Shobha Venkataraman,et al. Efficient Solution Algorithms for Factored MDPs , 2003, J. Artif. Intell. Res..

[7] Benjamin Van Roy,et al. On Constraint Sampling in the Linear Programming Approach to Approximate Dynamic Programming , 2004, Math. Oper. Res..