Flexible approximation of structured interactions in decentralized Markov decision processes

Our work is motivated by cooperative planning problems where agents can affect each others’ transitions and rewards, and so benefit from coordinating their actions, but in doing so must account for durational uncertainty in these actions. To reason about this uncertainty efficiently, agents can employ temporal decoupling (a paradigm that has been explored in a variety of restricted contexts [2, 3]) to constrain interactions to occur by selected time points, representing the uncertain occurrence for each time point with a probabilistic promise [5]. Here we summarize a reformulation of Becker’s Event-driven DEC-MDP problems [1] that uses commitment models to exploit temporal structure. We argue that, in addition to representing optimal solutions, our approach enables more efficient, scalable computation of approximate solutions and a natural flexibility by which interactions can be modeled with more or less detail.