Idempotent algorithms for discrete-time stochastic control through distributed dynamic programming

Previously, idempotent methods have been found to be extremely fast for solution of dynamic programming equations associated with deterministic control problems. The original methods exploited the idempotent (e.g., max-plus) linearity of the associated semigroup operator. However, it is now seen that the curse-of-dimensionality-free idempotent methods do not require this linearity. Here, we see that idempotent methods may be used to solve some classes of stochastic control problems. The key is the use of the idempotent distributive property.We demonstrate this approach for a class of nonlinear, discrete-time stochastic control problems.

[1]  A. Vannucci,et al.  BICS Bath Institute for Complex Systems A note on time-dependent DiPerna-Majda measures , 2008 .

[2]  R. Schilling Measures, Integrals and Martingales: Frontmatter , 2006 .

[3]  William M. McEneaney,et al.  Value-Based Tasking Controllers for Sensing Assets , 2008 .

[4]  W. Fleming,et al.  Controlled Markov processes and viscosity solutions , 1992 .

[5]  William M. McEneaney,et al.  A Max-Plus-Based Algorithm for a Hamilton--Jacobi--Bellman Equation of Nonlinear Filtering , 2000, SIAM J. Control. Optim..

[6]  William M. McEneaney,et al.  Max-plus methods for nonlinear control and estimation , 2005 .

[7]  V. Kolokoltsov,et al.  Idempotent Analysis and Its Applications , 1997 .

[8]  William M. McEneaney,et al.  Max-Plus Eigenvector Methods for Nonlinear Hinfinity Problems: Error Analysis , 2004, SIAM J. Control. Optim..

[9]  J. Quadrat,et al.  Duality and separation theorems in idempotent semimodules , 2002, math/0212294.

[10]  W.M. McEneaney,et al.  Curse-of-complexity attenuation in the curse-of-dimensionality-free method for HJB PDEs , 2008, 2008 American Control Conference.

[11]  William M. McEneaney,et al.  A Curse-of-Dimensionality-Free Numerical Method for Solution of Certain HJB PDEs , 2007, SIAM J. Control. Optim..

[12]  Gerald B. Folland,et al.  Real Analysis: Modern Techniques and Their Applications , 1984 .

[13]  Kellen Petersen August Real Analysis , 2009 .

[14]  Stéphane Gaubert,et al.  The Max-Plus Finite Element Method for Solving Deterministic Optimal Control Problems: Basic Properties and Convergence Analysis , 2008, SIAM J. Control. Optim..