Action Dependent Dual Heuristic Programming Solution for the Dynamic Graphical Games

The context of graphical games is employed to solve the cooperative control problem for multi-agent systems interacting on graphs. Together with the need to have faster solution mechanisms urged for new approaches that employ the Dual Heuristic and Action Dependent Dual Heuristic Programming. This class of gradient-based solutions undergoes two main challenges. First, they have to use complex update expressions for the solving gradient-based structures. Second, they may overlook the local neighborhood information, if simpler costate expressions are enforced. A novel approach based on Action Dependent Dual Heuristic Programming is developed to solve the dynamic graphical games and to handle the aforementioned concerns. This adaptive learning approach is implemented online using means of value iteration and neural networks. The approximation of the optimal policy does not have priori knowledge about the agents' dynamics, while the value function gradient approximation is shown to depend only on the drift dynamics of the agents. The convergence results of the adaptive learning approach are highlighted by simulation example.