Model-Free Distributed Consensus Control Based on Actor–Critic Framework for Discrete-Time Nonlinear Multiagent Systems

Conventionally, as the system’s dynamics is known, the optimal consensus control problem relies on solving the coupled Hamilton–Jacobi–Bellman (HJB) equations. In this paper, with the unknown system dynamics being considered, a local <inline-formula> <tex-math notation="LaTeX">${Q}$ </tex-math></inline-formula>-function-based adaptive dynamic programming method is put forward to deal with the optimal consensus control problem for unknown discrete-time nonlinear multiagent systems by approximating the solutions of the coupled HJB equations. First, a local <inline-formula> <tex-math notation="LaTeX">${Q}$ </tex-math></inline-formula>-function is defined, which considers the local consensus error and the actions of the agent and its neighbors. Using the <inline-formula> <tex-math notation="LaTeX">${Q}$ </tex-math></inline-formula>-function, it is convenient to get the derivatives with regard to the weights of the consensus control policies, even without the model of system dynamics. Then, with the defined local <inline-formula> <tex-math notation="LaTeX">${Q}$ </tex-math></inline-formula>-function, a distributed policy iteration technique is developed, which is theoretically proved to be convergent to the solutions of the coupled HJB equations. An actor–critic neural network framework for implementing the developed model-free optimal consensus control method is constructed to approximate the local <inline-formula> <tex-math notation="LaTeX">${Q}$ </tex-math></inline-formula>-functions and the control policies. Finally, the feasibility and effectiveness of the developed method are verified by a series of simulations.