Heterogeneous Formation Control of Multiple Rotorcrafts with Unknown Dynamics using Reinforcement Learning*

In this paper, a distributed model-free solution to the leader-follower formation control of heterogeneous multi-agent system is proposed using reinforcement learning. The multi-agent system consists of multiple rotorcrafts, including a virtual leader and multiple followers, and no knowledge of the dynamics of leaders and followers is assumed to be known a priori. The formation controller problem is first formulated as an optimal output regulation problem. A discounted performance function is then introduced to guarantee that the tracking error asymptotically converges to zero, and an online off-policy reinforcement learning algorithm is finally proposed to solve the optimal output problem online and using data generated along the agents’ trajectories. A simulation example is provided to validate the effectiveness of the proposed control method.