EvolveGraph: Heterogeneous Multi-Agent Multi-Modal Trajectory Prediction with Evolving Interaction Graphs

Multi-agent interacting systems are prevalent in the world, from pure physical systems to complicated social dynamic systems. In many applications, effective understanding of the environment and accurate trajectory prediction of interactive agents play a significant role in downstream tasks, such as decision and planning. In this paper, we propose a generic trajectory forecasting framework (named EvolveGraph) with explicit interaction modeling via a latent interaction graph among multiple heterogeneous, interactive agents. Considering the uncertainty and the possibility of different future behaviors, the model is designed to provide multi-modal prediction hypotheses. Since the interactions may be time-varying even with abrupt changes, and different modalities may have different interactions, we address the necessity and effectiveness of adaptively evolving the interaction graph and provide an effective solution. We also introduce a double-stage training pipeline which not only improves training efficiency and accelerates convergence, but also enhances model performance in terms of prediction error. The proposed framework is evaluated on multiple public benchmark datasets in various areas for trajectory prediction, where the agents cover on-road vehicles, pedestrians, cyclists and sports players. The experimental results illustrate that our approach achieves state-of-the-art performance in terms of prediction accuracy.