Embedding group and obstacle information in LSTM networks for human trajectory prediction in crowded scenes

Abstract Recurrent neural networks have shown good abilities in learning the spatio-temporal dependencies of moving agents in crowded scenes. Recently, they have been adopted to predict the motion of pedestrians by learning the relative motion of each individual in the crowd with respect to its neighbours. Crowded scenes present a wide variety of situations, which do not depend solely on the agents’ positions, but also relate to the structure of the environment, the density of the crowd, and the social relationships between pedestrians. In this work we propose a framework to improve the state-of-the-art models of crowd motion prediction by enriching the learning model with the social relationships between pedestrians walking in the crowd, as well as the layout of the environment. We observe that socially-related people tend to exhibit coherent motion patterns. Exploiting the motion coherency, we are able to cluster trajectories with similar motion properties and improve the trajectory prediction, especially at the group level. Furthermore, we incorporate into the model also the layout of the environment, to guarantee a more realistic and reliable learning framework. We evaluate our approach on standard crowd benchmark datasets, demonstrating its efficacy and applicability, improving the accuracy in trajectory prediction.