Human trajectory prediction in crowded scene using social-affinity Long Short-Term Memory

Abstract Object tracking in crowded spaces is a challenging but very important task in computer vision applications. However, due to interactions among large-scale pedestrians and common social rules, predicting the complex human mobility in a crowded scene becomes difficult. This paper proposes a novel human trajectory prediction model in a crowded scene called the social-affinity LSTM model. Our model can learn general human mobility patterns and predict individual’ s trajectories based on their past positions, in particular, with the influence of their neighbors in the Social Affinity Map (SAM). The SAM clusters the relative positions of surrounding individuals, and represents the distribution of the relative positions by different bins with semantic descriptions. We formulate the problem of trajectory prediction together with interactions among people as a sequence generation task with social affinity. The proposed model utilizes the LSTM to learn general human moving patterns as well as the Social Affinity Map to connect neighbors with a weight matrix corresponding to SAM bins for learning the social dependencies between correlated pedestrians. By capturing the object’ s past positions and connecting the hidden states of it’ s neighbors in different SAM bins with different elements of the weight matrix, the social-affinity LSTM is able to predict the trajectory of each pedestrian with its own features and neighbors’ influence. We compare the performance of our method with the Social LSTM model on several public datasets. Our model outperforms state-of-the-art methods on these datasets with the best results, especially the datasets with more social affinity phenomena.