Generating accurate and efficient predictions for the motion of the humans present in the scene is key to the development of effective motion planning algorithms for robots moving in promiscuous areas, where wrong planning decisions could generate safety hazard or simply make the presence of the robot “socially” unacceptable. Our approach to predict human motion is based on a neural network of a peculiar kind. Contrary to conventional deep neural networks, our network embeds in its structure the popular Social Force Model, a dynamic equation describing the motion in physical terms. This choice allows us to concentrate the learning phase in the aspects which are really unknown (i.e., the model’s parameters) and to keep the structure of the network simple and manageable. As a result, we are able to obtain a good prediction accuracy even by using a small and synthetically generated training set. Importantly, the prediction accuracy remains acceptable even when the network is applied in scenarios radically different from those for which it was trained. Finally, the choices of the network are “explainable”, as they can be interpreted in physical terms. Comparative and experimental results prove the effectiveness of the proposed approach.