Multimodal People Tracking and Trajectory Prediction based on Learned Generalized Motion Patterns

A sensor-based model of a service robot's environment is a prerequisite for interaction. Such a model should contain the positions of the robot's interaction partners. Additionally to the actual positions of the partners it is important for the service robot to predict their possible future positions. This knowledge could for example be used to realize efficient path planning for delivery tasks. In this paper we propose an extensible framework for systems that combine different sensor modalities in a general tracking system. Exemplarily, a tracking system is implemented that fuses tracking algorithms in laser range scans as well as in camera images by a particle filter. We point out the real-time capabilities of our tracking algorithms implementation. Practical experiments show that multimodality increases the system's robustness to incorrect measurements of single sensors. Furthermore, the observed trajectories are generalized to trajectory patterns by a novel method which uses self organizing maps. Those patterns are used to predict trajectories of the currently observed persons. It is also demonstrated that a self organizing map is suitable for learning and generalizing trajectories. Convenient predictions of future trajectories are presented which are deduced from these generalizations