Non-parametric motion-priors for flow understanding

We present a novel method for extracting the dominant dynamic properties of crowded scenes from a single, static, uncalibrated camera using a codebook of tracklets. Our approach relies only on tracklets of fixed length which are generated based on sparse optical flow. A grid of points is placed on the image plane and local meanshift clustering is employed to extract dominant directions of tracklets in the neighborhood. A Gaussian Process (GP) is fitted to each tracklet resulting in a codebook, with each codeword representing a local motion model. At test time, a mixture of weighted local GP experts is applied, providing multimodal density estimates for next object location and simulation of full object trajectories. Our scenarios come from challenging crowded scenes, from which we extract dominant local motion-patterns and use the model to simulate full object trajectories. In addition, we apply the learnt model to multiple object tracking. Random trajectories are sampled from the model that match the learnt scene dynamics. Minimum Description Length (MDL) is employed to pick the best trajectories in order to associate sparse detections over short time windows. Also, we modify a state-of-the-art multiple object tracking algorithm leading to significant improvement. Our results compare favorably to a state-of-the-art algorithm and we introduce a new challenging dataset for multiple object tracking.