Automatic Tracking of a Large Number of Moving Targets in 3D

This paper addresses the problem of tracking a large number of targets moving in 3D space using multiple calibrated video cameras. Most visual details of the targets are lost in the captured images because of limited image resolution, and the remainder can be easily corrupted due to frequent occlusion, which makes it difficult to determine both across-view and temporal correspondences. We propose a fully automatic tracking system that is capable of detecting and tracking a large number of flying targets in a 3D volume. The system includes a 3D tracking method in the framework of particle filter. Different from previous 2D tracking methods, the proposed method models the 3D attributes of targets and furthest collects weak visual information from multiple views, which makes the tracker robust against occlusion and distraction. The ambiguities in stereo matching when initializing trackers are handled by an effective multiple hypothesis generation and verification mechanism. The whole system is fully automatic in dealing with variable number of targets and robust against detection and matching errors. Our system has successfully been used by biologists to recover the 3D trajectories of hundreds of fruit flies flying freely in a 3D volume.