PWP3D: Real-time Segmentation and Tracking of 3D Objects

We formulate a probabilistic framework for simultaneous 2D segmentation and 2D– 3D pose tracking, using a known 3D model (of arbitrary shape) of the segmented object. Our technique is region-based; at each frame we maximise the discrimination between statistical foreground and background models, by adjusting the pose parameters iteratively. Unlike all previous work in 3D tracking, we use posterior membership probabilities for foreground and background pixels, rather than pixel likelihoods, and during periods of stable tracking we allow adaptation of the statistical foreground and background models. We support our ideas with a real-time implementation, and use this to generate experimental results on both real and artificial video sequences, with a number of 3D models, to showcase the qualities of our tracker, and to demonstrate the benefit of using pixel-wise posteriors rather than likelihoods.