Model-based sparse 3D reconstruction for online body tracking

In this paper a new approach to 3D human body tracking is proposed. A sparse 3D reconstruction of the subject to be tracked is made using a structured light system consisting of a precalibrated LCD projector and a camera. At a number of points-of-interest, easily detectable features are projected. The resulting sparse 3D reconstruction is used to estimate the body pose of the tracked person. This new estimate of the body pose is then fed back to the structured light system and allows to adapt the projected patterns, i.e. decide where to project features. Given the observations, a physical simulation is used to estimate the body pose by attaching forces to the limbs of the body model. The sparse 3D observations are augmented by denser silhouette information, in order to make the tracking more robust. Experiments demonstrate the feasibility of the proposed approach and show that the high speeds that are required due to the closed feedback loop can be achieved.