Recovering upper-body motion using a reinitialization particle filter

Abstract. An important problem associated with particle filtering is track drift, which is caused by the inaccurate likelihood measurement function and state model; another problem is the heavy computation cost in high state space. A human body is an articulated object with high degrees of freedom, and humans often perform versatile unconstrained motions. Therefore, these two problems are invariably encountered when human motion is reconstructed from monocular video sequences using particle filters. To overcome these problems, we present a novel approach to recover three-dimensional human upper-body pose using a combination of a deterministic and stochastic method that takes full advantage of the benefits of both methods—highly accurate pose reconstruction and low computation cost. The reconstruction of the human upper-body pose is divided into two parts: global pose estimation and pose estimation of the remaining joints. The global pose is determined by solving a system of six nonlinear equations established by three scale invariant feature transform (SIFT) correspondences within the left and right shoulder segments. Estimation of the pose of the remaining joints is accomplished using two particle filters, one for left arm pose estimation and the other for right arm pose estimation. The image projection of the segment model is obtained by forward kinematics under a perspective camera model, and the likelihood measurement functions involving two features are presented to enhance the model fitting performance. To avoid track drift, the particle filters can reinitialize particle sets if most of the particles deviate from the target object. The probability distribution of the new particle set is modeled by a Gaussian mixture model, in which each Gaussian is determined by matched SIFT correspondences. Experimental results show that our proposed approach effectively and efficiently achieves human upper-body tracking and is more accurate than the standard particle filter.