3D object tracking using shape-encoded particle propagation

We present a comprehensive treatment of 3D object tracking by posing it as a nonlinear state estimation problem. The measurements are derived using the outputs of shape-encoded filters. The nonlinear state estimation is performed by solving the Zakai equation, and we use the branching particle propagation method for computing the solution. The unnormalized conditional density for the solution to the Zakai equation is realized by the weight of the particle. We first sample a set of particles approximating the initial distribution of the state vector conditioned on the observations, where each particle encodes the set of geometric parameters of the object. The weight of the particle represents geometric and temporal fit, which is computed bottom-up from the raw image using a shape-encoded filter. The particles branch so that the mean number of offspring is proportional to the weight. Time update is handled by employing a second-order motion model, combined with local stochastic search to minimize the prediction error. The prediction adjustment suggested by system identification theory is empirically verified to contribute to global stability. The amount of diffusion is effectively adjusted using a Kalman updating of the covariance matrix. WE have successfully applied this method to human head tracking, where we estimate head motion and compute structure using simple head and facial feature models.

[1]  G. Kitagawa.  Monte Carlo Filter and Smoother for Non-Gaussian Nonlinear State Space Models , 1996 .

[2]  M. Zakai.  On the optimal filtering of diffusion processes , 1969 .

[3]  Dan Crisan,et al.  Convergence of a Branching Particle Method to the Solution of the Zakai Equation , 1998, SIAM J. Appl. Math..

[4]  Hankyu Moon.  Optimal shape detection , 2000, Proceedings 2000 International Conference on Image Processing (Cat. No.00CH37101).

[5]  Alex Pentland,et al.  Recursive Estimation of Motion, Structure, and Focal Length , 1995, IEEE Trans. Pattern Anal. Mach. Intell..

[6]  Andrew Blake,et al.  Articulated body motion capture by annealed particle filtering , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[7]  Hironobu Fujiyoshi,et al.  Moving target classification and tracking from real-time video , 1998, Proceedings Fourth IEEE Workshop on Applications of Computer Vision. WACV'98 (Cat. No.98EX201).

[8]  Lennart Ljung,et al.  The Extended Kalman Filter as a Parameter Estimator for Linear Systems , 1979 .

[9]  R. Chellappa,et al.  Recursive 3-D motion estimation from a monocular image sequence , 1990 .

[10]  Michael J. Black,et al.  EigenTracking: Robust Matching and Tracking of Articulated Objects Using a View-Based Representation , 1996, ECCV.

[11]  A. Bensoussan.  Stochastic Control of Partially Observable Systems , 1992 .

[12]  Rama Chellappa,et al.  Simultaneous tracking and verification via sequential posterior estimation , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[13]  Michael Isard,et al.  Object localization by Bayesian correlation , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[14]  Z. S. Haddad,et al.  Filtering Image Records Using Wavelets and the Zakai Equation , 1995, IEEE Trans. Pattern Anal. Mach. Intell..

[15]  Jun S. Liu,et al.  Sequential Monte Carlo methods for dynamic systems , 1997 .

[16]  James L. Crowley,et al.  Probabilistic recognition of activity using local appearance , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).