A multiple object tracking approach that combines colour and depth information using a confidence measure

Multiple object tracking is a difficult task, especifically when there is not an explicit model of the object being tracked or when it is not possible to estimate the background of the scene. This paper proposes a novel approach for multiple target tracking. It works without background information and uses an original method that merges colour and depth information. The fusion of both pieces of information is created taking into account a confidence measure about the depth information. The method proposed employs a multiple particle filter approach in which particle weights are modified by an interaction factor in order to avoid the ''coalescence'' problem. In addition, the method performs as a pure colour-based technique when no disparity information is available, and takes advantage of depth information to enhance tracking whenever it is possible. Our technique is compared with two pure colour-based tracking approaches (the particle filtering method proposed by Nummiaro et al. [Nummiaro, K., Koller-Meier, E., Van Gool, L., 2003. An adaptive color-based particle filter. Image and Vision Computing, 21, 99-110] and the Kalman/mean-shift tracker [Comaniciu, D., Ramesh, V. 2000. Mean shift and optimal prediction for efficient object tracking. In: IEEE International Conference on Image Processing (ICIP'00), vol. 3, pp. 70-73]) and a pure stereo-based approach derived from our problem formulation. The performance of the four algorithms is tested using several colour-with-depth sequences of images showing different coloured targets in complex situations. The results show that our proposal is able to track the targets in case of complex backgrounds and to properly determine the size of their projections in the camera image (while the other methods fail). Besides, the proposed method is fast enough for real-time applications and the use of 3D information helps to track several targets simultaneously without confusing their identities.

[1]  S. Godsill,et al.  Monte Carlo filtering for multi target tracking and data association , 2005, IEEE Transactions on Aerospace and Electronic Systems.

[2]  Takeo Kanade,et al.  People detection and tracking in high resolution panoramic video mosaic , 2004, 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (IEEE Cat. No.04CH37566).

[3]  Luc Van Gool,et al.  An adaptive color-based particle filter , 2003, Image Vis. Comput..

[4]  Michael Isard,et al.  Contour Tracking by Stochastic Propagation of Conditional Density , 1996, ECCV.

[5]  James L. Crowley,et al.  Face-Tracking and Coding for Video Compression , 1999, ICVS.

[6]  Frank Dellaert,et al.  MCMC-based particle filtering for tracking a variable number of interacting targets , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[7]  Irfan Essa,et al.  Robust Tracking of People by a Mobile Robotic Agent , 1999 .

[8]  François Chaumette,et al.  2½D visual servoing , 1999, IEEE Trans. Robotics Autom..

[9]  Manolis I. A. Lourakis,et al.  Three-dimensional tracking of multiple skin-colored regions by a moving stereoscopic system. , 2004, Applied optics.

[10]  Jake K. Aggarwal,et al.  Stochastic Analysis of Stereo Quantization Error , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[11]  Stanley T. Birchfield,et al.  Elliptical head tracking using intensity gradients and color histograms , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[12]  Justus H. Piater,et al.  Tracking by Cluster Analysis of Feature Points and Multiple Particle Filters , 2005, ICAPR.

[13]  Darius Burschka,et al.  Advances in Computational Stereo , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[14]  Justus H. Piater,et al.  Tracking by cluster analysis of feature points using a mixture particle filter , 2005, IEEE Conference on Advanced Video and Signal Based Surveillance, 2005..

[15]  Ramakant Nevatia,et al.  Stereo Error Detection, Correction, and Evaluation , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[16]  Horst-Michael Groß,et al.  An approach to multi-modal human-machine interaction for intelligent service robots , 2003, Robotics Auton. Syst..

[17]  Dorin Comaniciu,et al.  Mean shift and optimal prediction for efficient object tracking , 2000, Proceedings 2000 International Conference on Image Processing (Cat. No.00CH37101).

[18]  Christian Micheloni,et al.  Video security for ambient intelligence , 2005, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[19]  Dorin Comaniciu,et al.  Real-time tracking of non-rigid objects using mean shift , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[20]  Reinhard Koch,et al.  Realtime multi-camera person tracking for immersive environments , 2004, IEEE 6th Workshop on Multimedia Signal Processing, 2004..

[21]  Patrick Pérez,et al.  Maintaining multimodality through mixture tracking , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[22]  James J. Little,et al.  A Boosted Particle Filter: Multitarget Detection and Tracking , 2004, ECCV.

[23]  G. Kitagawa.  Monte Carlo Filter and Smoother for Non-Gaussian Nonlinear State Space Models , 1996 .

[24]  Rafael Muñoz-Salinas,et al.  A multi-agent system architecture for mobile robot navigation based on fuzzy and visual behaviour , 2005, Robotica.

[25]  Neil J. Gordon,et al.  Bayesian State Estimation for Tracking and Guidance Using the Bootstrap Filter , 1993 .

[26]  M. Brunig,et al.  Face detection and tracking for video coding applications , 2000, Conference Record of the Thirty-Fourth Asilomar Conference on Signals, Systems and Computers (Cat. No.00CH37154).

[27]  Trevor Darrell,et al.  Plan-view trajectory estimation with dense stereo background models , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[28]  Trevor Darrell,et al.  Integrated Person Tracking Using Stereo, Color, and Pattern Detection , 2000, International Journal of Computer Vision.

[29]  Patrick Pérez,et al.  Sequential Monte Carlo methods for multiple target tracking and data fusion , 2002, IEEE Trans. Signal Process..

[30]  James D. Foley,et al.  Fundamentals of interactive computer graphics , 1982 .

[31]  Manabu Hashimoto,et al.  Multiple-person tracker with a fixed slanting stereo camera , 2004, Sixth IEEE International Conference on Automatic Face and Gesture Recognition, 2004. Proceedings..

[32]  Neil A. Thacker,et al.  The Bhattacharyya metric as an absolute similarity measure for frequency coded data , 1998, Kybernetika.

[33]  Rafael Muñoz-Salinas,et al.  People Detection and Tracking Through Stereo Vision for Human-Robot Interaction , 2005, MICAI.

[34]  T. Kailath.  The Divergence and Bhattacharyya Distance Measures in Signal Selection , 1967 .

[35]  Michael Isard,et al.  CONDENSATION—Conditional Density Propagation for Visual Tracking , 1998, International Journal of Computer Vision.