Cooperative Multitarget Tracking With Efficient Split and Merge Handling

For applications such as behavior recognition it is important to maintain the identity of multiple targets, while tracking them in the presence of splits and merges, or occlusion of the targets by background obstacles. Here we propose an algorithm to handle multiple splits and merges of objects based on dynamic programming and a new geometric shape matching measure. We then cooperatively combine Kalman filter-based motion and shape tracking with the efficient and novel geometric shape matching algorithm. The system is fully automatic and requires no manual input of any kind for initialization of tracking. The target track initialization problem is formulated as computation of shortest paths in a directed and attributed graph using Dijkstra's shortest path algorithm. This scheme correctly initializes multiple target tracks for tracking even in the presence of clutter and segmentation errors which may occur in detecting a target. We present results on a large number of real world image sequences, where upto 17 objects have been tracked simultaneously in real-time, despite clutter, splits, and merges in measurements of objects. The complete tracking system including segmentation of moving objects works at 25 Hz on 352times288 pixel color image sequences on a 2.8-GHz Pentium-4 workstation

[1]  Hai Tao,et al.  Object Tracking with Bayesian Estimation of Dynamic Layer Representations , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[2]  Mubarak Shah,et al.  Tracking and Object Classification for Automated Surveillance , 2002, ECCV.

[3]  Ingemar J. Cox,et al.  An Efficient Implementation of Reid's Multiple Hypothesis Tracking Algorithm and Its Evaluation for the Purpose of Visual Tracking , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[4]  Jesús García,et al.  A Multitarget Tracking Video System Based on Fuzzy and Neuro-Fuzzy Techniques , 2005, EURASIP J. Adv. Signal Process..

[5]  Terry E. Weymouth,et al.  Using Dynamic Programming For Minimizing The Energy Of Active Contours In The Presence Of Hard Constraints , 1988, [1988 Proceedings] Second International Conference on Computer Vision.

[6]  Gregory D. Hager,et al.  Probabilistic Data Association Methods for Tracking Complex Visual Objects , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[7]  Neil J. Gordon,et al.  A tutorial on particle filters for online nonlinear/non-Gaussian Bayesian tracking , 2002, IEEE Trans. Signal Process..

[8]  Alberto Martelli,et al.  Optimal Smoothing in Picture Processing: An Application to Fingerprints , 1971, IFIP Congress.

[9]  J. Olivo-Marin,et al.  Split and merge data association filter for dense multi-target tracking , 2004, Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004..

[10]  Martin D. Levine,et al.  Intermediate level picture interpretation using complete two-dimensional models , 1981 .

[11]  Greg Welch,et al.  An Introduction to Kalman Filter , 1995, SIGGRAPH 2001.

[12]  Andrew Blake,et al.  Surface Orientation and Time to Contact from Image Divergence and Deformation , 1992, ECCV.

[13]  R. K. Shyamasundar,et al.  Introduction to algorithms , 1996 .

[14]  Kuntal Sengupta,et al.  Framework for real-time behavior interpretation from traffic video , 2005, IEEE Transactions on Intelligent Transportation Systems.

[15]  Pankaj Kumar,et al.  Queue based fast background modelling and fast hysteresis thresholding for better foreground segmentation , 2003, Fourth International Conference on Information, Communications and Signal Processing, 2003 and the Fourth Pacific Rim Conference on Multimedia. Proceedings of the 2003 Joint.

[16]  Andrew W. Fitzgibbon,et al.  Direct Least Square Fitting of Ellipses , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[17]  Amir Averbuch,et al.  Interacting Multiple Model Methods in Target Tracking: A Survey , 1988 .

[18]  Rama Chellappa,et al.  Automatic feature point extraction and tracking in image sequences for unknown camera motion , 1993, 1993 (4th) International Conference on Computer Vision.

[19]  V. Kovalevsky.  Image Pattern Recognition , 1980, Springer New York.

[20]  Larry S. Davis,et al.  W4S : A real-time system for detecting and tracking people in 2 D , 1998, eccv 1998.

[21]  R. Deriche,et al.  Geodesic active regions for motion estimation and tracking , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[22]  Vladimir I. Levenshtein,et al.  Binary codes capable of correcting deletions, insertions, and reversals , 1965 .

[23]  Hiromitsu Yamada,et al.  Recognition of Kidney Glomerulus by Dynamic Programming Matching Method , 1988, IEEE Trans. Pattern Anal. Mach. Intell..

[24]  Yaakov Bar-Shalom,et al.  Tracking methods in a multitarget environment , 1978 .

[25]  T. Başar,et al.  A New Approach to Linear Filtering and Prediction Problems , 2001 .

[26]  Azriel Rosenfeld,et al.  Tracking Groups of People , 2000, Comput. Vis. Image Underst..

[27]  Larry S. Davis,et al.  W/sup 4/: Who? When? Where? What? A real time system for detecting and tracking people , 1998, Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition.

[28]  Ramakant Nevatia,et al.  Event Detection and Analysis from Video Streams , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[29]  Alok Gupta,et al.  Dynamic Programming for Detecting, Tracking, and Matching Deformable Contours , 1995, IEEE Trans. Pattern Anal. Mach. Intell..

[30]  Samuel S. Blackman,et al.  Multiple-Target Tracking with Radar Applications , 1986 .

[31]  Ingemar J. Cox,et al.  A review of statistical data association techniques for motion correspondence , 1993, International Journal of Computer Vision.

[32]  Vladimir Kovalevsky,et al.  Sequential optimization in pattern recognition and pattern description , 1968, IFIP Congress.

[33]  Sean R Eddy,et al.  What is dynamic programming? , 2004, Nature Biotechnology.

[34]  Rachid Deriche,et al.  Geodesic Active Contours and Level Sets for the Detection and Tracking of Moving Objects , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[35]  Jack K. Wolf,et al.  Finding the best set of K paths through a trellis with application to multitarget tracking , 1989 .

[36]  Naonori Ueda,et al.  Tracking Moving Contours Using Energy-Minimizing Elastic Contour Models , 1992, ECCV.

[37]  Martin A. Fischler,et al.  The Representation and Matching of Pictorial Structures , 1973, IEEE Transactions on Computers.

[38]  Jesús García,et al.  Fuzzy data association for image-based tracking in dense scenarios , 2002, 2002 IEEE World Congress on Computational Intelligence. 2002 IEEE International Conference on Fuzzy Systems. FUZZ-IEEE'02. Proceedings (Cat. No.02CH37291).