An online system for multiple interacting targets tracking: Fusion of laser and vision, tracking and learning

Multitarget tracking becomes significantly more challenging when the targets are in close proximity or frequently interact with each other. This article presents a promising online system to deal with these problems. The novelty of this system is that laser and vision are integrated with tracking and online learning to complement each other in one framework: when the targets do not interact with each other, the laser-based independent trackers are employed and the visual information is extracted simultaneously to train some classifiers online for “possible interacting targets”. When the targets are in close proximity, the classifiers learned online are used alongside visual information to assist in tracking. Therefore, this mode of cooperation not only deals with various tough problems encountered in tracking, but also ensures that the entire process can be completely online and automatic. Experimental results demonstrate that laser and vision fully display their respective advantages in our system, and it is easy for us to obtain a good trade-off between tracking accuracy and the time-cost factor.

[1]  Roland Siegwart,et al.  Human detection using multimodal and multidimensional features , 2008, 2008 IEEE International Conference on Robotics and Automation.

[2]  Gregory D. Hager,et al.  Probabilistic Data Association Methods for Tracking Complex Visual Objects , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[3]  Stefan Carlsson,et al.  Tracking and Labelling of Interacting Multiple Targets , 2006, ECCV.

[4]  Horst Bischof,et al.  On-line Boosting and Vision , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[5]  Shai Avidan.  Ensemble Tracking , 2007, IEEE Trans. Pattern Anal. Mach. Intell..

[6]  Ryosuke Shibasaki,et al.  Multi-modal tracking of people using laser scanners and video camera , 2008, Image Vis. Comput..

[7]  Xuan Song,et al.  Bayesian fusion of laser and vision for multiple People Detection and tracking , 2008, 2008 SICE Annual Conference.

[8]  Gérard G. Medioni,et al.  Multiple Target Tracking Using Spatio-Temporal Markov Chain Monte Carlo Data Association , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[9]  Yair Weiss,et al.  Learning object detection from a small number of examples: the importance of good features , 2004, CVPR 2004.

[10]  Wolfram Burgard,et al.  Fast face detection for mobile robots by integrating laser range data with vision , 2003 .

[11]  Frank Dellaert,et al.  MCMC Data Association and Sparse Factorization Updating for Real Time Multitarget Tracking with Merged and Multiple Measurements , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[12]  James J. Little,et al.  A Boosted Particle Filter: Multitarget Detection and Tracking , 2004, ECCV.

[13]  Wolfram Burgard,et al.  People Tracking with Mobile Robots Using Sample-Based Joint Probabilistic Data Association Filters , 2003, Int. J. Robotics Res..

[14]  Ba-Ngu Vo,et al.  The Cardinality Balanced Multi-Target Multi-Bernoulli Filter and Its Implementations , 2009, IEEE Transactions on Signal Processing.

[15]  Leonidas J. Guibas,et al.  Fourier Theoretic Probabilistic Inference over Permutations , 2009, J. Mach. Learn. Res..

[16]  Wolfram Burgard,et al.  Efficient people tracking in laser range data using a multi-hypothesis leg-tracker with adaptive occlusion probabilities , 2008, 2008 IEEE International Conference on Robotics and Automation.

[17]  Pascal Fua,et al.  Tracking multiple people under global appearance constraints , 2011, 2011 International Conference on Computer Vision.

[18]  SongXuan,et al.  An online system for multiple interacting targets tracking , 2013 .

[19]  Ramakant Nevatia,et al.  Global data association for multi-object tracking using network flows , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[20]  Huosheng Hu,et al.  Vision and Laser Data Fusion for Tracking People with a Mobile Robot , 2006, 2006 IEEE International Conference on Robotics and Biomimetics.

[21]  Grzegorz Cielniak,et al.  Improved data association and occlusion handling for vision-based people tracking by mobile robots , 2007, IROS.

[22]  Yair Weiss,et al.  Learning object detection from a small number of examples: the importance of good features , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[23]  Xuan Song,et al.  Probabilistic Detection-based Particle Filter for Multi-target Tracking , 2008, BMVC.

[24]  Xuan Song,et al.  Tracking interacting targets with laser scanner via on-line supervised learning , 2008, 2008 IEEE International Conference on Robotics and Automation.

[25]  Dirk Schulz,et al.  A Probabilistic Exemplar Approach to Combine Laser and Vision for Person Tracking , 2006, Robotics: Science and Systems.

[26]  Shai Avidan,et al.  Ensemble Tracking , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[27]  Yakov Bar-Shalom,et al.  Multitarget-Multisensor Tracking: Principles and Techniques , 1995 .

[28]  Xuan Song,et al.  An online approach: Learning-Semantic-Scene-by-Tracking and Tracking-by-Learning-Semantic-Scene , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[29]  R. Mahler.  PHD filters of higher order in target number , 2007 .

[30]  Yuan Li,et al.  Tracking in Low Frame Rate Video: A Cascade Particle Filter with Discriminative Observers of Different Lifespans , 2007, CVPR.

[31]  James J. Little,et al.  A Linear Programming Approach for Multiple Object Tracking , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[32]  Patrick Pérez,et al.  Maintaining multimodality through mixture tracking , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[33]  Ramakant Nevatia,et al.  Tracking multiple humans in complex situations , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[34]  I. R. Goodman,et al.  Mathematics of Data Fusion , 1997 .

[35]  Wei-Yin Loh,et al.  Classification and regression trees , 2011, WIREs Data Mining Knowl. Discov..

[36]  Gregory D. Hager,et al.  Probabilistic data association methods in visual tracking of groups , 2004, CVPR 2004.

[37]  R. Mahler,et al.  PHD filters of higher order in target number , 2006, IEEE Transactions on Aerospace and Electronic Systems.

[38]  Matthias Scheutz,et al.  Fast, reliable, adaptive, bimodal people tracking for indoor environments , 2004, 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (IEEE Cat. No.04CH37566).

[39]  Dan Schonfeld,et al.  Real-time interactively distributed multi-object tracking using a magnetic-inertia potential model , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[40]  R. Mahler.  Multitarget Bayes filtering via first-order multitarget moments , 2003 .

[41]  Ray A. Jarvis,et al.  Panoramic Vision and Laser Range Finder Fusion for Multiple Person Tracking , 2006, 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[42]  Roland Siegwart,et al.  Multisensor on-the-fly localization using laser and vision , 2000, Proceedings. 2000 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2000) (Cat. No.00CH37113).

[43]  Yuan Li,et al.  Tracking in Low Frame Rate Video: A Cascade Particle Filter with Discriminative Observers of Different Lifespans , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[44]  Grzegorz Cielniak,et al.  Improved data association and occlusion handling for vision-based people tracking by mobile robots , 2007, 2007 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[45]  Ying Wu,et al.  Decentralized multiple target tracking using netted collaborative autonomous trackers , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[46]  A. Doucet,et al.  Sequential Monte Carlo methods for multitarget filtering with random finite sets , 2005, IEEE Transactions on Aerospace and Electronic Systems.

[47]  Y. Bar-Shalom.  Tracking and data association , 1988 .

[48]  Stefan Carlsson,et al.  Multi-Target Tracking - Linking Identities using Bayesian Network Inference , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[49]  Paulo Peixoto,et al.  Semantic fusion of laser and vision in pedestrian detection , 2010, Pattern Recognit..

[50]  Donald Reid.  An algorithm for tracking multiple targets , 1978 .

[51]  Ba-Ngu Vo,et al.  The Gaussian Mixture Probability Hypothesis Density Filter , 2006, IEEE Transactions on Signal Processing.

[52]  Songhwai Oh,et al.  Markov chain Monte Carlo data association for general multiple-target tracking problems , 2004, 2004 43rd IEEE Conference on Decision and Control (CDC) (IEEE Cat. No.04CH37601).

[53]  James J. Little,et al.  Robust Visual Tracking for Multiple Targets , 2006, ECCV.

[54]  Xuan Song,et al.  Fusion of laser and vision for multiple targets tracking via on-line learning , 2010, 2010 IEEE International Conference on Robotics and Automation.

[55]  Simon J. Godsill,et al.  On sequential Monte Carlo sampling methods for Bayesian filtering , 2000, Stat. Comput..

[56]  Ryosuke Shibasaki,et al.  Detection and tracking of multiple pedestrians by using laser range scanners , 2007, 2007 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[57]  Ronald P. S. Mahler,et al.  Statistical Multisource-Multitarget Information Fusion , 2007 .

[58]  Hai Yang,et al.  ACM Transactions on Intelligent Systems and Technology - Special Section on Urban Computing , 2014 .

[59]  Ramakant Nevatia,et al.  Multi-target tracking by on-line learned discriminative appearance models , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[60]  Ryosuke Shibasaki,et al.  A novel system for tracking pedestrians using multiple single-row laser-range scanners , 2005, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[61]  Dorin Comaniciu,et al.  Distribution Free Decomposition of Multivariate Data , 1998, Pattern Analysis & Applications.

[62]  Maja J. Mataric,et al.  A laser-based people tracker , 2002, Proceedings 2002 IEEE International Conference on Robotics and Automation (Cat. No.02CH37292).

[63]  Frank Dellaert,et al.  MCMC-based particle filtering for tracking a variable number of interacting targets , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[64]  Roberto Manduchi,et al.  Hybrid joint-separable multibody tracking , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[65]  Peter Willett,et al.  GMTI Tracking via the Gaussian Mixture Cardinalized Probability Hypothesis Density Filter , 2010, IEEE Transactions on Aerospace and Electronic Systems.

[66]  Xuan Song,et al.  Vision-Based Multiple Interacting Targets Tracking via On-Line Supervised Learning , 2008, ECCV.

[67]  Luc Van Gool,et al.  Coupled Object Detection and Tracking from Static Cameras and Moving Vehicles , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[68]  Gregory D. Hager,et al.  Dynamic Foreground/Background Extraction from Images and Videos using Random Patches , 2006, NIPS.

[69]  S. Godsill,et al.  Monte Carlo filtering for multi target tracking and data association , 2005, IEEE Transactions on Aerospace and Electronic Systems.

[70]  Rama Chellappa,et al.  Estimation of Object Motion Parameters from Noisy Images , 1986, IEEE Transactions on Pattern Analysis and Machine Intelligence.