An online approach: Learning-Semantic-Scene-by-Tracking and Tracking-by-Learning-Semantic-Scene

Learning the knowledge of scene structure and tracking a large number of targets are both active topics of computer vision in recent years, which plays a crucial role in surveillance, activity analysis, object classification and etc. In this paper, we propose a novel system which simultaneously performs the Learning-Semantic-Scene and Tracking, and makes them supplement each other in one framework. The trajectories obtained by the tracking are utilized to continually learn and update the scene knowledge via an online un-supervised learning. On the other hand, the learned knowledge of scene in turn is utilized to supervise and improve the tracking results. Therefore, this “adaptive learning-tracking loop” can not only perform the robust tracking in high density crowd scene, dynamically update the knowledge of scene structure and output semantic words, but also ensures that the entire process is completely automatic and online. We successfully applied the proposed system into the JR subway station of Tokyo, which can dynamically obtain the semantic scene structure and robustly track more than 150 targets at the same time.

[1]  Ryosuke Shibasaki,et al.  Fusion of Detection and Matching Based Approaches for Laser Based Multiple People Tracking , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[2]  Tieniu Tan,et al.  Similarity based vehicle trajectory clustering and anomaly detection , 2005, IEEE International Conference on Image Processing 2005.

[3]  Margrit Betke,et al.  Tracking Large Variable Numbers of Objects in Clutter , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[4]  Gregory D. Hager,et al.  Probabilistic Data Association Methods for Tracking Complex Visual Objects , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[5]  Frank Dellaert,et al.  MCMC Data Association and Sparse Factorization Updating for Real Time Multitarget Tracking with Merged and Multiple Measurements , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[6]  Stefan Carlsson,et al.  Tracking and Labelling of Interacting Multiple Targets , 2006, ECCV.

[7]  Wolfram Burgard,et al.  People Tracking with Mobile Robots Using Sample-Based Joint Probabilistic Data Association Filters , 2003, Int. J. Robotics Res..

[8]  W. Eric L. Grimson,et al.  Trajectory analysis and semantic region modeling using a nonparametric Bayesian model , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[9]  Ryosuke Shibasaki,et al.  A novel system for tracking pedestrians using multiple single-row laser-range scanners , 2005, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[10]  S. Godsill,et al.  Monte Carlo filtering for multi target tracking and data association , 2005, IEEE Transactions on Aerospace and Electronic Systems.

[11]  Hassan Foroosh,et al.  Trajectory Rectification and Path Modeling for Video Surveillance , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[12]  Xuan Song,et al.  Tracking interacting targets with laser scanner via on-line supervised learning , 2008, 2008 IEEE International Conference on Robotics and Automation.

[13]  Mubarak Shah,et al.  Floor Fields for Tracking in High Density Crowd Scenes , 2008, ECCV.

[14]  Roberto Manduchi,et al.  Hybrid joint-separable multibody tracking , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[15]  Ming Yang,et al.  Game-Theoretic Multiple Target Tracking , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[16]  Ramakant Nevatia,et al.  Tracking multiple humans in complex situations , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[17]  Gregory D. Hager,et al.  Probabilistic data association methods in visual tracking of groups , 2004, CVPR 2004.

[18]  James J. Little,et al.  A Linear Programming Approach for Multiple Object Tracking , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[19]  Y. Bar-Shalom.  Tracking and data association , 1988 .

[20]  Stefan Carlsson,et al.  Multi-Target Tracking - Linking Identities using Bayesian Network Inference , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[21]  Joe W. Harris,et al.  Algebraic Geometry: A First Course , 1995 .

[22]  René Vidal,et al.  Online Clustering of Moving Hyperplanes , 2006, NIPS.

[23]  Dan Schonfeld,et al.  Real-time interactively distributed multi-object tracking using a magnetic-inertia potential model , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[24]  Tim J. Ellis,et al.  Automatic learning of an activity-based semantic scene model , 2003, Proceedings of the IEEE Conference on Advanced Video and Signal Based Surveillance, 2003..

[25]  Mubarak Shah,et al.  Multi feature path modeling for video surveillance , 2004, Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004..

[26]  W. Eric L. Grimson,et al.  Learning Semantic Scene Models by Trajectory Analysis , 2006, ECCV.

[27]  Gérard G. Medioni,et al.  Multiple Target Tracking Using Spatio-Temporal Markov Chain Monte Carlo Data Association , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[28]  Donald Reid.  An algorithm for tracking multiple targets , 1978 .

[29]  Luc Van Gool,et al.  Coupled Detection and Trajectory Estimation for Multi-Object Tracking , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[30]  Biswajit Bose,et al.  Multi-class object tracking algorithm that handles fragmentation and grouping , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[31]  Simon J. Godsill,et al.  On sequential Monte Carlo sampling methods for Bayesian filtering , 2000, Stat. Comput..