Tracking with a mixed continuous-discrete Conditional Random Field

Tracking algorithms are an indispensable prerequisite for many higher-level computer vision tasks, ranging from surveillance to animation to automotive applications. A complete tracker is a complex system with many modules that need to cooperate. It is important to exploit all the sources of information, such as the appearance, the physical constraints and, though less commonly used, social factors like the walking patterns of people that belong to the same group. Given this complexity, a tracker often resorts to ad hoc solutions and scene specific customizations to improve the performance. We propose here a multi-target tracking model that succeeds in uniformly including the mentioned sources of information and is amenable to further extensions. We build our model within the Conditional Random Field framework. As the model cannot be globally optimized, we adopt an approximate inference strategy. Therefore we use a recently published sampling-based inference method that we customize to our needs and show the effectiveness of the choice in the experimental results.

[1]  E. Hall,et al.  The Hidden Dimension , 1970 .

[2]  Yaakov Bar-Shalom,et al.  Sonar tracking of multiple targets using joint probabilistic data association , 1983 .

[3]  Jake K. Aggarwal,et al.  Stochastic Representation and Recognition of High-Level Group Activities , 2011, International Journal of Computer Vision.

[4]  Andreas Krause,et al.  Unfreezing the robot: Navigation in dense, interacting crowds , 2010, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[5]  Silvio Savarese,et al.  Multiple Target Tracking in World Coordinate with Single, Minimally Calibrated Camera , 2010, ECCV.

[6]  Luc Van Gool,et al.  Improving Data Association by Joint Modeling of Pedestrian Trajectories and Groupings , 2010, ECCV.

[7]  Christopher M. Bishop,et al.  Pattern Recognition and Machine Learning (Information Science and Statistics) , 2006 .

[8]  Helbing,et al.  Social force model for pedestrian dynamics. , 1995, Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics.

[9]  Ming-Ching Chang,et al.  Probabilistic group-level motion analysis and scenario recognition , 2011, 2011 International Conference on Computer Vision.

[10]  Frank Dellaert,et al.  MCMC-based particle filtering for tracking a variable number of interacting targets , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  Kai Oliver Arras,et al.  People tracking with human motion predictions from social forces , 2010, 2010 IEEE International Conference on Robotics and Automation.

[12]  Luc Van Gool,et al.  You'll never walk alone: Modeling social behavior for multi-target tracking , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[13]  A. Schadschneider.  Cellular Automaton Approach to Pedestrian Dynamics - Theory , 2001, cond-mat/0112117.

[14]  Ram Nevatia,et al.  Learning to associate: HybridBoosted multi-target tracker for crowded scene , 2009, CVPR.

[15]  Juergen Gall,et al.  Class-specific Hough forests for object detection , 2009, CVPR.

[16]  Makoto Miyata,et al.  Gliding ghosts of Mycoplasma mobile. , 2005, Proceedings of the National Academy of Sciences of the United States of America.

[17]  Dinesh Manocha,et al.  Reciprocal Velocity Obstacles for real-time multi-agent navigation , 2008, 2008 IEEE International Conference on Robotics and Automation.

[18]  Robert T. Collins,et al.  Automatically detecting the small group structure of a crowd , 2009, 2009 Workshop on Applications of Computer Vision (WACV).

[19]  Horst Bischof,et al.  Online multi-class LPBoost , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[20]  Michel Bierlaire,et al.  Behavioral Priors for Detection and Tracking of Pedestrians in Video Sequences , 2006, International Journal of Computer Vision.

[21]  Rainer Stiefelhagen,et al.  Evaluating Multiple Object Tracking Performance: The CLEAR MOT Metrics , 2008, EURASIP J. Image Video Process..

[22]  Jonathan L. Freedman.  Crowding and behavior , 1975 .

[23]  Luc Van Gool,et al.  Coupled Object Detection and Tracking from Static Cameras and Moving Vehicles , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[24]  Stéphane Donikian,et al.  A synthetic-vision based steering approach for crowd simulation , 2010, SIGGRAPH 2010.

[25]  Wolfram Burgard,et al.  Multi-model Hypothesis Group Tracking and Group Size Estimation , 2010, Int. J. Soc. Robotics.

[26]  Dinesh Manocha,et al.  Modeling collision avoidance behavior for virtual humans , 2010, AAMAS.

[27]  Takeo Kanade,et al.  Tracking in unstructured crowded scenes , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[28]  Ian D. Reid,et al.  Stable multi-target tracking in real-time surveillance video , 2011, CVPR 2011.

[29]  Bi Song,et al.  A Stochastic Graph Evolution Framework for Robust Multi-target Tracking , 2010, ECCV.

[30]  Mohamed R. Amer,et al.  Multiobject tracking as maximum weight independent set , 2011, CVPR 2011.

[31]  Ingemar J. Cox,et al.  A review of statistical data association techniques for motion correspondence , 1993, International Journal of Computer Vision.

[32]  Luc Van Gool,et al.  Online Multiperson Tracking-by-Detection from a Single, Uncalibrated Camera , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[33]  Ramin Mehran,et al.  Abnormal crowd behavior detection using social force model , 2009, CVPR.

[34]  Horst Bischof,et al.  PROST: Parallel robust online simple tracking , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[35]  Mubarak Shah,et al.  Floor Fields for Tracking in High Density Crowd Scenes , 2008, ECCV.

[36]  Philippe C. Cattin,et al.  Tracking the invisible: Learning where the object might be , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[37]  Ramakant Nevatia,et al.  Detection and Tracking of Multiple, Partially Occluded Humans by Bayesian Combination of Edgelet based Part Detectors , 2007, International Journal of Computer Vision.

[38]  Michael J. Black,et al.  The outlier process: unifying line processes and robust statistics , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[39]  Ramakant Nevatia,et al.  Learning affinities and dependencies for multi-target tracking using a CRF model , 2011, CVPR 2011.

[40]  Andrew McCallum,et al.  Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data , 2001, ICML.

[41]  Luc Van Gool,et al.  A mobile vision system for robust multi-person tracking , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[42]  Clark McPhail,et al.  Using Film to Analyze Pedestrian Behavior , 1982 .

[43]  Donald Reid.  An algorithm for tracking multiple targets , 1978 .

[44]  Michael I. Jordan,et al.  Loopy Belief Propagation for Approximate Inference: An Empirical Study , 1999, UAI.

[45]  William T. Freeman,et al.  Nonparametric belief propagation , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[46]  Ramakant Nevatia,et al.  Global data association for multi-object tracking using network flows , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[47]  A. Doucet,et al.  Sequential auxiliary particle belief propagation , 2005, 2005 7th International Conference on Information Fusion.

[48]  Charless C. Fowlkes,et al.  Globally-optimal greedy algorithms for tracking a variable number of objects , 2011, CVPR 2011.

[49]  Alessio Del Bue,et al.  Social interaction discovery by statistical analysis of F-formations , 2011, BMVC.

[50]  Geoffrey E. Hinton.  Training Products of Experts by Minimizing Contrastive Divergence , 2002, Neural Computation.

[51]  Bodo Rosenhahn,et al.  Everybody needs somebody: Modeling social and grouping behavior on a linear programming multiple people tracker , 2011, 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops).

[52]  Nasser M. Nasrabadi,et al.  Pattern Recognition and Machine Learning , 2006, Technometrics.

[53]  Luis E. Ortiz,et al.  Who are you with and where are you going? , 2011, CVPR 2011.

[54]  Marshall F. Tappen,et al.  Learning pedestrian dynamics from the real world , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[55]  Andrew Peter French,et al.  Visual Tracking: From An Individual To Groups Of Animals , 2005 .

[56]  Junseok Kwon,et al.  Tracking by Sampling Trackers , 2011, 2011 International Conference on Computer Vision.

[57]  Gregory D. Hager,et al.  Probabilistic data association methods in visual tracking of groups , 2004, CVPR 2004.

[58]  Dani Lischinski,et al.  Crowds by Example , 2007, Comput. Graph. Forum.

[59]  Martial Hebert,et al.  Learning to Find Object Boundaries Using Motion Cues , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[60]  James J. Little,et al.  A Boosted Particle Filter: Multitarget Detection and Tracking , 2004, ECCV.

[61]  Stéphane Donikian,et al.  Experiment-based modeling, simulation and validation of interactions between virtual walkers , 2009, SCA '09.