Long term people trackers for video monitoring systems. (Suivi long terme de personnes pour les systèmes de vidéo monitoring)

Multiple Object Tracking (MOT) is an important computer vision task and many MOT issues are still unsolved. Factors such as occlusions, illumination, object densities are big challenges for MOT. Therefore, this thesis proposes three MOT approaches to handle these challenges. The proposed approaches can be distinguished through two properties: their generality and their effectiveness.The first approach selects automatically the most reliable features to characterize each tracklet in a video scene. No training process is needed which makes this algorithm generic and deployable within a large variety of tracking frameworks. The second method tunes online tracking parameters for each tracklet according to the variation of the tracklet's surrounding context. There is no requirement on the number of tunable tracking parameters as well as their mutual dependence in the learning process. However, there is a need of training data which should be representative enough to make this algorithm generic. The third approach takes full advantage of features (hand-crafted and learned features) and tracklet affinity measurements proposed for the Re-id task and adapting them to MOT. Framework can work with or without training step depending on the tracklet affinity measurement.The experiments over three datasets, MOT2015, MOT2017 and ParkingLot show that the third approach is the most effective. The first and the third (without training) approaches are the most generic while the third approach (with training) necessitates the most supervision. Therefore, depending on the application as well as the availability of a training dataset, the most appropriate MOT algorithm could be selected.

[1]  Haroon Idrees,et al.  Detection and Tracking of Large Number of Targets in Wide Area Surveillance , 2010, ECCV.

[2]  Ioannis A. Kakadiaris,et al.  What Do I See? Modeling Human Visual Perception for Multi-person Tracking , 2014, ECCV.

[3]  Bodo Rosenhahn,et al.  Everybody needs somebody: Modeling social and grouping behavior on a linear programming multiple people tracker , 2011, 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops).

[4]  Jesús Martínez del Rincón,et al.  Enhancing Linear Programming with Motion Modeling for Multi-target Tracking , 2015, 2015 IEEE Winter Conference on Applications of Computer Vision.

[5]  Pascal Fua,et al.  Tracking multiple people under global appearance constraints , 2011, 2011 International Conference on Computer Vision.

[6]  Konrad Schindler,et al.  Multi-target tracking by continuous energy minimization , 2011, CVPR 2011.

[7]  Bastian Leibe,et al.  Multi-person Tracking with Sparse Detection and Continuous Segmentation , 2010, ECCV.

[8]  Volker Eiselein,et al.  High-Speed tracking-by-detection without using image information , 2017, 2017 14th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS).

[9]  Ko Nishino,et al.  Tracking with local spatio-temporal motion patterns in extremely crowded scenes , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[10]  Bodo Rosenhahn,et al.  Improvements to Frank-Wolfe optimization for multi-detector multi-object tracking , 2017, ArXiv.

[11]  Mohamed R. Amer,et al.  Multiobject tracking as maximum weight independent set , 2011, CVPR 2011.

[12]  Afshin Dehghan,et al.  GMCP-Tracker: Global Multi-object Tracking Using Generalized Minimum Clique Graphs , 2012, ECCV.

[13]  Luc Van Gool,et al.  Robust tracking-by-detection using a detector confidence particle filter , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[14]  Margrit Betke,et al.  Coupling detection and data association for multiple object tracking , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[15]  Gang Wang,et al.  Joint Learning of Convolutional Neural Networks and Temporally Constrained Metrics for Tracklet Association , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[16]  Thomas Brox,et al.  A Multi-cut Formulation for Joint Segmentation and Tracking of Multiple Objects , 2016, ArXiv.

[17]  Qiang Wang,et al.  Robust Object Tracking Based on Temporal and Spatial Deep Networks , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[18]  François Brémond,et al.  Multi-shot Person Re-Identification Using Part Appearance Mixture , 2017, 2017 IEEE Winter Conference on Applications of Computer Vision (WACV).

[19]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[20]  Duc Phu Chau,et al.  Online parameter tuning for object tracking algorithms , 2014, Image Vis. Comput..

[21]  Silvio Savarese,et al.  Tracking the Untrackable: Learning to Track Multiple Cues with Long-Term Dependencies , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[22]  Martin Lauer,et al.  3D Traffic Scene Understanding From Movable Platforms , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[23]  Ramakant Nevatia,et al.  Learning affinities and dependencies for multi-target tracking using a CRF model , 2011, CVPR 2011.

[24]  Takeo Kanade,et al.  Tracking in unstructured crowded scenes , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[25]  Alexandre Heili,et al.  Exploiting Long-Term Connectivity and Visual Motion in CRF-Based Multi-Person Tracking , 2014, IEEE Transactions on Image Processing.

[26]  Charless C. Fowlkes,et al.  Globally-optimal greedy algorithms for tracking a variable number of objects , 2011, CVPR 2011.

[27]  Afshin Dehghan,et al.  Part-based multiple-person tracking with partial occlusion handling , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[28]  Bi Song,et al.  A Stochastic Graph Evolution Framework for Robust Multi-target Tracking , 2010, ECCV.

[29]  Gang Wang,et al.  Tracklet Association with Online Target-Specific Metric Learning , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[30]  Lei Hu,et al.  Efficient person re-identification by hybrid spatiogram and covariance descriptor , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[31]  E. Land,et al.  Lightness and retinex theory. , 1971, Journal of the Optical Society of America.

[32]  Horst Bischof,et al.  Large scale metric learning from equivalence constraints , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[33]  Patrick Pérez,et al.  Color-Based Probabilistic Tracking , 2002, ECCV.

[34]  Xiaoqin Zhang,et al.  Single and Multiple Object Tracking Using Log-Euclidean Riemannian Subspace and Block-Division Appearance Model , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[35]  Ming Yang,et al.  Detection driven adaptive multi-cue integration for multiple human tracking , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[36]  Luc Van Gool,et al.  Coupled Object Detection and Tracking from Static Cameras and Moving Vehicles , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[37]  Fatih Murat Porikli,et al.  Region Covariance: A Fast Descriptor for Detection and Classification , 2006, ECCV.

[38]  Jonathon A. Chambers,et al.  GM-PHD Filter Based Online Multiple Human Tracking Using Deep Discriminative Correlation Matching , 2018, 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[39]  Radu Horaud,et al.  Tracking Multiple Persons Based on a Variational Bayesian Model , 2016, ECCV Workshops.

[40]  S. Savarese,et al.  Learning an Image-Based Motion Context for Multiple People Tracking , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[41]  Romaric Audigier,et al.  Improving Multi-frame Data Association with Sparse Representations for Robust Near-online Multi-object Tracking , 2016, ECCV.

[42]  Xiaogang Wang,et al.  Visual Tracking with Fully Convolutional Networks , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[43]  Slawomir Bak,et al.  Multiple-shot human re-identification by Mean Riemannian Covariance Grid , 2011, 2011 8th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS).

[44]  Junjie Yan,et al.  Multiple Target Tracking Based on Undirected Hierarchical Relation Hypergraph , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[45]  Roberto Cipolla,et al.  Unsupervised Bayesian Detection of Independent Motion in Crowds , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[46]  Bernt Schiele,et al.  Learning People Detectors for Tracking in Crowded Scenes , 2013, 2013 IEEE International Conference on Computer Vision.

[47]  Loic Fagot-Bouquet,et al.  Online multi-person tracking based on global sparse collaborative representations , 2015, 2015 IEEE International Conference on Image Processing (ICIP).

[48]  Ian D. Reid,et al.  Joint tracking and segmentation of multiple targets , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[49]  Mubarak Shah,et al.  Floor Fields for Tracking in High Density Crowd Scenes , 2008, ECCV.

[50]  Ming-Hsuan Yang,et al.  Bayesian Multi-object Tracking Using Motion Context from Multiple Objects , 2015, 2015 IEEE Winter Conference on Applications of Computer Vision.

[51]  Bogdan Kwolek,et al.  Region Covariance Matrix-Based Object Tracking with Occlusions Handling , 2010, ICCVG.

[52]  Kuk-Jin Yoon,et al.  Visual Tracking via Adaptive Tracker Selection with Multiple Features , 2012, ECCV.

[53]  Silvio Savarese,et al.  A Unified Framework for Multi-target Tracking and Collective Activity Recognition , 2012, ECCV.

[54]  Ramakant Nevatia,et al.  Multi-target tracking by on-line learned discriminative appearance models , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[55]  Andrea Cavallaro,et al.  Multi-target tracking on confidence maps: An application to people tracking , 2013, Comput. Vis. Image Underst..

[56]  Robert T. Collins,et al.  Multi-target Tracking by Lagrangian Relaxation to Min-cost Network Flow , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[57]  Luis E. Ortiz,et al.  Who are you with and where are you going? , 2011, CVPR 2011.

[58]  Dariu Gavrila,et al.  Multi-cue Pedestrian Detection and Tracking from a Moving Vehicle , 2007, International Journal of Computer Vision.

[59]  W. Förstner,et al.  A Metric for Covariance Matrices , 2003 .

[60]  Matti Pietikäinen,et al.  Modeling pixel process with scale invariant local patterns for background subtraction in complex scenes , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[61]  Volker Eiselein,et al.  Sequential sensor fusion combining probability hypothesis density and kernelized correlation filters for multi-object tracking in video data , 2017, 2017 14th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS).

[62]  Larry S. Davis,et al.  Background modeling and subtraction by codebook construction , 2004, 2004 International Conference on Image Processing, 2004. ICIP '04..

[63]  Francois Bremond,et al.  Multi-Object tracking using multi-channel part appearance representation , 2017, 2017 14th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS).

[64]  Hui Li,et al.  Automatic Tracking of a Large Number of Moving Targets in 3D , 2012, ECCV.

[65]  Fabio Poiesi,et al.  Online Multi-target Tracking with Strong and Weak Detections , 2016, ECCV Workshops.

[66]  Afshin Dehghan,et al.  GMMCP tracker: Globally optimal Generalized Maximum Multi Clique problem for multiple object tracking , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[67]  Kilian Q. Weinberger,et al.  Distance Metric Learning for Large Margin Nearest Neighbor Classification , 2005, NIPS.

[68]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[69]  Wei-Han Chang,et al.  A fast MPEG-7 dominant color extraction with new similarity measure for image retrieval , 2008, J. Vis. Commun. Image Represent..

[70]  Jana Trojanová,et al.  Multi-object tracking of pedestrian driven by context , 2016, 2016 13th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS).

[71]  Ian D. Reid,et al.  Stable multi-target tracking in real-time surveillance video , 2011, CVPR 2011.

[72]  Konrad Schindler,et al.  Learning by Tracking: Siamese CNN for Robust Target Association , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[73]  Wenhan Luo,et al.  Multiple object tracking: A literature review , 2014, Artif. Intell..

[74]  Ming-Hsuan Yang,et al.  Online Multi-object Tracking via Structural Constraint Event Aggregation , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[75]  Ramakant Nevatia,et al.  An online learned CRF model for multi-target tracking , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[76]  Konrad Schindler,et al.  Detection- and Trajectory-Level Exclusion in Multiple Object Tracking , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[77]  Michael J. Swain,et al.  Color indexing , 1991, International Journal of Computer Vision.

[78]  Yu Liu,et al.  POI: Multiple Object Tracking with High Performance Detection and Appearance Feature , 2016, ECCV Workshops.

[79]  Rui Caseiro,et al.  Globally optimal solution to multi-object tracking with merged measurements , 2011, 2011 International Conference on Computer Vision.

[80]  Duc Phu Chau,et al.  A multi-feature tracking algorithm enabling adaptation to context variations , 2011, ICDP.

[81]  Slawomir Bak,et al.  Recovering People Tracking Errors Using Enhanced Covariance-Based Signatures , 2012, 2012 IEEE Ninth International Conference on Advanced Video and Signal-Based Surveillance.

[82]  Christophe De Vleeschouwer,et al.  Discriminative Label Propagation for Multi-object Tracking with Sporadic Appearance Features , 2013, 2013 IEEE International Conference on Computer Vision.

[83]  Ian D. Reid,et al.  Robust Real-Time Visual Tracking Using Pixel-Wise Posteriors , 2008, ECCV.

[84]  Ramakant Nevatia,et al.  Global data association for multi-object tracking using network flows , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[85]  François Brémond,et al.  Global tracker: An online evaluation framework to improve tracking quality , 2014, 2014 11th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS).

[86]  Mohand Saïd Djouadi,et al.  Suspicious motion patterns detection and tracking in crowded scenes , 2013, 2013 IEEE International Symposium on Safety, Security, and Rescue Robotics (SSRR).

[87]  Bastian Leibe,et al.  Real-time multi-person tracking with detector assisted structure propagation , 2011, 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops).

[88]  Luc Van Gool,et al.  A mobile vision system for robust multi-person tracking , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[89]  Konrad Schindler,et al.  Continuous Energy Minimization for Multitarget Tracking , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[90]  Volker Eiselein,et al.  Real-Time Multi-human Tracking Using a Probability Hypothesis Density Filter and Multiple Detectors , 2012, 2012 IEEE Ninth International Conference on Advanced Video and Signal-Based Surveillance.

[91]  Francois Bremond,et al.  Online tracking parameter adaptation based on evaluation , 2013, 2013 10th IEEE International Conference on Advanced Video and Signal Based Surveillance.

[92]  Patrick Valduriez,et al.  Pre-processing and Indexing Techniques for Constellation Queries in Big Data , 2017, DaWaK.

[93]  Yang Zhang,et al.  Enhancing Detection Model for Multiple Hypothesis Tracking , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[94]  Ramakant Nevatia,et al.  Learning to associate: HybridBoosted multi-target tracker for crowded scene , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[95]  Trevor Darrell,et al.  The pyramid match kernel: discriminative classification with sets of image features , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[96]  Frédéric Jurie,et al.  Motion Models that Only Work Sometimes , 2012, BMVC.

[97]  Hilke Kieritz,et al.  Online multi-person tracking using Integral Channel Features , 2016, 2016 13th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS).

[98]  Carlo Tomasi,et al.  Good features to track , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[99]  Duc Phu Chau,et al.  Robust global tracker based on an online estimation of tracklet descriptor reliability , 2015, 2015 12th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS).

[100]  Kuk-Jin Yoon,et al.  Robust Online Multi-object Tracking Based on Tracklet Confidence and Online Discriminative Appearance Learning , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[101]  Shin Ishii,et al.  Expanding histogram of colors with gridding to improve tracking accuracy , 2015, 2015 14th IAPR International Conference on Machine Vision Applications (MVA).

[102]  A.t NGHIEM,et al.  A New Evaluation Approach for Video Processing Algorithms , 2007, 2007 IEEE Workshop on Motion and Video Computing (WMVC'07).

[103]  Ramakant Nevatia,et al.  Online Learned Discriminative Part-Based Appearance Models for Multi-human Tracking , 2012, ECCV.

[104]  James M. Rehg,et al.  Multiple Hypothesis Tracking Revisited , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[105]  Konrad Schindler,et al.  Online Multi-Target Tracking Using Recurrent Neural Networks , 2016, AAAI.

[106]  Takahiro Okabe,et al.  Using individuality to track individuals: Clustering individual trajectories in crowds using local appearance and frequency trait , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[107]  Shihong Lao,et al.  Multi-object tracking through occlusions by local tracklets filtering and global tracklets association with detection responses , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[108]  Zhen Qin,et al.  Improving multi-target tracking via social grouping , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[109]  Francesco Solera,et al.  Towards the evaluation of reproducible robustness in tracking-by-detection , 2015, 2015 12th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS).

[110]  Takahiro Okabe,et al.  Hierarchical Gaussian Descriptor for Person Re-identification , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[111]  Bonhwa Ku,et al.  Online multi-object tracking with efficient track drift and fragmentation handling. , 2017, Journal of the Optical Society of America. A, Optics, image science, and vision.

[112]  Gang Wang,et al.  Learning deep features for multiple object tracking by using a multi-task learning strategy , 2014, 2014 IEEE International Conference on Image Processing (ICIP).

[113]  François Brémond,et al.  ETISEO, performance evaluation for video surveillance systems , 2007, 2007 IEEE Conference on Advanced Video and Signal Based Surveillance.

[114]  Ivan Laptev,et al.  Data-driven crowd analysis in videos , 2011, ICCV.

[115]  Cordelia Schmid,et al.  Beyond Bags of Features: Spatial Pyramid Matching for Recognizing Natural Scene Categories , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).