Online Multi-Object Tracking With GMPHD Filter and Occlusion Group Management

In this paper, we propose an efficient online multi-object tracking method based on the Gaussian mixture probability hypothesis density (GMPHD) filter and occlusion group management scheme where a hierarchical data association is utilized for the GMPHD filter to reduce the false negatives caused by missed detection. The hierarchical data association consisting of two modules, detection-to-track and track-to-track associations, can recover the lost tracks and their switched IDs. In addition, the proposed grouping management scheme handles occlusion problems with two main parts. The first part, “track merging” can merge the false positive tracks caused by false positive detections from occlusions. The occlusion of the false positive tracks is usually measured with some metric. In this research, we define the occlusion measure between visual objects, as sum-of-intersection-over-each-area (SIOA) instead of the commonly used intersection-over-union (IOU). The second part, “occlusion group energy minimization (OGEM)” prevents the occluded true positive tracks from false “track merging”. Each group of the occluded objects is expressed with an energy function and an optimal hypothesis will be obtained by minimizing the energy. We evaluate the proposed tracker in benchmarks such as MOT15 and MOT17 which are public datasets for multi-person tracking. An ablation study in training dataset reveals not only that “track merging” and “OGEM” complement each other, but also that the proposed tracking method shows more robust performance and less sensitiveness than baseline methods. Also, the tracking performance with SIOA is better than that with IOU for various sizes of false positives. Experimental results show that the proposed tracker efficiently handles occlusion situations and achieves competitive performance compared to the state-of-the-art methods. In fact, our method shows the best multi-object tracking accuracy among the online and real-time executable methods.

[1]  Volker Eiselein,et al.  Sequential sensor fusion combining probability hypothesis density and kernelized correlation filters for multi-object tracking in video data , 2017, 2017 14th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS).

[2]  S. Savarese,et al.  Learning an Image-Based Motion Context for Multiple People Tracking , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[3]  Konrad Schindler,et al.  Continuous Energy Minimization for Multitarget Tracking , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[4]  Konrad Schindler,et al.  Learning by Tracking: Siamese CNN for Robust Target Association , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[5]  Samuel Murray,et al.  Real-Time Multiple Object Tracking - A Study on the Importance of Speed , 2017, ArXiv.

[6]  A. Volgenant,et al.  A shortest augmenting path algorithm for dense and sparse linear assignment problems , 1987, Computing.

[7]  David A. McAllester,et al.  Object Detection with Discriminatively Trained Part Based Models , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[8]  Ming-Hsuan Yang,et al.  Online Multi-object Tracking via Structural Constraint Event Aggregation , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[9]  Volker Eiselein,et al.  Real-Time Multi-human Tracking Using a Probability Hypothesis Density Filter and Multiple Detectors , 2012, 2012 IEEE Ninth International Conference on Advanced Video and Signal-Based Surveillance.

[10]  Zeyu Fu,et al.  Particle PHD Filter Based Multiple Human Tracking Using Online Group-Structured Dictionary Learning , 2018, IEEE Access.

[11]  T. Başar,et al.  A New Approach to Linear Filtering and Prediction Problems , 2001 .

[12]  Bodo Rosenhahn,et al.  Fusion of Head and Full-Body Detectors for Multi-object Tracking , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[13]  Young-min Song,et al.  Online multiple object tracking with the hierarchically adopted GM-PHD filter using motion and appearance , 2016, 2016 IEEE International Conference on Consumer Electronics-Asia (ICCE-Asia).

[14]  Stefan Roth,et al.  MOT16: A Benchmark for Multi-Object Tracking , 2016, ArXiv.

[15]  Kwangjin Yoon,et al.  Online and Real-Time Tracking with the GM-PHD Filter using Group Management and Relative Motion Analysis , 2018, 2018 15th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS).

[16]  Ba-Ngu Vo,et al.  The Gaussian Mixture Probability Hypothesis Density Filter , 2006, IEEE Transactions on Signal Processing.

[17]  Charless C. Fowlkes,et al.  Globally-optimal greedy algorithms for tracking a variable number of objects , 2011, CVPR 2011.

[18]  Seung-Hwan Bae,et al.  Confidence-Based Data Association and Discriminative Deep Appearance Learning for Robust Online Multi-Object Tracking , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[19]  Ba Tuong Vo,et al.  Random finite sets in Multi-object filtering , 2008 .

[20]  Sumeetpal S. Singh,et al.  Sequential monte carlo implementation of the phd filter for multi-target tracking , 2003, Sixth International Conference of Information Fusion, 2003. Proceedings of the.

[21]  Ian D. Reid,et al.  Joint Probabilistic Data Association Revisited , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[22]  Rubén Heras Evangelio,et al.  Counting People in Crowded Environments by Fusion of Shape and Motion Information , 2010, 2010 7th IEEE International Conference on Advanced Video and Signal Based Surveillance.

[23]  R. Mahler Multitarget Bayes filtering via first-order multitarget moments , 2003 .

[24]  Fan Yang,et al.  Exploit All the Layers: Fast and Accurate CNN Object Detector with Scale Dependent Pooling and Cascaded Rejection Classifiers , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[25]  Gregory Dudek,et al.  Where is your dive buddy: tracking humans underwater using spatio-temporal features , 2007, 2007 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[26]  Stefan Roth,et al.  MOTChallenge 2015: Towards a Benchmark for Multi-Target Tracking , 2015, ArXiv.

[27]  Jingdong Wang,et al.  Deeply-Learned Part-Aligned Representations for Person Re-identification , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[28]  Fuqiang Liu,et al.  Adaptive Object Detection and Visibility Improvement in Foggy Image , 2011, J. Multim..

[29]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[30]  Yang Zhang,et al.  Iterative Multiple Hypothesis Tracking With Tracklet-Level Association , 2019, IEEE Transactions on Circuits and Systems for Video Technology.

[31]  Andrew M. Wallace,et al.  Development of a N-type GM-PHD Filter for Multiple Target, Multiple Type Visual Tracking , 2019, J. Vis. Commun. Image Represent..

[32]  Jonathon A. Chambers,et al.  Multi-Level Cooperative Fusion of GM-PHD Filters for Online Multiple Human Tracking , 2019, IEEE Transactions on Multimedia.

[33]  Ali Farhadi,et al.  You Only Look Once: Unified, Real-Time Object Detection , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[34]  Hong Wang,et al.  Evolving boxes for fast vehicle detection , 2017, 2017 IEEE International Conference on Multimedia and Expo (ICME).

[35]  Yang Zhang,et al.  Heterogeneous Association Graph Fusion for Target Association in Multiple Object Tracking , 2019, IEEE Transactions on Circuits and Systems for Video Technology.

[36]  Luc Van Gool,et al.  The Pascal Visual Object Classes Challenge: A Retrospective , 2014, International Journal of Computer Vision.

[37]  Yang Zhang,et al.  Enhancing Detection Model for Multiple Hypothesis Tracking , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[38]  Ba-Ngu Vo,et al.  A labeled random finite set online multi-object tracker for video data , 2019, Pattern Recognit..

[39]  Loic Fagot-Bouquet,et al.  Online multi-person tracking based on global sparse collaborative representations , 2015, 2015 IEEE International Conference on Image Processing (ICIP).

[40]  Ian D. Reid,et al.  Joint tracking and segmentation of multiple targets , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[41]  Andreas Geiger,et al.  Are we ready for autonomous driving? The KITTI vision benchmark suite , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[42]  Volker Eiselein,et al.  High-Speed tracking-by-detection without using image information , 2017, 2017 14th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS).

[43]  Rainer Stiefelhagen,et al.  Evaluating Multiple Object Tracking Performance: The CLEAR MOT Metrics , 2008, EURASIP J. Image Video Process..

[44]  Junaed Sattar,et al.  Visual Diver Recognition for Underwater Human-Robot Collaboration , 2018, 2019 International Conference on Robotics and Automation (ICRA).

[45]  Ronald L Fante Detection of Targets in Noise and Clutter , 1977 .

[46]  Ba-Ngu Vo,et al.  Analytic Implementations of the Cardinalized Probability Hypothesis Density Filter , 2007, IEEE Transactions on Signal Processing.

[47]  Kwangjin Yoon,et al.  Online Multi-Object Tracking with Historical Appearance Matching and Scene Adaptive Detection Filtering , 2018, 2018 15th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS).

[48]  Fabio Poiesi,et al.  Online Multi-target Tracking with Strong and Weak Detections , 2016, ECCV Workshops.

[49]  Ming-Hsuan Yang,et al.  Bayesian Multi-object Tracking Using Motion Context from Multiple Objects , 2015, 2015 IEEE Winter Conference on Applications of Computer Vision.

[50]  Michael S. Bernstein,et al.  ImageNet Large Scale Visual Recognition Challenge , 2014, International Journal of Computer Vision.

[51]  Rubén Heras Evangelio,et al.  Detection of static objects for the task of video surveillance , 2011, 2011 IEEE Workshop on Applications of Computer Vision (WACV).

[52]  Rui Caseiro,et al.  High-Speed Tracking with Kernelized Correlation Filters , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[53]  James M. Rehg,et al.  Multiple Hypothesis Tracking Revisited , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[54]  Silvio Savarese,et al.  Learning to Track: Online Multi-object Tracking by Decision Making , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[55]  Hang Dong,et al.  Online Multi-Object Tracking with Structural Invariance Constraint , 2018, BMVC.

[56]  Hua Yang,et al.  Online Multi-Object Tracking with Dual Matching Attention Networks , 2018, ECCV.

[57]  Seung-Hwan Bae,et al.  Learning Discriminative Appearance Models for Online Multi-Object Tracking With Appearance Discriminability Measures , 2018, IEEE Access.

[58]  Andrea Cavallaro,et al.  A Predictor of Moving Objects for First-Person Vision , 2019, 2019 IEEE International Conference on Image Processing (ICIP).

[59]  Jesús Martínez del Rincón,et al.  Enhancing Linear Programming with Motion Modeling for Multi-target Tracking , 2015, 2015 IEEE Winter Conference on Applications of Computer Vision.

[60]  Thomas Brox,et al.  Motion Segmentation & Multiple Object Tracking by Correlation Co-Clustering , 2020, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[61]  Gang Wang,et al.  Joint Learning of Convolutional Neural Networks and Temporally Constrained Metrics for Tracklet Association , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[62]  Pietro Perona,et al.  Fast Feature Pyramids for Object Detection , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[63]  Long Chen,et al.  Real-Time Multiple People Tracking with Deeply Learned Candidate Selection and Person Re-Identification , 2018, 2018 IEEE International Conference on Multimedia and Expo (ICME).

[64]  Ram Nevatia,et al.  Learning to associate: HybridBoosted multi-target tracker for crowded scene , 2009, CVPR.

[65]  Ba-Ngu Vo,et al.  Labeled Random Finite Sets and Multi-Object Conjugate Priors , 2013, IEEE Transactions on Signal Processing.

[66]  Mario Sznaier,et al.  The Way They Move: Tracking Multiple Targets with Similar Appearance , 2013, 2013 IEEE International Conference on Computer Vision.

[67]  James M. Rehg,et al.  Multi-object Tracking with Neural Gating Using Bilinear LSTM , 2018, ECCV.

[68]  Yann LeCun,et al.  Learning a similarity metric discriminatively, with application to face verification , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[69]  Kuk-Jin Yoon,et al.  Robust Online Multi-object Tracking Based on Tracklet Confidence and Online Discriminative Appearance Learning , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[70]  Wongun Choi,et al.  Near-Online Multi-target Tracking with Aggregated Local Flow Descriptor , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[71]  Bernt Schiele,et al.  DeeperCut: A Deeper, Stronger, and Faster Multi-person Pose Estimation Model , 2016, ECCV.