Data Association Methods via Video Signal Processing in Imperfect Tracking Scenarios: A Review and Evaluation

In 5G scenarios, there are a large number of video signals that need to be processed. Multiobject tracking is one of the main directions in video signal processing. Data association is a very important link in tracking algorithms. Complexity and efficiency of association method have a direct impact on the performance of multiobject tracking. Breakthroughs have been made in data association methods based on deep learning, and the performance has been greatly improved compared with traditional methods. However, there is a lack of overviews about data association methods. Therefore, this article first analyzes characteristics and performance of three traditional data association methods and then focuses on data association methods based on deep learning, which is divided into different deep network structures: SOT methods, end-to-end methods, and Wasserstein metric methods. The performance of each tracking method is compared and analyzed. Finally, it summarizes the current common datasets and evaluation criteria for multiobject tracking and discusses challenges and development trends of data association technology and data association methods which ensure robust and real time need to be continuously improved.

[1]  Konrad Schindler,et al.  Continuous Energy Minimization for Multitarget Tracking , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[2]  I. Olkin,et al.  The distance between two random vectors with given dispersion matrices , 1982 .

[3]  R. Streit,et al.  JPDA intensity filter for tracking multiple extended objects in clutter , 2016, 2016 19th International Conference on Information Fusion (FUSION).

[4]  Krzysztof Czarnecki,et al.  FANTrack: 3D Multi-Object Tracking with Feature Association Network , 2019, 2019 IEEE Intelligent Vehicles Symposium (IV).

[5]  Maoguo Gong,et al.  Nearest-neighbour Joint Probabilistic Data Association Filter Based on Random Finite Set , 2019, 2019 International Conference on Control, Automation and Information Sciences (ICCAIS).

[6]  Siniša Šegvić,et al.  Pedestrian Tracking by Probabilistic Data Association and Correspondence Embeddings , 2019, 2019 22th International Conference on Information Fusion (FUSION).

[7]  Lingwei Xu,et al.  Physical Layer Security Performance of Mobile Vehicular Networks , 2020, Mob. Networks Appl..

[8]  Xianpeng Wang,et al.  Low-Complexity Channel Estimation for Circular and Noncircular Signals in Virtual MIMO Vehicle Communication Systems , 2020, IEEE Transactions on Vehicular Technology.

[9]  Qihui Wu,et al.  Extended Coprime Array Configuration Generating Large-Scale Antenna Co-Array in Massive MIMO System , 2019, IEEE Transactions on Vehicular Technology.

[10]  Ramakant Nevatia,et al.  An online learned CRF model for multi-target tracking , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[11]  Huadong Meng,et al.  A Multiple Hypothesis Tracking Method for Extended Target Tracking , 2010, 2010 International Conference on Electrical and Control Engineering.

[12]  Sing Kiong Nguang,et al.  Multi-Target Video Tracking Based on Improved Data Association and Mixed Kalman/ $H_{\infty }$ Filtering , 2016, IEEE Sensors Journal.

[13]  Gabriel Peyré,et al.  Wasserstein barycentric coordinates , 2016, ACM Trans. Graph..

[14]  Antonios Tsourdos,et al.  Joint Probabilistic Data Association Filter with Unknown Detection Probability and Clutter Rate , 2018, Sensors.

[15]  Minho Shin,et al.  Extended smoothing joint data association for multi‐target tracking in cluttered environments , 2020, IET Radar, Sonar & Navigation.

[16]  Xinggang Wang,et al.  A Simple Baseline for Multi-Object Tracking , 2020, ArXiv.

[17]  Roy Streit.  Analytic Combinatorics and Labeling in High Level Fusion and Multihypothesis Tracking , 2018, 2018 21st International Conference on Information Fusion (FUSION).

[18]  Xiaofei Zhang,et al.  Direction of Departure (DOD) and Direction of Arrival (DOA) Estimation in MIMO Radar with Reduced-Dimension MUSIC , 2010, IEEE Communications Letters.

[19]  Fan Yang,et al.  Deep Association: End-to-end Graph-Based Learning for Multiple Object Tracking with Conv-Graph Neural Network , 2019, ICMR.

[20]  Chengjian Sun,et al.  Multi-target tracking via hierarchical association learning , 2016, Neurocomputing.

[21]  Xavier Alameda-Pineda,et al.  DeepMOT: A Differentiable Framework for Training Multiple Object Trackers , 2019, ArXiv.

[22]  Xiaojuan Qi,et al.  3D Motion Decomposition for RGBD Future Dynamic Scene Synthesis , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[23]  Afshin Dehghan,et al.  GMCP-Tracker: Global Multi-object Tracking Using Generalized Minimum Clique Graphs , 2012, ECCV.

[24]  Hang-Bong Kang,et al.  Multiple Objects Fusion Tracker Using a Matching Network for Adaptively Represented Instance Pairs , 2017, Sensors.

[25]  Ahmet Cakiroglu.  Tracking variable number of targets with Joint Probabilistic Data Association Filter , 2016, 2016 24th Signal Processing and Communication Application Conference (SIU).

[26]  Lingwei Xu,et al.  Low-Complexity MIMO-FBMC Sparse Channel Parameter Estimation for Industrial Big Data Communications , 2020, IEEE Transactions on Industrial Informatics.

[27]  Haibin Ling,et al.  FAMNet: Joint Learning of Feature, Affinity and Multi-Dimensional Assignment for Online Multiple Object Tracking , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[28]  Ling Shao,et al.  Multiobject Tracking by Submodular Optimization , 2019, IEEE Transactions on Cybernetics.

[29]  Dietrich Paulus,et al.  Simple online and realtime tracking with a deep association metric , 2017, 2017 IEEE International Conference on Image Processing (ICIP).

[30]  Junpeng Shi,et al.  Fast direction finding for bistatic EMVS-MIMO radar without pairing , 2020, Signal Process..

[31]  Taek Lyul Song,et al.  Reduction of computational load for implementing iJIPDA filter , 2017, 2017 20th International Conference on Information Fusion (Fusion).

[32]  Julien Rabin,et al.  Convex Histogram-Based Joint Image Segmentation with Regularized Optimal Transport Cost , 2016, Journal of Mathematical Imaging and Vision.

[33]  Ke Wang,et al.  Angle estimation and mutual coupling self-calibration for ULA-based bistatic MIMO radar , 2018, Signal Process..

[34]  Yun Zhu,et al.  Covariance control joint integrated probabilistic data association filter for multi-target tracking , 2019 .

[35]  S.S. Blackman,et al.  Multiple hypothesis tracking for multiple target tracking , 2004, IEEE Aerospace and Electronic Systems Magazine.

[36]  Sumit Chakravarty,et al.  Pedestrian Tracker Utilizing Sensor Fusion and a Gaussian-Mixture Probability Hypothesis Density Filter , 2019, 2019 SoutheastCon.

[37]  Mubarak Shah,et al.  Deep Affinity Network for Multiple Object Tracking , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[38]  James M. Rehg,et al.  Multiple Hypothesis Tracking Revisited , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[39]  Lei Ma,et al.  Dynamic target tracking based on corner enhancement with Markov decision process , 2018 .

[40]  Konrad Schindler,et al.  Multi-Target Tracking by Discrete-Continuous Energy Minimization , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[41]  Rainer Stiefelhagen,et al.  Evaluating Multiple Object Tracking Performance: The CLEAR MOT Metrics , 2008, EURASIP J. Image Video Process..

[42]  Andreas Geiger,et al.  MOTS: Multi-Object Tracking and Segmentation , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[43]  Liu Liu,et al.  Addressing ambiguity in multi-target tracking by hierarchical strategy , 2017, 2017 IEEE International Conference on Image Processing (ICIP).

[44]  Mrinal Kumar,et al.  A Window-based Tensor Decomposition Approach to Data-Association for Multi Target Tracking , 2019, 2019 American Control Conference (ACC).

[45]  Faliang Chang,et al.  Multi-target tracking with hierarchical data association using main-parts and spatial-temporal feature models , 2018, Multimedia Tools and Applications.

[46]  Haifeng Li,et al.  Robust Multivehicle Tracking With Wasserstein Association Metric in Surveillance Videos , 2020, IEEE Access.

[47]  Laura Leal-Taix'e,et al.  Learning a Neural Solver for Multiple Object Tracking , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[48]  Konrad Schindler,et al.  Multi-target tracking by continuous energy minimization , 2011, CVPR 2011.

[49]  Baochang Zhang,et al.  Online Multi-Object Tracking Using Hierarchical Constraints for Complex Scenarios , 2018, IEEE Transactions on Intelligent Transportation Systems.

[50]  Fabio Tozeto Ramos,et al.  Simple online and realtime tracking , 2016, 2016 IEEE International Conference on Image Processing (ICIP).

[51]  Zhiping Zhou,et al.  Multi-target tracking by non-linear motion patterns based on hierarchical network flows , 2019, Multimedia Systems.

[52]  Ian D. Reid,et al.  Stable multi-target tracking in real-time surveillance video , 2011, CVPR 2011.

[53]  Antonios Tsourdos,et al.  Distributed Joint Probabilistic Data Association Filter With Hybrid Fusion Strategy , 2020, IEEE Transactions on Instrumentation and Measurement.

[54]  Kiyoung Moon,et al.  UA-DETRAC 2018: Report of AVSS2018 & IWT4S Challenge on Advanced Traffic Monitoring , 2018, 2018 15th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS).

[55]  Kazuma Sekiguchi,et al.  Visual Object Tracking by Moving Horizon Estimation with Probabilistic Data Association , 2020, 2020 IEEE/SICE International Symposium on System Integration (SII).

[56]  John Stein,et al.  An optimal tracking filter for processing sensor data of imprecisely determined origin in surveillance systems , 1971, CDC 1971.

[57]  Wei Wu,et al.  Multi-Object Tracking with Multiple Cues and Switcher-Aware Classification , 2019, ArXiv.

[58]  Xavier Alameda-Pineda,et al.  How to Train Your Deep Multi-Object Tracker , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[59]  Moe Z. Win,et al.  Data Association for Tracking Extended Targets , 2019, MILCOM 2019 - 2019 IEEE Military Communications Conference (MILCOM).

[60]  Lingwei Xu,et al.  BP neural network-based ABEP performance prediction for mobile Internet of Things communication systems , 2019, Neural Computing and Applications.

[61]  Ning Sun,et al.  Multi-Target Tracking Based on High-Order Appearance Feature Fusion , 2019, IEEE Access.

[62]  Facundo Mémoli,et al.  Gromov–Wasserstein Distances and the Metric Approach to Object Matching , 2011, Found. Comput. Math..

[63]  Lei Zhang,et al.  Off-Grid DOA Estimation for Colocated MIMO Radar via Reduced-Complexity Sparse Bayesian Learning , 2019, IEEE Access.

[64]  Karl Granström,et al.  Performance evaluation of multi-bernoulli conjugate priors for multi-target filtering , 2017, 2017 20th International Conference on Information Fusion (Fusion).

[65]  Ramakant Nevatia,et al.  Multi-target tracking by online learning of non-linear motion patterns and robust appearance models , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[66]  M. Emiliano.  Mathematical Problems in Engineering , 2014 .

[67]  Lingwei Xu,et al.  GR and BP neural network-based performance prediction of dual-antenna mobile communication networks , 2020, Comput. Networks.

[68]  Y. Bar-Shalom,et al.  The probabilistic data association filter , 2009, IEEE Control Systems.

[69]  Peter Willett,et al.  Set JPDA Filter for Multitarget Tracking , 2011, IEEE Transactions on Signal Processing.

[70]  Jun Wang,et al.  Efficient joint probabilistic data association filter based on Kullback–Leibler divergence for multi-target tracking , 2017 .

[71]  Antonios Tsourdos,et al.  Real-time Implementation of YOLO+JPDA for Small Scale UAV Multiple Object Tracking , 2018, 2018 International Conference on Unmanned Aircraft Systems (ICUAS).

[72]  Hua Yang,et al.  Online Multi-Object Tracking with Dual Matching Attention Networks , 2018, ECCV.