Reinforcement Learning-Based Data Association for Multiple Target Tracking in Clutter

Data association is a crucial component of multiple target tracking, in which each measurement obtained by the sensor can be determined whether it belongs to the target. However, many methods reported in the literature may not be able to ensure the accuracy and low computational complexity during the association process, especially in the presence of dense clutters. In this paper, a novel data association method based on reinforcement learning (RL), i.e., the so-called RL-JPDA method, has been proposed for solving the aforementioned problem. In the presented method, the RL is leveraged to acquire available information of measurements. In addition, the motion characteristics of the targets are utilized to ensure the accuracy of the association results. Experiments are performed to compare the proposed method with the global nearest neighbor data association method, the joint probabilistic data association method, the fuzzy optimal membership data association method and the intuitionistic fuzzy joint probabilistic data association method. The results show that the proposed method yields a shorter execution time compared to other methods. Furthermore, it can obtain an effective and feasible estimation in the environment with dense clutters.

[1]  Yaakov Bar-Shalom,et al.  Adaptive nonlinear filtering for tracking with measurements of uncertain origin , 1972, CDC 1972.

[2]  Robert Babuska,et al.  A Survey of Actor-Critic Reinforcement Learning: Standard and Natural Policy Gradients , 2012, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[3]  R. Singer,et al.  New results in optimizing surveillance system tracking and data correlation performance in dense multitarget environments , 1973 .

[4]  Kuo-Chu Chang,et al.  Joint probabilistic data association for multitarget tracking with possibly unresolved measurements and maneuvers , 1984 .

[5]  Antonios Tsourdos,et al.  Multi-Sensor Multi-Target Tracking Using Domain Knowledge and Clustering , 2018, IEEE Sensors Journal.

[6]  Thia Kirubarajan,et al.  Track Quality Based Multitarget Tracking Approach for Global Nearest-Neighbor Association , 2012, IEEE Transactions on Aerospace and Electronic Systems.

[7]  Gnane Swarnadh Satapathi,et al.  Soft and evolutionary computation based data association approaches for tracking multiple targets in the presence of ECM , 2017, Expert Syst. Appl..

[8]  Xiaohong Su,et al.  Deep Reinforcement Learning for Data Association in Cell Tracking , 2020, Frontiers in Bioengineering and Biotechnology.

[9]  Li We Improved Integrated Probabilistic Data Association Algorithm Based on Amplitude Information , 2015 .

[10]  Yuming Bo,et al.  Multi-target tracking method based on improved firefly algorithm optimized particle filter , 2019, Neurocomputing.

[11]  Song Wang,et al.  UGV Navigation Optimization Aided by Reinforcement Learning-Based Path Tracking , 2018, IEEE Access.

[12]  Ashraf M. Aziz,et al.  A new nearest-neighbor association approach based on fuzzy clustering , 2013 .

[13]  Sen Wang,et al.  Adaptive low-level control of autonomous underwater vehicles using deep reinforcement learning , 2018, Robotics Auton. Syst..

[14]  Taek Lyul Song,et al.  Modified smoothing data association for target tracking in clutter , 2015, Expert Syst. Appl..

[15]  Thiagalingam Kirubarajan,et al.  GP-PDA Filter for Extended Target Tracking With Measurement Origin Uncertainty , 2019, IEEE Transactions on Aerospace and Electronic Systems.

[16]  Ángel F. García-Fernández,et al.  Trajectory PHD and CPHD Filters , 2018, IEEE Transactions on Signal Processing.

[17]  A. Aziz A new multitarget tracking approach based on a non-iterative fuzzy clustering means algorithm , 2015, 2015 IEEE Aerospace Conference.

[18]  Zhenyu He,et al.  One global optimization method in network flow model for multiple object tracking , 2015, Knowl. Based Syst..

[19]  Amit Konar,et al.  A Deterministic Improved Q-Learning for Path Planning of a Mobile Robot , 2013, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[20]  Dominique Gruyer,et al.  Dual multi-targets tracking for ambiguities' identification and solving , 2014, 2014 IEEE Intelligent Vehicles Symposium Proceedings.

[21]  Liang-qun Li,et al.  Intuitionistic fuzzy joint probabilistic data association filter and its application to multitarget tracking , 2014, Signal Process..

[22]  Kerim Guney,et al.  Cheap Joint Probabilistic Data Association with Adaptive Neuro-Fuzzy Inference System State Filter for Tracking Multiple Targets in Cluttered Environment , 2004 .

[23]  Frank L. Lewis,et al.  Optimal and Autonomous Control Using Reinforcement Learning: A Survey , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[24]  S. Singh,et al.  Novel data association schemes for the probability hypothesis density filter , 2007, IEEE Transactions on Aerospace and Electronic Systems.

[25]  Ba-Ngu Vo,et al.  A Generalized Labeled Multi-Bernoulli Filter With Object Spawning , 2017, IEEE Transactions on Signal Processing.

[26]  Ba-Ngu Vo,et al.  Labeled Random Finite Sets and Multi-Object Conjugate Priors , 2013, IEEE Transactions on Signal Processing.

[27]  Jan Peters,et al.  Reinforcement learning in robotics: A survey , 2013, Int. J. Robotics Res..

[28]  Penina Axelrad,et al.  An AEGIS-CPHD Filter to Maintain Custody of GEO Space Objects with Limited Tracking Data , 2014 .

[29]  A. Dallil,et al.  Sensor Fusion and Target Tracking Using Evidential Data Association , 2013, IEEE Sensors Journal.

[30]  Dominique Gruyer,et al.  Multi-criteria similarity operator based on the Belief Theory: Management of similarity, dissimilarity, conflict and ambiguities , 2017, 2017 IEEE Intelligent Vehicles Symposium (IV).

[31]  Afshin Dehghan,et al.  On Detection, Data Association and Segmentation for Multi-Target Tracking , 2019, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[32]  Y. M. Chen Information fusion in data association applications , 2006, Appl. Soft Comput..

[33]  Yaakov Bar-Shalom,et al.  Multi-target tracking using joint probabilistic data association , 1980, 1980 19th IEEE Conference on Decision and Control including the Symposium on Adaptive Processes.

[34]  Tat-Jun Chin,et al.  A Multiple Hypothesis Tracker for Multitarget Tracking With Multiple Simultaneous Measurements , 2013, IEEE Journal of Selected Topics in Signal Processing.

[35]  Taek Lyul Song,et al.  A probabilistic nearest neighbor filter algorithm for m validated measurements , 2006, IEEE Transactions on Signal Processing.

[36]  Jeffrey K. Uhlmann,et al.  Unscented filtering and nonlinear estimation , 2004, Proceedings of the IEEE.

[37]  Rachid Belaroussi,et al.  Multi-Hypotheses Tracking using the Dempster-Shafer Theory, application to ambiguous road context , 2016, Inf. Fusion.

[38]  Amadou Gning,et al.  A Box Particle Filter Method for Tracking Multiple Extended Objects , 2019, IEEE Transactions on Aerospace and Electronic Systems.

[39]  Xie Weixin,et al.  Intuitionistic fuzzy joint probabilistic data association filter and its application to multitarget tracking , 2014 .