Online multi-object tracking with pedestrian re-identification and occlusion processing

Tracking-by-detection is a common approach for online multi-object tracking problem. At present, the following challenges still exist in the multi-object tracking scenarios: (1) The result of object re-tracking after full occlusion is not ideal; (2) The predicted position of object is not accurate enough in the complicated video scenarios. Aiming at these two problems, this paper proposes a multi-object tracking framework called DROP (Deep Re-identification Occlusion Processing). The framework consists of object detection, fast pedestrian re-identification, and a confidence-based data association algorithm. A lightweight convolutional neural network that can solve the re-tracking problem is constructed by increasing and learning the affinity of appearance features of the same object in different frames. And this paper proposes to judge the occlusion of the object that can solve inaccurate position predicted by Kalman filter by using the data association result of the appearance features of the object, and to reduce the matching error by improving the data association formula. The experimental results on the multi-object tracking datasets MOT15 and MOT16 show that the proposed method can improve the precision while ensure the real-time tracking performance.

[1]  Ehsan Fazl-Ersi,et al.  Revisiting correlation-based filters for low-resolution and long-term visual tracking , 2019, The Visual Computer.

[2]  Dietrich Paulus,et al.  Simple online and realtime tracking with a deep association metric , 2017, 2017 IEEE International Conference on Image Processing (ICIP).

[3]  Konrad Schindler,et al.  Online Multi-Target Tracking Using Recurrent Neural Networks , 2016, AAAI.

[4]  Bacha Rehman,et al.  Face detection and tracking using hybrid margin-based ROI techniques , 2019, The Visual Computer.

[5]  Volker Eiselein,et al.  High-Speed tracking-by-detection without using image information , 2017, 2017 14th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS).

[6]  Silvio Savarese,et al.  Learning to Track: Online Multi-object Tracking by Decision Making , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[7]  Nenghai Yu,et al.  Online Multi-object Tracking Using CNN-Based Single Object Tracker with Spatial-Temporal Attention Mechanism , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[8]  Guixi Liu,et al.  Coupled-layer based visual tracking via adaptive kernelized correlation filters , 2016, The Visual Computer.

[9]  Zhen Qin,et al.  Improving multi-target tracking via social grouping , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[10]  Hao Guan,et al.  How do deep convolutional features affect tracking performance: an experimental study , 2017, The Visual Computer.

[11]  Konrad Schindler,et al.  Continuous Energy Minimization for Multitarget Tracking , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[12]  Ehsan Fazl Ersi,et al.  Revisiting correlation-based filters for low-resolution and long-term visual tracking , 2019, Vis. Comput..

[13]  Kwangjin Yoon,et al.  Online Multi-Object Tracking with Historical Appearance Matching and Scene Adaptive Detection Filtering , 2018, 2018 15th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS).

[14]  Fabio Poiesi,et al.  Online Multi-target Tracking with Strong and Weak Detections , 2016, ECCV Workshops.

[15]  Jiping Sun,et al.  Real-time multi-scale tracking based on compressive sensing , 2014, The Visual Computer.

[16]  Yu Liu,et al.  POI: Multiple Object Tracking with High Performance Detection and Appearance Feature , 2016, ECCV Workshops.

[17]  Hua Yang,et al.  Online Multi-Object Tracking with Dual Matching Attention Networks , 2018, ECCV.

[18]  Francois Bremond,et al.  Multi-Object tracking using multi-channel part appearance representation , 2017, 2017 14th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS).

[19]  Oscar Koller,et al.  Using Convolutional 3D Neural Networks for User-independent continuous gesture recognition , 2016, 2016 23rd International Conference on Pattern Recognition (ICPR).

[20]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[21]  Yue Cao,et al.  Spatial-Temporal Relation Networks for Multi-Object Tracking , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[22]  Mubarak Shah,et al.  (MP)2T: Multiple People Multiple Parts Tracker , 2012, ECCV.

[23]  Amir Khajepour,et al.  A Potential Field-Based Model Predictive Path-Planning Controller for Autonomous Road Vehicles , 2017, IEEE Transactions on Intelligent Transportation Systems.

[24]  Mohammad Rahmati,et al.  Multi-target tracking using CNN-based features: CNNMTT , 2018, Multimedia Tools and Applications.

[25]  Lucas Beyer,et al.  In Defense of the Triplet Loss for Person Re-Identification , 2017, ArXiv.

[26]  Kwangjin Yoon,et al.  Online Multi-Object Tracking Using Selective Deep Appearance Matching , 2018, 2018 IEEE International Conference on Consumer Electronics - Asia (ICCE-Asia).

[27]  Pengfei Li,et al.  Moving object tracking based on multi-independent features distribution fields with comprehensive spatial feature similarity , 2014, The Visual Computer.

[28]  Francesco Solera,et al.  Performance Measures and a Data Set for Multi-target, Multi-camera Tracking , 2016, ECCV Workshops.

[29]  Fabio Tozeto Ramos,et al.  Simple online and realtime tracking , 2016, 2016 IEEE International Conference on Image Processing (ICIP).

[30]  Silvio Savarese,et al.  Recurrent Autoregressive Networks for Online Multi-object Tracking , 2017, 2018 IEEE Winter Conference on Applications of Computer Vision (WACV).

[31]  Stefan Roth,et al.  MOTChallenge 2015: Towards a Benchmark for Multi-Target Tracking , 2015, ArXiv.

[32]  Jian Sun,et al.  Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[33]  Silvio Savarese,et al.  A Unified Framework for Multi-target Tracking and Collective Activity Recognition , 2012, ECCV.

[34]  Moongu Jeon,et al.  Joint cost minimization for multi-object tracking , 2017, 2017 14th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS).

[35]  Luc Van Gool,et al.  Robust tracking-by-detection using a detector confidence particle filter , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[36]  Silvio Savarese,et al.  Tracking the Untrackable: Learning to Track Multiple Cues with Long-Term Dependencies , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[37]  Lu Zhang,et al.  Preserving Structure in Model-Free Tracking , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[38]  Kwangjin Yoon,et al.  Data Association for Multi-Object Tracking via Deep Neural Networks , 2019, Sensors.

[39]  Robert T. Collins,et al.  Multi-target Tracking by Lagrangian Relaxation to Min-cost Network Flow , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[40]  Shandong Wu,et al.  Object tracking based on Huber loss function , 2019, The Visual Computer.