Multiple-Object-Tracking Algorithm Based on Dense Trajectory Voting in Aerial Videos

In recent years, UAV technology has developed rapidly. Due to the mobility, low cost, and variable monitoring altitude of UAVs, multiple-object detection and tracking in aerial videos has become a research hotspot in the field of computer vision. However, due to camera motion, small target size, target adhesion, and unpredictable target motion, it is still difficult to detect and track targets of interest in aerial videos, especially in the case of a low frame rate where the target position changes too much. In this paper, we propose a multiple-object-tracking algorithm based on dense-trajectory voting in aerial videos. The method models the multiple-target-tracking problem as a voting problem of the dense-optical-flow trajectory to the target ID, which can be applied to aerial-surveillance scenes and is robust to low-frame-rate videos. More specifically, we first built an aerial video dataset for vehicle targets, including a training dataset and a diverse test dataset. Based on this, we trained the neural network model by using a deep-learning method to detect vehicles in aerial videos. Thereafter, we calculated the dense optical flow in adjacent frames, and generated effective dense-optical-flow trajectories in each detection bounding box at the current time. When target IDs of optical-flow trajectories are known, the voting results of the optical-flow trajectories in each detection bounding box are counted. Finally, similarity between detection objects in adjacent frames was measured based on the voting results, and tracking results were obtained by data association. In order to evaluate the performance of this algorithm, we conducted experiments on self-built test datasets. A large number of experimental results showed that the proposed algorithm could obtain good target-tracking results in various complex scenarios, and performance was still robust at a low frame rate by changing the video frame rate. In addition, we carried out qualitative and quantitative comparison experiments between the algorithm and three state-of-the-art tracking algorithms, which further proved that this algorithm could not only obtain good tracking results in aerial videos with a normal frame rate, but also had excellent performance under low-frame-rate conditions.

[1]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[2]  Zhiyong Cui,et al.  Real-Time Bidirectional Traffic Flow Parameter Estimation From Aerial Videos , 2017, IEEE Transactions on Intelligent Transportation Systems.

[3]  Alex Bewley,et al.  Deep Cosine Metric Learning for Person Re-identification , 2018, 2018 IEEE Winter Conference on Applications of Computer Vision (WACV).

[4]  Dietrich Paulus,et al.  Simple online and realtime tracking with a deep association metric , 2017, 2017 IEEE International Conference on Image Processing (ICIP).

[5]  Harpreet S. Sawhney,et al.  Vehicle detection and tracking in wide field-of-view aerial video , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[6]  Rainer Stiefelhagen,et al.  Evaluating Multiple Object Tracking Performance: The CLEAR MOT Metrics , 2008, EURASIP J. Image Video Process..

[7]  Ali Farhadi,et al.  YOLOv3: An Incremental Improvement , 2018, ArXiv.

[8]  Liang Sun,et al.  A Scalable Multitarget Tracking System for Cooperative Unmanned Aerial Vehicles , 2017, IEEE Transactions on Aerospace and Electronic Systems.

[9]  Thomas Brox,et al.  Motion Trajectory Segmentation via Minimum Cost Multicuts , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[10]  Pingkun Yan,et al.  Correlation-Based Tracking of Multiple Targets With Hierarchical Layered Structure , 2018, IEEE Transactions on Cybernetics.

[11]  Volker Eiselein,et al.  High-Speed tracking-by-detection without using image information , 2017, 2017 14th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS).

[12]  Congcong Li,et al.  Visual Detail Augmented Mapping for Small Aerial Target Detection , 2019, Remote. Sens..

[13]  Wei Liu,et al.  SSD: Single Shot MultiBox Detector , 2015, ECCV.

[14]  Jing Li,et al.  Panoramic UAV Surveillance and Recycling System Based on Structure-Free Camera Array , 2019, IEEE Access.

[15]  Ullrich Köthe,et al.  Globally Optimal Closed-Surface Segmentation for Connectomics , 2012, ECCV.

[16]  Fukun Bi,et al.  Remote Sensing Target Tracking in UAV Aerial Video Based on Saliency Enhanced MDnet , 2019, IEEE Access.

[17]  Gunnar Farnebäck,et al.  Two-Frame Motion Estimation Based on Polynomial Expansion , 2003, SCIA.

[18]  Yunhong Wang,et al.  Moving Object Detection in Aerial Video , 2012, 2012 11th International Conference on Machine Learning and Applications.

[19]  H. Kuhn The Hungarian method for the assignment problem , 1955 .

[20]  Joseph Redmon,et al.  YOLOv3: An Incremental Improvement , 2018, ArXiv.

[21]  Matthew J. Hoffman,et al.  Real-Time Vehicle Tracking in Aerial Video Using Hyperspectral Features , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[22]  De Xu,et al.  Robust Visual Detection–Learning–Tracking Framework for Autonomous Aerial Refueling of UAVs , 2016, IEEE Transactions on Instrumentation and Measurement.

[23]  Fangling Pu,et al.  Detection, Tracking, and Geolocation of Moving Vehicle From UAV Using Monocular Camera , 2019, IEEE Access.

[24]  Supun Samarasekera,et al.  Aerial video surveillance and exploitation , 2001, Proc. IEEE.

[25]  Arturo de la Escalera,et al.  An Appearance-Based Tracking Algorithm for Aerial Search and Rescue Purposes † , 2019, Sensors.

[26]  Hichem Sahli,et al.  A Hierarchical Association Framework for Multi-Object Tracking in Airborne Videos , 2018, Remote. Sens..

[27]  M. Shah,et al.  Moving Object Detection and Tracking in Forward Looking Infra-Red Aerial Imagery , 2011 .

[28]  Isabella Szottka,et al.  Tracking multiple vehicles in airborne image sequences of complex urban environments , 2011, 2011 Joint Urban Remote Sensing Event.

[29]  Jing Li,et al.  Real-Time Ground Vehicle Detection in Aerial Infrared Imagery Based on Convolutional Neural Network , 2018 .

[30]  Jitendra Malik,et al.  Ieee Transactions on Pattern Analysis and Machine Intelligence Segmentation of Moving Objects by Long Term Video Analysis , 2022 .

[31]  Fabio Tozeto Ramos,et al.  Simple online and realtime tracking , 2016, 2016 IEEE International Conference on Image Processing (ICIP).

[32]  Bernt Schiele,et al.  Learning Non-maximum Suppression , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[33]  Ali Farhadi,et al.  YOLO9000: Better, Faster, Stronger , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[34]  Saeid Nahavandi,et al.  Efficient Road Detection and Tracking for Unmanned Aerial Vehicle , 2015, IEEE Transactions on Intelligent Transportation Systems.