Parallel Ensemble Deep Learning for Real-Time Remote Sensing Video Multi-Target Detection

Unmanned aerial vehicle (UAV) is one of the main means of information warfare, such as in battlefield cruises, reconnaissance, and military strikes. Rapid detection and accurate recognition of key targets in UAV images are the basis of subsequent military tasks. The UAV image has characteristics of high resolution and small target size, and in practical application, the detection speed is often required to be fast. Existing algorithms are not able to achieve an effective trade-off between detection accuracy and speed. Therefore, this paper proposes a parallel ensemble deep learning framework for unmanned aerial vehicle video multi-target detection, which is a global and local joint detection strategy. It combines a deep learning target detection algorithm with template matching to make full use of image information. It also integrates multi-process and multi-threading mechanisms to speed up processing. Experiments show that the system has high detection accuracy for targets with focal lengths varying from one to ten times. At the same time, the real-time and stable display of detection results is realized by aiming at the moving UAV video image.

[1]  Aníbal Ollero,et al.  Autonomous Aerial Filming With Distributed Lighting by a Team of Unmanned Aerial Vehicles , 2021, IEEE Robotics and Automation Letters.

[2]  Wen-Cheng Liu,et al.  Large-Scale Particle Image Velocimetry to Measure Streamflow from Videos Recorded from Unmanned Aerial Vehicle and Fixed Imaging System , 2021, Remote. Sens..

[3]  W S McCulloch,et al.  A logical calculus of the ideas immanent in nervous activity , 1990, The Philosophy of Artificial Intelligence.

[4]  Peng Lu,et al.  Towards Robust Visual Tracking for Unmanned Aerial Vehicle with Tri-Attentional Correlation Filters , 2020, 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[5]  Mohamed S. Shehata,et al.  Local null space pursuit for real-time moving object detection in aerial surveillance , 2020, Signal Image Video Process..

[6]  W. Saad,et al.  Deep Learning for Optimal Deployment of UAVs With Visible Light Communications , 2019, IEEE Transactions on Wireless Communications.

[7]  Alexander Wong,et al.  YOLO Nano: a Highly Compact You Only Look Once Convolutional Neural Network for Object Detection , 2019, 2019 Fifth Workshop on Energy Efficient Machine Learning and Cognitive Computing - NeurIPS Edition (EMC2-NIPS).

[8]  Yuehu Liu,et al.  Performance Evaluation of Visual Object Detection for Moving Vehicle , 2019, Communications in Computer and Information Science.

[9]  Heesung Kwon,et al.  Delving Into Robust Object Detection From Unmanned Aerial Vehicles: A Deep Nuisance Disentanglement Approach , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[10]  Pengyi Zhang,et al.  SlimYOLOv3: Narrower, Faster and Better for Real-Time UAV Applications , 2019, 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW).

[11]  Praveen Kumar,et al.  Real-time implementation of moving object detection in UAV videos using GPUs , 2019, Journal of Real-Time Image Processing.

[12]  Tony Q. S. Quek,et al.  Multi-layer Unmanned Aerial Vehicle Networks: Modeling and Performance Analysis , 2019, IEEE Transactions on Wireless Communications.

[13]  Xiaoliang Wang,et al.  Fast and Accurate, Convolutional Neural Network Based Approach for Object Detection from UAV , 2018, IECON 2018 - 44th Annual Conference of the IEEE Industrial Electronics Society.

[14]  Ali Farhadi,et al.  YOLOv3: An Incremental Improvement , 2018, ArXiv.

[15]  Zhi Zhang,et al.  Fast Deep Neural Networks With Knowledge Guided Training and Predicted Regions of Interests for Real-Time Video Object Detection , 2018, IEEE Access.

[16]  Ari Legowo,et al.  Object Detection Technique for Small Unmanned Aerial Vehicle , 2017 .

[17]  Hongyi Zhang,et al.  mixup: Beyond Empirical Risk Minimization , 2017, ICLR.

[18]  Yang Long,et al.  Airport Detection Based on a Multiscale Fusion Feature for Optical Remote Sensing Images , 2017, IEEE Geoscience and Remote Sensing Letters.

[19]  Yong Dou,et al.  Airport Detection on Optical Satellite Images Using Deep Convolutional Neural Networks , 2017, IEEE Geoscience and Remote Sensing Letters.

[20]  Kaiming He,et al.  Feature Pyramid Networks for Object Detection , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[21]  Silvio Savarese,et al.  Learning Social Etiquette: Human Trajectory Understanding In Crowded Scenes , 2016, ECCV.

[22]  Sammy A. Perdomo,et al.  Weed detection in rice fields using aerial images and neural networks , 2016, 2016 XXI Symposium on Signal Processing, Images and Artificial Vision (STSIVA).

[23]  Andrey A. Filchenkov,et al.  Application of deep learning to the problem of vehicle detection in UAV images , 2016, 2016 XIX IEEE International Conference on Soft Computing and Measurements (SCM).

[24]  Yi Li,et al.  R-FCN: Object Detection via Region-based Fully Convolutional Networks , 2016, NIPS.

[25]  Rui Zhang,et al.  Wireless communications with unmanned aerial vehicles: opportunities and challenges , 2016, IEEE Communications Magazine.

[26]  Wei Liu,et al.  SSD: Single Shot MultiBox Detector , 2015, ECCV.

[27]  Vladlen Koltun,et al.  Multi-Scale Context Aggregation by Dilated Convolutions , 2015, ICLR.

[28]  Ali Farhadi,et al.  You Only Look Once: Unified, Real-Time Object Detection , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[29]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[30]  Ross B. Girshick,et al.  Fast R-CNN , 2015, 1504.08083.

[31]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[32]  Rongjun Qin,et al.  An Object-Based Hierarchical Method for Change Detection Using Unmanned Aerial Vehicle Images , 2014, Remote. Sens..

[33]  Jian Sun,et al.  Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[34]  Trevor Darrell,et al.  Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation , 2013, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[35]  Koen E. A. van de Sande,et al.  Selective Search for Object Recognition , 2013, International Journal of Computer Vision.

[36]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[37]  Yoshua Bengio,et al.  Understanding the difficulty of training deep feedforward neural networks , 2010, AISTATS.

[38]  Jae-Young Choi,et al.  Multiple Vehicles Detection and Tracking based on Scale-Invariant Feature Transform , 2007, 2007 IEEE Intelligent Transportation Systems Conference.

[39]  Paul A. Viola,et al.  Rapid object detection using a boosted cascade of simple features , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[40]  Р Ю Чуйков,et al.  Обнаружение транспортных средств на изображениях загородных шоссе на основе метода Single shot multibox Detector , 2017 .

[41]  Nuri Yilmazer,et al.  Application of Object Detection and Tracking Techniques for Unmanned Aerial Vehicles , 2015, Complex Adaptive Systems.

[42]  Toby P. Breckon,et al.  AUTOMATIC SALIENT OBJECT DETECTION IN UAV IMAGERY , 2010 .

[43]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.