Efficient Object Detection Model for Real-Time UAV Applications

Unmanned Aerial Vehicles (UAVs) especially drones, equipped with vision techniques have become very popular in recent years, with their extensive use in wide range of applications. Many of these applications require use of computer vision techniques, particularly object detection from the information captured by on-board camera. In this paper, we propose an end to end object detection model running on a UAV platform which is suitable for real-time applications. We propose a deep feature pyramid architecture which makes use of inherent properties of features extracted from Convolutional Networks by capturing more generic features in the images (such as edge, color etc.) along with the minute detailed features specific to the classes contained in our problem. We use VisDrone-18 dataset for our studies which contain different objects such as pedestrians, vehicles, bicycles etc. We provide software and hardware architecture of our platform used in this study. We implemented our model with both ResNet and MobileNet as convolutional bases. Our model combined with modified focal loss function, produced a desirable performance of 30.6 mAP for object detection with an inference time of 14 fps. We compared our results with RetinaNet-ResNet-50 and HAL-RetinaNet and shown that our model combined with MobileNet as backend feature extractor gave the best results in terms of accuracy, speed and memory efficiency and is best suitable for real time object detection with drones.

[1]  Trevor Darrell,et al.  Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation , 2013, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[2]  Gaurav S. Sukhatme,et al.  Vision-based autonomous landing of an unmanned aerial vehicle , 2002, Proceedings 2002 IEEE International Conference on Robotics and Automation (Cat. No.02CH37292).

[3]  Lin Lei,et al.  Fast vehicle detection in UAV images , 2017, 2017 International Workshop on Remote Sensing with Intelligent Processing (RSIP).

[4]  Truong Q. Nguyen,et al.  Low-complexity object detection with deep convolutional neural network for embedded systems , 2017, Optical Engineering + Applications.

[5]  Jerry D. Gibson,et al.  Low-complexity video encoding for UAV reconnaissance and surveillance , 2011, 2011 - MILCOM 2011 Military Communications Conference.

[6]  Wei Liu,et al.  SSD: Single Shot MultiBox Detector , 2015, ECCV.

[7]  Peter I. Corke,et al.  Outdoor Flight Testing of a Pole Inspection UAV Incorporating High-speed Vision , 2013, FSR.

[8]  Erik Blasch,et al.  Enabling Smart Urban Surveillance at The Edge , 2017, 2017 IEEE International Conference on Smart Cloud (SmartCloud).

[9]  Bo Chen,et al.  MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications , 2017, ArXiv.

[10]  Flavio Fontana,et al.  Continuous on-board monocular-vision-based elevation mapping applied to autonomous landing of micro aerial vehicles , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[11]  L. Samarakoon,et al.  LOW COST AERIAL MAPPING WITH CONSUMER-GRADE DRONES , 2016 .

[12]  G. Anitha,et al.  Vision Based Autonomous Landing of an Unmanned Aerial Vehicle , 2012 .

[13]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[14]  Abhinav Gupta,et al.  Training Region-Based Object Detectors with Online Hard Example Mining , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[15]  Yi Li,et al.  R-FCN: Object Detection via Region-based Fully Convolutional Networks , 2016, NIPS.

[16]  Roland Siegwart,et al.  Vision-Controlled Micro Flying Robots: From System Design to Autonomous Navigation and Mapping in GPS-Denied Environments , 2014, IEEE Robotics & Automation Magazine.

[17]  Patrick Doherty,et al.  A UAV Search and Rescue Scenario with Human Body Detection and Geolocalization , 2007, Australian Conference on Artificial Intelligence.

[18]  Marc Pollefeys,et al.  PIXHAWK: A micro aerial vehicle design for autonomous flight using onboard computer vision , 2012, Auton. Robots.

[19]  Terrence Fong,et al.  Vehicle detection from aerial imagery , 2011, 2011 IEEE International Conference on Robotics and Automation.

[20]  Ali Farhadi,et al.  You Only Look Once: Unified, Real-Time Object Detection , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[21]  Kaiming He,et al.  Feature Pyramid Networks for Object Detection , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[22]  Qinghua Hu,et al.  Vision Meets Drones: A Challenge , 2018, ArXiv.

[23]  Sudipta N. Sinha,et al.  Monocular Localization of a moving person onboard a Quadrotor MAV , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[24]  Kaiming He,et al.  Focal Loss for Dense Object Detection , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[25]  Ranveer Chandra,et al.  FarmBeats: Automating Data Aggregation , 2018 .

[26]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[27]  J.K. Hedrick,et al.  An overview of emerging results in cooperative UAV control , 2004, 2004 43rd IEEE Conference on Decision and Control (CDC) (IEEE Cat. No.04CH37601).

[28]  Takeo Kanade,et al.  Human Face Detection in Visual Scenes , 1995, NIPS.

[29]  Christos-Savvas Bouganis,et al.  DroNet: Efficient convolutional neural network detector for real-time UAV applications , 2018, 2018 Design, Automation & Test in Europe Conference & Exhibition (DATE).