SeaDroneSim: Simulation of Aerial Images for Detection of Objects Above Water

Unmanned Aerial Vehicles (UAVs) are known for their speed and versatility in collecting aerial images and remote sensing for land use surveys and precision agriculture. With UAVs' growth in availability and accessibility, they are now of vital importance as technological support in marine-based applications such as vessel monitoring and search-and-rescue (SAR) operations. High-resolution cameras and Graphic processing units (GPUs) can be equipped on the UAVs to effectively and efficiently aid in locating objects of interest, lending themselves to emergency rescue operations or, in our case, precision aquaculture applications. Modern computer vision algorithms allow us to detect objects of interest in a dynamic environment; however, these algorithms are dependent on large training datasets collected from UAVs, which are currently time-consuming and labor-intensive to collect for maritime environments. To this end, we present a new benchmark suite, SeaD-roneSim, that can be used to create photo-realistic aerial image datasets with ground truth for segmentation masks of any given object. Utilizing only the synthetic data gen-erated from SeaDroneSim, we obtained 71 a mean Average Precision (mAP) on real aerial images for detecting our ob-ject of interest, a popular, open source, remotely operated underwater vehicle (BlueROV) in this feasibility study. The results of this new simulation suit serve as a baseline for the detection of the BlueROV, which can be used in underwater surveys of oyster reefs and other marine applications.

[1]  Y. Aloimonos,et al.  OysterSim: Underwater Simulation for Enhancing Oyster Reef Monitoring , 2022, OCEANS 2022, Hampton Roads.

[2]  Y. Aloimonos,et al.  OysterNet: Enhanced Oyster Detection Using Simulation , 2022, 2023 IEEE International Conference on Robotics and Automation (ICRA).

[3]  Andreas Zell,et al.  Leveraging Synthetic Data in Object Detection on Unmanned Aerial Vehicles , 2021, 2022 26th International Conference on Pattern Recognition (ICPR).

[4]  Chahat Deep Singh,et al.  PRGFlow: Unified SWAP‐aware deep global optical flow for aerial robot navigation , 2021, Electronics Letters.

[5]  Andreas Zell,et al.  SeaDronesSee: A Maritime Benchmark for Detecting Humans in Open Water , 2021, 2022 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV).

[6]  A. Aibinu,et al.  Unmanned Aerial Vehicle (UAV) applications in coastal zone management—a review , 2021, Environmental Monitoring and Assessment.

[7]  Pei-Chun Chen,et al.  Imaging Using Unmanned Aerial Vehicles for Agriculture Land Use Classification , 2020, Agriculture.

[8]  João Valente,et al.  An Open Simulation Strategy for Rapid Control Design in Aerial and Maritime Drone Teams: A Comprehensive Tutorial , 2020 .

[9]  Edison Pignaton de Freitas,et al.  Multi-UAV Based Crowd Monitoring System , 2020, IEEE Transactions on Aerospace and Electronic Systems.

[10]  Vipul Mishra,et al.  Drone-surveillance for search and rescue in natural disaster , 2020, Comput. Commun..

[11]  Vincenzo Sciancalepore,et al.  SARDO: An Automated Search-and-Rescue Drone-Based Solution for Victims Localization , 2020, IEEE Transactions on Mobile Computing.

[12]  Panagiotis G. Sarigiannidis,et al.  A Review on UAV-Based Applications for Precision Agriculture , 2019, Inf..

[13]  Antonios Gasteratos,et al.  Unsupervised Human Detection with an Embedded Vision System on a Fully Autonomous UAV for Search and Rescue Operations , 2019, Sensors.

[14]  Antonio-Javier Gallego,et al.  Detection of bodies in maritime rescue operations using unmanned aerial vehicles with multispectral cameras , 2018, J. Field Robotics.

[15]  Rishad A. Irani,et al.  Unmanned Aerial Vehicle Landing on Maritime Vessels using Signal Prediction of the Ship Motion , 2018, OCEANS 2018 MTS/IEEE Charleston.

[16]  Bo Zhu,et al.  A Vision-Aided Navigation System by Ground-Aerial Vehicle Cooperation for UAV in GNSS-Denied Environments , 2018, 2018 IEEE CSAA Guidance, Navigation and Control Conference (CGNCC).

[17]  Xiao Xiang Zhu,et al.  HSF-Net: Multiscale Deep Feature Embedding for Ship Detection in Optical Remote Sensing Imagery , 2018, IEEE Transactions on Geoscience and Remote Sensing.

[18]  Antonio-Javier Gallego,et al.  Automatic Ship Classification from Optical Aerial Images with Convolutional Neural Networks , 2018, Remote. Sens..

[19]  Yunus Karaca,et al.  The potential use of unmanned aircraft systems (drones) in mountain search and rescue operations , 2017, The American journal of emergency medicine.

[20]  Tarik Taleb,et al.  UAV-Based IoT Platform: A Crowd Surveillance Use Case , 2017, IEEE Communications Magazine.

[21]  Özlem Akar,et al.  Mapping land use with using Rotation Forest algorithm from UAV images , 2017 .

[22]  Vincenzo Lippiello,et al.  Multimodal Interaction with Multiple Co-located Drones in Search and Rescue Missions , 2016, ArXiv.

[23]  Sharul Sham Dol,et al.  A Review on Marine Search and Rescue Operations Using Unmanned Aerial Vehicles , 2015 .

[24]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[25]  John S. Duncan,et al.  The Use of Simulation to De-risk Maritime UAV Operations , 2007 .

[26]  Soumyashree Kar,et al.  Precision Agriculture and Unmanned Aerial Vehicles (UAVs) , 2019 .