AirSim-W: A Simulation Environment for Wildlife Conservation with UAVs

Increases in poaching levels have led to the use of unmanned aerial vehicles (UAVs or drones) to count animals, locate animals in parks, and even find poachers. Finding poachers is often done at night through the use of long wave thermal infrared cameras mounted on these UAVs. Unfortunately, monitoring the live video stream from the conservation UAVs all night is an arduous task. In order to assist in this monitoring task, new techniques in computer vision have been developed. This work is based on a dataset which took approximately six months to label. However, further improvement in detection and future testing of autonomous flight require not only more labeled training data, but also an environment where algorithms can be safely tested. In order to meet both goals efficiently, we present AirSim-W, a simulation environment that has been designed specifically for the domain of wildlife conservation. This includes (i) creation of an African savanna environment in Unreal Engine, (ii) integration of a new thermal infrared model based on radiometry, (iii) API code expansions to follow objects of interest or fly in zig-zag patterns to generate simulated training data, and (iv) demonstrated detection improvement using simulated data generated by AirSim-W. With these additional simulation features, AirSim-W will be directly useful for wildlife conservation research.

[1]  Ramakant Nevatia,et al.  SPOT Poachers in Action: Augmenting Conservation Drones With Automatic Detection in Near Real Time , 2018, AAAI.

[2]  Haifeng Xu,et al.  Strategic Coordination of Human Patrollers and Mobile Sensors With Signaling for Security Games , 2018, AAAI.

[3]  Gustavo Carneiro,et al.  A Bayesian Data Augmentation Approach for Learning Deep Models , 2017, NIPS.

[4]  Haifeng Xu,et al.  Optimal Patrol Planning for Green Security Games with Black-Box Attackers , 2017, GameSec.

[5]  Milind Tambe,et al.  VIOLA: Video Labeling Application for Security Domains , 2017, GameSec.

[6]  Nicholas Roy,et al.  GeneSIS-Rt: Generating Synthetic Images for Training Secondary Real-World Tasks , 2017, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[7]  Milind Tambe,et al.  Taking It for a Test Drive: A Hybrid Spatio-Temporal Model for Wildlife Poaching Prediction Evaluated Through a Controlled Field Test , 2017, ECML/PKDD.

[8]  I. C. Prentice,et al.  Global climatic drivers of leaf size , 2017, Science.

[9]  Dongwoo Yeom,et al.  Study of data-driven thermal sensation prediction model as a function of local body skin temperatures in a built environment , 2017 .

[10]  Philip David,et al.  Domain Adaptation for Semantic Segmentation of Urban Scenes , 2017 .

[11]  Ashish Kapoor,et al.  AirSim: High-Fidelity Visual and Physical Simulation for Autonomous Vehicles , 2017, FSR.

[12]  Alexei A. Efros,et al.  Unpaired Image-to-Image Translation Using Cycle-Consistent Adversarial Networks , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[13]  Sylvain Paris,et al.  Deep Photo Style Transfer , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[14]  Hong Liu,et al.  A simplified thermoregulation model of the human body in warm conditions. , 2017, Applied ergonomics.

[15]  Tomas Pfister,et al.  Learning from Simulated and Unsupervised Images through Adversarial Training , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[16]  Dumitru Erhan,et al.  Unsupervised Pixel-Level Domain Adaptation with Generative Adversarial Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[17]  Matthew Johnson-Roberson,et al.  Driving in the Matrix: Can virtual worlds replace human-generated annotations for real world tasks? , 2016, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[18]  Musaed Alhussein,et al.  Simulation and Analysis of Uncooled Microbolometer for Serial Readout Architecture , 2016, J. Sensors.

[19]  Antonio M. López,et al.  The SYNTHIA Dataset: A Large Collection of Synthetic Images for Semantic Segmentation of Urban Scenes , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[20]  Bo An,et al.  Deploying PAWS: Field Optimization of the Protection Assistant for Wildlife Security , 2016, AAAI.

[21]  Hui Jiang,et al.  Generating images with recurrent adversarial networks , 2016, ArXiv.

[22]  Miguel A. Olivares-Méndez,et al.  Towards an Autonomous Vision-Based Unmanned Aerial System against Wildlife Poachers , 2015, Sensors.

[23]  Soumith Chintala,et al.  Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks , 2015, ICLR.

[24]  A. Plumptre,et al.  Spatiotemporal trends of illegal activities from ranger‐collected data in a Ugandan national park , 2015, Conservation biology : the journal of the Society for Conservation Biology.

[25]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[26]  Alex Graves,et al.  DRAW: A Recurrent Neural Network For Image Generation , 2015, ICML.

[27]  Pascal Mettes,et al.  Nature Conservation Drones for Automatic Localization and Counting of Animals , 2014, ECCV Workshops.

[28]  Yanfeng Liu,et al.  A study of human skin and surface temperatures in stable and unstable thermal environments , 2013 .

[29]  D. B. Shah,et al.  Field Measurements of Plant Emissivity Spectra: An Experimental Study on Remote Sensing of Vegetation in the Thermal Infrared Region , 2013, Journal of the Indian Society of Remote Sensing.

[30]  G. S. Bakken,et al.  Heat storage in Asian elephants during submaximal exercise: behavioral regulation of thermoregulatory constraints on activity in endothermic gigantotherms , 2013, Journal of Experimental Biology.

[31]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[32]  R. K. Bhan,et al.  PSPICE circuit simulation of microbolometer infrared detectors with noise sources , 2012 .

[33]  C. Downs,et al.  Body temperature and basking behaviour of Nile crocodiles (Crocodylus niloticus) during winter , 2008 .

[34]  Ian Ridpath,et al.  A Dictionary of Astronomy , 2007 .

[35]  D. McCafferty The value of infrared thermography for research on mammals: previous applications and future directions , 2007 .

[36]  Andrew Howard,et al.  Design and use paradigms for Gazebo, an open-source multi-robot simulator , 2004, 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (IEEE Cat. No.04CH37566).

[37]  Scott D. Brown,et al.  Advances in wide-area hyperspectral image simulation , 2003, SPIE Defense + Commercial Sensing.

[38]  B. Sharratt,et al.  Thermal properties of soil , 2001 .

[39]  K. Lomas,et al.  Computer prediction of human thermoregulatory and temperature responses to a wide range of environmental conditions , 2001, International journal of biometeorology.

[40]  Jack J. Lennon,et al.  Red-shifts and red herrings in geographical ecology , 2000 .

[41]  John R. Schott,et al.  Remote Sensing: The Image Chain Approach , 1996 .

[42]  D. Gillespie The mathematics of Brownian motion and Johnson noise , 1996 .

[43]  J E Heath,et al.  Heat exchange by the pinna of the African elephant (Loxodonta africana). , 1992, Comparative biochemistry and physiology. Comparative physiology.

[44]  Manfred Owe,et al.  Measurement and spatial variation of thermal infrared surface emissivity in a savanna environment , 1991 .

[45]  T. Williams Heat transfer in elephants: thermal partitioning based on skin temperature profiles , 1990 .

[46]  D. C. Robertson,et al.  MODTRAN: A Moderate Resolution Model for LOWTRAN , 1987 .

[47]  D. Phinney,et al.  Ecological variations in thermal infrared emissivity of vegetation , 1980 .

[48]  D. Phinney,et al.  The ecological variations in thermal infrared emissivity of vegetation. [in Texas, Arizona, New Mexico, and Mexico] , 1979 .

[49]  J. Hattingh A comparative study of transepidermal water loss through the skin of various animals. , 1972, Comparative biochemistry and physiology. A, Comparative physiology.

[50]  F. G. Benedict,et al.  THE SURFACE TEMPERATURE OF THE ELEPHANT, RHINOCEROS AND HIPPOPOTAMUS , 1921 .

[51]  F. Villalobos,et al.  Soil Temperature and Soil Heat Flux , 2016 .

[52]  Abbreviazioni Periodici Giuridici N. D. I. , 2013 .

[53]  Ali Asghar Ghaemi,et al.  Tree-Temperature Monitoring for Frost Protection of Orchards in Semi-Arid Regions Using Sprinkler Irrigation , 2009 .

[54]  T. Piatkowski,et al.  Analysis of a thermal scene using computer simulations , 2007 .

[55]  Grigg,et al.  Crocodiles as dinosaurs: behavioural thermoregulation in very large ectotherms leads to high and stable body temperatures , 1999, The Journal of experimental biology.

[56]  William Alexander Lentz,et al.  Characterization of noise in uncooled IR bolometer arrays , 1998 .

[57]  Steven W. Running,et al.  A daily soil temperature model based on air temperature and precipitation for continental applications , 1993 .