Multi-Object Tracking in Heterogeneous environments (MOTHe) for animal video recordings

Video recordings of animals are used for many areas of research such as collective movement, animal space-use, animal censuses and behavioural neuroscience. They provide us with behavioural data at scales and resolutions not possible with manual observations. Many automated methods are being developed to extract data from these high-resolution videos. However, the task of animal detection and tracking for videos taken in natural settings remains challenging due to heterogeneous environments. We present an open-source end-to-end pipeline called Multi-Object Tracking in Heterogenous environments (MOTHe), a python-based application that uses a basic convolutional neural network for object detection. MOTHe allows researchers with minimal coding experience to track multiple animals in their natural habitats. It identifies animals even when individuals are stationary or partially camouflaged. MOTHe has a command-line-based interface with one command for each action, for example, finding animals in an image and tracking each individual. Parameters used by the algorithm are well described in a configuration file along with example values for different types of tracking scenario. MOTHe doesn’t require any sophisticated infrastructure and can be run on basic desktop computing units. We demonstrate MOTHe on six video clips from two species in their natural habitat - wasp colonies on their nests (up to 12 individuals per colony) and antelope herds in four different types of habitats (up to 156 individuals in a herd). Using MOTHe, we are able to detect and track all individuals in these animal group videos. MOTHe’s computing time on a personal computer with 4 GB RAM and i5 processor is 5 minutes for a 30-second long ultra-HD (4K resolution) video recorded at 30 frames per second. MOTHe is available as an open-source repository with a detailed user guide and demonstrations at Github (https://github.com/tee-lab/MOTHe).

[1]  A. Zaslaver,et al.  A multi-animal tracker for studying complex behaviors , 2017, BMC Biology.

[2]  Lian Pin Koh,et al.  Drones count wildlife more accurately and precisely than humans , 2017, bioRxiv.

[3]  R. Gadagkar,et al.  A place for everything and everything in its place: spatial organization of individuals on nests of the primitively eusocial wasp Ropalidia marginata , 2019, Proceedings of the Royal Society B.

[4]  Dumitru Erhan,et al.  Deep Neural Networks for Object Detection , 2013, NIPS.

[5]  L. Edelstein-Keshet,et al.  Complexity, pattern, and evolutionary trade-offs in animal aggregation. , 1999, Science.

[6]  Mehdi Madani,et al.  Real time object detection using a novel adaptive color thresholding method , 2011, Ubi-MUI '11.

[7]  Osamu Yamanaka,et al.  UMATracker: an intuitive image-based tracking platform , 2018, Journal of Experimental Biology.

[8]  Magnus Andersson,et al.  ToxTrac: A fast and robust software for tracking organisms , 2017, ArXiv.

[9]  David C Moyer,et al.  A comparison of deep learning and citizen science techniques for counting wildlife in aerial survey images , 2019, Methods in Ecology and Evolution.

[10]  Margaret Kosmala,et al.  Automatically identifying, counting, and describing wild animals in camera-trap images with deep learning , 2017, Proceedings of the National Academy of Sciences.

[11]  Ali Farhadi,et al.  XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks , 2016, ECCV.

[12]  D. Bird,et al.  Population Census of a Large Common Tern Colony with a Small Unmanned Aircraft , 2015, PloS one.

[13]  K. Isvaran.  Intraspecific variation in group size in the blackbuck antelope: the roles of habitat structure and forage at different spatial scales , 2007, Oecologia.

[14]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[15]  A. Crespi,et al.  Tracking Individuals Shows Spatial Fidelity Is a Key Regulator of Ant Social Organization , 2013, Science.

[16]  Xiaoyi Jiang,et al.  FIMTrack: An open source tracking and locomotion analysis software for small animals , 2017, PLoS Comput. Biol..

[17]  K. Isvaran.  Female grouping best predicts lekking in blackbuck (Antilope cervicapra) , 2004, Behavioral Ecology and Sociobiology.

[18]  Tim Landgraf,et al.  BioTracker: An Open-Source Computer Vision Framework for Visual Animal Tracking , 2018, ArXiv.

[19]  I. Couzin,et al.  Inferring the structure and dynamics of interactions in schooling fish , 2011, Proceedings of the National Academy of Sciences.

[20]  Massimo Piccardi,et al.  Background subtraction techniques: a review , 2004, 2004 IEEE International Conference on Systems, Man and Cybernetics (IEEE Cat. No.04CH37583).

[21]  Ross B. Girshick,et al.  Fast R-CNN , 2015, 1504.08083.

[22]  Raghavendra Gadagkar,et al.  Quantitative ethology of social wasps: Time-activity budgets and caste differentiation in Ropalidia marginata (Lep.) (Hymenoptera: Vespidae) , 1983, Animal Behaviour.

[23]  G. Parisi,et al.  Interaction ruling animal collective behavior depends on topological rather than metric distance: Evidence from a field study , 2007, Proceedings of the National Academy of Sciences.

[24]  Colin J Torney,et al.  Inferring the rules of social interaction in migrating caribou , 2018, Philosophical Transactions of the Royal Society B: Biological Sciences.

[25]  A. Pérez-Escudero,et al.  idTracker: tracking individuals in a group by automatic identification of unmarked animals , 2014, Nature Methods.

[26]  V. Guttal,et al.  Noise-induced schooling of fish , 2019, Nature Physics.

[27]  Ali Farhadi,et al.  YOLOv3: An Incremental Improvement , 2018, ArXiv.

[28]  R. Kays,et al.  Terrestrial animal tracking as an eye on life and planet , 2015, Science.

[29]  Stephen Gould,et al.  Region-based Segmentation and Object Detection , 2009, NIPS.

[30]  Sandra Johnson,et al.  Unmanned Aerial Vehicles (UAVs) and Artificial Intelligence Revolutionizing Wildlife Monitoring and Conservation , 2016, Sensors.

[31]  Damien R. Farine,et al.  Both Nearest Neighbours and Long-term Affiliates Predict Individual Locations During Collective Movement in Wild Baboons , 2016, Scientific Reports.

[32]  Vivek Hari Sridhar,et al.  Tracktor: image-based automated tracking of animal movement and behaviour , 2018, bioRxiv.

[33]  Pietro Perona,et al.  Automated image-based tracking and its application in ecology. , 2014, Trends in ecology & evolution.

[34]  Iain D Couzin,et al.  From local collective behavior to global migratory patterns in white storks , 2018, Science.

[35]  Travis Desell,et al.  Detecting wildlife in uncontrolled outdoor video using convolutional neural networks , 2016, 2016 IEEE 12th International Conference on e-Science (e-Science).

[36]  Ali Farhadi,et al.  You Only Look Once: Unified, Real-Time Object Detection , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[37]  Jacob M. Graving,et al.  DeepPoseKit, a software toolkit for fast and robust animal pose estimation using deep learning , 2019, bioRxiv.