Applications for deep learning in ecology

A lot of hype has recently been generated around deep learning, a novel group of artificial intelligence approaches able to break accuracy records in pattern recognition. Over the course of just a few years, deep learning has revolutionized several research fields such as bioinformatics and medicine with its flexibility and ability to process large and complex datasets. As ecological datasets are becoming larger and more complex, we believe these methods can be useful to ecologists as well. In this paper, we review existing implementations and show that deep learning has been used successfully to identify species, classify animal behaviour and estimate biodiversity in large datasets like camera‐trap images, audio recordings and videos. We demonstrate that deep learning can be beneficial to most ecological disciplines, including applied contexts, such as management and conservation. We also identify common questions about how and when to use deep learning, such as what are the steps required to create a deep learning network, which tools are available to help, and what are the requirements in terms of data and computer power. We provide guidelines, recommendations and useful resources, including a reference flowchart to help ecologists get started with deep learning. We argue that at a time when automatic monitoring of populations and ecosystems generates a vast amount of data that cannot be effectively processed by humans anymore, deep learning could become a powerful reference tool for ecologists.

[1]  Juan Manuel Górriz,et al.  Deep Learning in Medical Image Analysis , 2021, J. Imaging.

[2]  Nicholas G. Polson,et al.  Deep learning for finance: deep portfolios: J. B. HEATON, N. G. POLSON AND J. H. WITTE , 2017 .

[3]  Mohammad Najafi,et al.  Deep phenotyping: deep learning for temporal phenotype/genotype classification , 2017, Plant Methods.

[4]  Avelino Javer,et al.  Recurrent Neural Networks with Interpretable Cells Predict and Classify Worm Behaviour , 2017, NIPS 2017.

[5]  Jesús Francisco Vargas-Bonilla,et al.  Towards automatic wild animal monitoring: Identification of animal species in camera-trap images using very deep convolutional neural networks , 2016, Ecol. Informatics.

[6]  Jürgen Schmidhuber,et al.  Deep learning in neural networks: An overview , 2014, Neural Networks.

[7]  D. R. Cutler,et al.  Utah State University From the SelectedWorks of , 2017 .

[8]  Hervé Glotin,et al.  Audio Bird Classification with Inception-v4 extended with Time and Time-Frequency Attention Mechanisms , 2017, CLEF.

[9]  Sidarta Ribeiro,et al.  Head and gaze tracking of unrestrained marmosets , 2016, bioRxiv.

[10]  Hervé Glotin,et al.  Overview of LifeCLEF 2018: A Large-Scale Evaluation of Species Identification and Recommendation Algorithms in the Era of AI , 2018, CLEF.

[11]  C. Lintott,et al.  Snapshot Serengeti, high-frequency annotated camera trap images of 40 mammalian species in an African savanna , 2015, Scientific Data.

[12]  Yunpeng Li,et al.  Bioacoustic detection with wavelet-conditioned convolutional neural networks , 2018, Neural Computing and Applications.

[13]  Christoph Fink,et al.  Machine learning for tracking illegal wildlife trade on social media , 2018, Nature Ecology & Evolution.

[14]  Julian D Olden,et al.  Machine Learning Methods Without Tears: A Primer for Ecologists , 2008, The Quarterly Review of Biology.

[15]  Ella Browning,et al.  Predicting animal behaviour using deep learning: GPS data alone accurately predict diving in seabirds , 2017 .

[16]  Mark A. Girolami,et al.  Bat detective—Deep learning tools for bat acoustic signal detection , 2017, bioRxiv.

[17]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[18]  Tara N. Sainath,et al.  Deep Neural Networks for Acoustic Modeling in Speech Recognition: The Shared Views of Four Research Groups , 2012, IEEE Signal Processing Magazine.

[19]  Marc Chaumont,et al.  A Deep learning method for accurate and fast identification of coral reef fishes in underwater images , 2018, Ecol. Informatics.

[20]  Oliver R. Wearn,et al.  Responsible AI for conservation , 2019, Nature Machine Intelligence.

[21]  David J. Klein,et al.  A convolutional neural network for detecting sea turtles in drone imagery , 2018, Methods in Ecology and Evolution.

[22]  Mikhail Kislin,et al.  Fast animal pose estimation using deep neural networks , 2018, Nature Methods.

[23]  Steve Kelling,et al.  Fusing shallow and deep learning for bioacoustic bird species classification , 2017, 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[24]  Erin M. Bayne,et al.  Recommendations for acoustic recognizer performance assessment with application to five common automated signal recognition programs , 2017 .

[25]  Barbara A. Block,et al.  Tracking the global footprint of fisheries , 2018, Science.

[26]  Friedrich Recknagel,et al.  Applications of machine learning to ecological modelling , 2001 .

[27]  Christopher W. Clark,et al.  Phase 2: DCL System Using Deep Learning Approaches for Land-based or Ship-based Real-Time Recognition and Localization of Marine Mammals - Machine Learning Detection Algorithms , 2016, ArXiv.

[28]  I. Dimopoulos,et al.  Application of neural networks to modelling nonlinear relationships in ecology , 1996 .

[29]  J. Drake,et al.  Modelling ecological niches with support vector machines , 2006 .

[30]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[31]  Michael A. Tabak,et al.  Machine learning to classify animal species in camera trap images: applications in ecology , 2018, bioRxiv.

[32]  Sotirios A. Tsaftaris,et al.  Leveraging multiple datasets for deep leaf counting , 2017, bioRxiv.

[33]  François Chollet,et al.  Xception: Deep Learning with Depthwise Separable Convolutions , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[34]  Friedrich Recknagel,et al.  Prediction and elucidation of phytoplankton dynamics in the Nakdong River (Korea) by means of a recurrent artificial neural network , 2001 .

[35]  Gary Marcus,et al.  Deep Learning: A Critical Appraisal , 2018, ArXiv.

[36]  Peter Arzberger,et al.  Enhancing collaboration between ecologists and computer scientists: lessons learned and recommendations forward , 2019, Ecosphere.

[37]  Magnus Enquist,et al.  The evolution of courtship rituals in monogamous species , 2000 .

[38]  Sovan Lek,et al.  Artificial Neuronal Networks: Application To Ecology And Evolution , 2012 .

[39]  Ursula Kälin,et al.  Defoliation estimation of forest trees from ground-level images , 2019 .

[40]  Eric Hervet,et al.  Applications for deep learning in ecology , 2018, bioRxiv.

[41]  Heung-Il Suk,et al.  Deep Learning in Medical Image Analysis. , 2017, Annual review of biomedical engineering.

[42]  Michael J. Ryan,et al.  Signal Decoding and Receiver Evolution , 2000, Brain, Behavior and Evolution.

[43]  Young-Seuk Park,et al.  Patterning and short-term predictions of benthic macroinvertebrate community dynamics by using a recurrent artificial neural network , 2001 .

[44]  Margaret Kosmala,et al.  Automatically identifying, counting, and describing wild animals in camera-trap images with deep learning , 2017, Proceedings of the National Academy of Sciences.

[45]  Ben C. Stöver,et al.  LeafNet: A computer vision system for automatic plant species identification , 2017, Ecol. Informatics.

[46]  Bakhtyar Ahmed Mohammed,et al.  using Deep Learning , 2020 .

[47]  Ning Jiang,et al.  Our path to better science in less time using open data science tools , 2017, Nature Ecology &Evolution.

[48]  Francisco Herrera,et al.  Automatic whale counting in satellite images with deep learning , 2018, bioRxiv.

[49]  Erle C. Ellis.  Ecology in an anthropogenic biosphere , 2015 .

[50]  Leon Sixt,et al.  Automatic localization and decoding of honeybee markers using deep convolutional neural networks , 2018, ArXiv.

[51]  David N. Bonter,et al.  Citizen Science as an Ecological Research Tool: Challenges and Benefits , 2010 .

[52]  John Joseph Valletta,et al.  Applications of machine learning in animal behaviour studies , 2017, Animal Behaviour.

[53]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[54]  Yexiang Xue,et al.  Deep Multi-species Embedding , 2016, IJCAI.

[55]  Yu Jiang,et al.  Aerial Images and Convolutional Neural Network for Cotton Bloom Detection , 2018, Front. Plant Sci..

[56]  Bernhard Seeger,et al.  Taxon and trait recognition from digitized herbarium specimens using deep convolutional neural networks , 2018, 1803.07892.

[57]  Hanno Scharr,et al.  ARIGAN: Synthetic Arabidopsis Plants using Generative Adversarial Network , 2017, bioRxiv.

[58]  Pietro Perona,et al.  Mouse Academy: high-throughput automated training and trial-by-trial behavioral analysis during learning , 2018, bioRxiv.

[59]  Paolo Manghi,et al.  Data journals: A survey , 2014, J. Assoc. Inf. Sci. Technol..

[60]  Patrick Mäder,et al.  Machine learning for image based species identification , 2018, Methods in Ecology and Evolution.

[61]  Byunghan Lee,et al.  Deep learning in bioinformatics , 2016, Briefings Bioinform..

[62]  Quoc V. Le,et al.  Sequence to Sequence Learning with Neural Networks , 2014, NIPS.

[63]  Graham W. Taylor,et al.  Deep Learning Object Detection Methods for Ecological Camera Trap Data , 2018, 2018 15th Conference on Computer and Robot Vision (CRV).

[64]  Patrick Mäder,et al.  Acquiring and preprocessing leaf images for automated plant identification: understanding the tradeoff between effort and information gain , 2017, Plant Methods.

[65]  Ilyas Potamitis,et al.  Unsupervised dictionary extraction of bird vocalisations and new tools on assessing and visualising bird activity , 2015, Ecol. Informatics.

[66]  Mei-rong Zhao,et al.  An evolutionary deep neural network for predicting morbidity of gastrointestinal infections by food contamination , 2017, Neurocomputing.

[67]  Philippe Desjardins-Proulx,et al.  Ecological interactions and the Netflix problem , 2016, bioRxiv.

[68]  Erle C. Ellis,et al.  Designing Autonomy: Opportunities for New Wildness in the Anthropocene. , 2017, Trends in ecology & evolution.

[69]  Anne E. Goodenough,et al.  Testing the potential of Twitter mining methods for data acquisition: Evaluating novel opportunities for ecological research in multiple taxa , 2018, Methods in Ecology and Evolution.

[70]  Guillaume Lample,et al.  Playing FPS Games with Deep Reinforcement Learning , 2016, AAAI.

[71]  Marcel Salathé,et al.  Using Deep Learning for Image-Based Plant Disease Detection , 2016, Front. Plant Sci..

[72]  Paul D. Meek,et al.  "Which camera trap type and how many do I need?" A review of camera features and study designs for a range of wildlife research applications , 2013 .

[73]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[74]  Roberta E. Martin,et al.  Mapped aboveground carbon stocks to advance forest conservation and recovery in Malaysian Borneo , 2018 .

[75]  Anirban Mukhopadhyay,et al.  Habitat-Net: Segmentation of habitat images using deep learning , 2018, bioRxiv.

[76]  Trevor Darrell,et al.  Quantification in-the-wild: data-sets and baselines , 2015, ArXiv.

[77]  Lex Nederbragt,et al.  Good enough practices in scientific computing , 2016, PLoS Comput. Biol..

[78]  Kate E. Jones,et al.  CityNet—Deep learning tools for urban ecoacoustic assessment , 2018, Methods in Ecology and Evolution.

[79]  Thomas Koellner,et al.  Mapping cultural ecosystem services 2.0 – Potential and shortcomings from unlabeled crowd sourced images , 2019, Ecological Indicators.

[80]  Jürgen Schmidhuber,et al.  An Application of Recurrent Neural Networks to Discriminative Keyword Spotting , 2007, ICANN.

[81]  David R. B. Stockwell,et al.  Induction of sets of rules from animal distribution data: a robust and informative method of data analysis , 1992 .