Uncovering Ecological Patterns with Convolutional Neural Networks.

Using remotely sensed imagery to identify biophysical components across landscapes is an important avenue of investigation for ecologists studying ecosystem dynamics. With high-resolution remotely sensed imagery, algorithmic utilization of image context is crucial for accurate identification of biophysical components at large scales. In recent years, convolutional neural networks (CNNs) have become ubiquitous in image processing, and are rapidly becoming more common in ecology. Because the quantity of high-resolution remotely sensed imagery continues to rise, CNNs are increasingly essential tools for large-scale ecosystem analysis. We discuss here the conceptual advantages of CNNs, demonstrate how they can be used by ecologists through distinct examples of their application, and provide a walkthrough of how to use them for ecological applications.

[1]  Ali Farhadi,et al.  YOLO9000: Better, Faster, Stronger , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[2]  J. Kerr,et al.  From space to species: ecological applications for remote sensing , 2003 .

[3]  Michele Volpi,et al.  Deep multi-task learning for a geographically-regularized semantic segmentation of aerial images , 2018, ISPRS Journal of Photogrammetry and Remote Sensing.

[4]  Iasonas Kokkinos,et al.  DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[5]  Z. Bainbridge,et al.  Fine sediment and particulate organic matter: A review and case study on ridge-to-reef transport, transformations, fates, and impacts on marine ecosystems. , 2018, Marine pollution bulletin.

[6]  David C Moyer,et al.  A comparison of deep learning and citizen science techniques for counting wildlife in aerial survey images , 2019, Methods in Ecology and Evolution.

[7]  Thomas Blaschke,et al.  Geographic Object-Based Image Analysis – Towards a new paradigm , 2014, ISPRS journal of photogrammetry and remote sensing : official publication of the International Society for Photogrammetry and Remote Sensing.

[8]  Lin Yang,et al.  Spatial Clockwork Recurrent Neural Network for Muscle Perimysium Segmentation , 2016, MICCAI.

[9]  Nico Karssemeijer,et al.  Non-uniform patch sampling with deep convolutional neural networks for white matter hyperintensity segmentation , 2016, 2016 IEEE 13th International Symposium on Biomedical Imaging (ISBI).

[10]  A. Huth,et al.  High resolution analysis of tropical forest fragmentation and its impact on the global carbon cycle , 2017, Nature Communications.

[11]  Patrick Mäder,et al.  Machine learning for image based species identification , 2018, Methods in Ecology and Evolution.

[12]  P. Mumby,et al.  Global disparity in the resilience of coral reefs. , 2012, Trends in ecology & evolution.

[13]  R. Primack,et al.  Long-term carbon sink in Borneo’s forests halted by drought and vulnerable to edge effects , 2017, Nature Communications.

[14]  Stephen P. Hubbell,et al.  Density-dependent adult recruitment in a low-density tropical tree , 2018, Proceedings of the National Academy of Sciences.

[15]  O. Phillips,et al.  Using the U‐net convolutional network to map forest types and disturbance in the Atlantic rainforest with very high resolution images , 2019, Remote Sensing in Ecology and Conservation.

[16]  Weijia Li,et al.  Deep Learning Based Oil Palm Tree Detection and Counting for High-Resolution Remote Sensing Images , 2016, Remote. Sens..

[17]  Christopher B Anderson,et al.  Biodiversity monitoring, earth observations and the ecology of scale. , 2018, Ecology letters.

[18]  Jiri Matas,et al.  Systematic evaluation of convolution neural network advances on the Imagenet , 2017, Comput. Vis. Image Underst..

[19]  Michele Volpi,et al.  Land cover mapping at very high resolution with rotation equivariant CNNs: towards small yet accurate models , 2018, ISPRS Journal of Photogrammetry and Remote Sensing.

[20]  Ellen I. Damschen,et al.  Habitat fragmentation and its lasting impact on Earth’s ecosystems , 2015, Science Advances.

[21]  Ross B. Girshick,et al.  Fast R-CNN , 2015, 1504.08083.

[22]  Iasonas Kokkinos,et al.  Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs , 2014, ICLR.

[23]  J Elith,et al.  A working guide to boosted regression trees. , 2008, The Journal of animal ecology.

[24]  Maggi Kelly,et al.  Identification of Citrus Trees from Unmanned Aerial Vehicle Imagery Using Convolutional Neural Networks , 2018, Drones.

[25]  Ross B. Girshick,et al.  Mask R-CNN , 2017, 1703.06870.

[26]  Gregory P. Asner,et al.  Spatial variability and abiotic determinants of termite mounds throughout a savanna catchment , 2014 .

[27]  Ben. G. Weinstein.  A computer vision for animal ecology. , 2018, The Journal of animal ecology.

[28]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[29]  Travis Desell,et al.  Detecting wildlife in uncontrolled outdoor video using convolutional neural networks , 2016, 2016 IEEE 12th International Conference on e-Science (e-Science).

[30]  Bernhard Seeger,et al.  Taxon and trait recognition from digitized herbarium specimens using deep convolutional neural networks , 2018, 1803.07892.

[31]  Sarah Sim,et al.  Degradation in carbon stocks near tropical forest edges , 2015, Nature Communications.

[32]  Sarah Webb.  Deep learning for biology , 2018, Nature.

[33]  Daniel J. Hayes,et al.  The Use of Three-Dimensional Convolutional Neural Networks to Interpret LiDAR for Forest Inventory , 2018, Remote. Sens..

[34]  Kannappan Palaniappan,et al.  Malaria parasite detection and cell counting for human and mouse using thin blood smear microscopy , 2018, Journal of medical imaging.

[35]  Ola Spjuth,et al.  Deep Learning in Image Cytometry: A Review , 2018, Cytometry. Part A : the journal of the International Society for Analytical Cytology.

[36]  Roberta E. Martin,et al.  Mapped aboveground carbon stocks to advance forest conservation and recovery in Malaysian Borneo , 2018 .

[37]  M. Keller,et al.  Forest fragmentation and edge effects from deforestation and selective logging in the Brazilian Amazon , 2008 .

[38]  Roberta E. Martin,et al.  Carnegie Airborne Observatory-2: Increasing science data dimensionality via high-fidelity multi-sensor fusion , 2012 .

[39]  Jian Sun,et al.  Identity Mappings in Deep Residual Networks , 2016, ECCV.

[40]  Wei Liu,et al.  SSD: Single Shot MultiBox Detector , 2015, ECCV.

[41]  Gregory P. Asner,et al.  Termite mounds alter the spatial distribution of African savanna tree species , 2016 .

[42]  Luc Van Gool,et al.  The Pascal Visual Object Classes Challenge: A Retrospective , 2014, International Journal of Computer Vision.

[43]  Aditya Singh,et al.  Mapping Species Composition of Forests and Tree Plantations in Northeastern Costa Rica with an Integration of Hyperspectral and Multitemporal Landsat Imagery , 2015, Remote. Sens..

[44]  Suchendra M. Bhandarkar,et al.  A Comparison of Deep Learning Methods for Semantic Segmentation of Coral Reef Survey Images , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[45]  Daniel Cohen-Or,et al.  Patch2Vec: Globally Consistent Image Patch Representation , 2017, Comput. Graph. Forum.

[46]  Ali Farhadi,et al.  You Only Look Once: Unified, Real-Time Object Detection , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[47]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[48]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[49]  J. Townshend,et al.  NDVI-derived land cover classifications at a global scale , 1994 .

[50]  Marc'Aurelio Ranzato,et al.  Building high-level features using large scale unsupervised learning , 2011, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[51]  Vladlen Koltun,et al.  Efficient Inference in Fully Connected CRFs with Gaussian Edge Potentials , 2011, NIPS.

[52]  Danny Ziyi Chen,et al.  A Deep Learning Approach for Semantic Segmentation in Histology Tissue Images , 2016, MICCAI.

[53]  Pierre Alliez,et al.  Convolutional Neural Networks for Large-Scale Remote-Sensing Image Classification , 2017, IEEE Transactions on Geoscience and Remote Sensing.

[54]  Marco Willi,et al.  Identifying animal species in camera trap images using deep learning and citizen science , 2018, Methods in Ecology and Evolution.

[55]  Jordi Pont-Tuset,et al.  Convolutional Oriented Boundaries: From Image Segmentation to High-Level Tasks , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[56]  Jonathan T. Barron,et al.  Multiscale Combinatorial Grouping for Image Segmentation and Object Proposal Generation , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[57]  Ovidiu Csillik,et al.  Fast Segmentation and Classification of Very High Resolution Remote Sensing Data Using SLIC Superpixels , 2017, Remote. Sens..

[58]  Vladlen Koltun,et al.  Multi-Scale Context Aggregation by Dilated Convolutions , 2015, ICLR.

[59]  David J. Klein,et al.  A convolutional neural network for detecting sea turtles in drone imagery , 2018, Methods in Ecology and Evolution.

[60]  Thomas Blaschke,et al.  Object based image analysis for remote sensing , 2010 .

[61]  Thomas Brox,et al.  U-Net: deep learning for cell counting, detection, and morphometry , 2018, Nature Methods.

[62]  Tarin Paz-Kagan,et al.  What mediates tree mortality during drought in the southern Sierra Nevada? , 2017, Ecological applications : a publication of the Ecological Society of America.

[63]  Corina E Tarnita,et al.  Spatial Self-Organization of Ecosystems: Integrating Multiple Mechanisms of Regular-Pattern Formation. , 2017, Annual review of entomology.

[64]  Changshui Zhang,et al.  DeepFish: Accurate underwater live fish recognition with a deep architecture , 2016, Neurocomputing.

[65]  Takeshi Naemura,et al.  Detection of small birds in large images by combining a deep detector with semantic segmentation , 2016, 2016 IEEE International Conference on Image Processing (ICIP).

[66]  Geoffrey E. Hinton,et al.  Machine Learning for Aerial Image Labeling , 2013 .

[67]  Roberto Cipolla,et al.  SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[68]  Gregory Asner,et al.  An Approach for High-Resolution Mapping of Hawaiian Metrosideros Forest Mortality Using Laser-Guided Imaging Spectroscopy , 2018, Remote. Sens..

[69]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[70]  Ty Kennedy-Bowdoin,et al.  Regional insight into savanna hydrogeomorphology from termite mounds. , 2010, Nature communications.

[71]  Chien-Hung Chen,et al.  Stingray detection of aerial images with region-based convolution neural network , 2017, 2017 IEEE International Conference on Consumer Electronics - Taiwan (ICCE-TW).

[72]  Yun Zhang,et al.  Deep Convolutional Neural Network for Complex Wetland Classification Using Optical Remote Sensing Imagery , 2018, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[73]  Xiao Xiang Zhu,et al.  Deep Learning in Remote Sensing: A Comprehensive Review and List of Resources , 2017, IEEE Geoscience and Remote Sensing Magazine.

[74]  German Díez,et al.  Animal Identification in Low Quality Camera-Trap Images Using Very Deep Convolutional Neural Networks and Confidence Thresholds , 2016, ISVC.

[75]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[76]  Robert P. Anderson,et al.  Maximum entropy modeling of species geographic distributions , 2006 .

[77]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[78]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[79]  D. Doak,et al.  Abstracts, Reviews, and Meetings , 2011, Ecological Restoration.

[80]  Scott F. Heron,et al.  Risk‐sensitive planning for conserving coral reefs under rapid climate change , 2018, Conservation Letters.

[81]  Gudeta W. Sileshi,et al.  Termite-induced heterogeneity in African savanna vegetation: mechanisms and patterns , 2010 .

[82]  Trevor Darrell,et al.  Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation , 2013, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[83]  Xiaogang Wang,et al.  Pyramid Scene Parsing Network , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[84]  Anne E Carpenter,et al.  Applying Faster R-CNN for Object Detection on Malaria Images , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[85]  O. Csillik,et al.  Automated parameterisation for multi-scale image segmentation on multiple layers , 2014, ISPRS journal of photogrammetry and remote sensing : official publication of the International Society for Photogrammetry and Remote Sensing.

[86]  S. Running,et al.  What does Remote Sensing Do for Ecology , 1991 .

[87]  Paolo Remagnino,et al.  Deep-plant: Plant identification with convolutional neural networks , 2015, 2015 IEEE International Conference on Image Processing (ICIP).

[88]  K. Fabricius.  Effects of terrestrial runoff on the ecology of corals and coral reefs: review and synthesis. , 2005, Marine pollution bulletin.

[89]  Thomas A. Funkhouser,et al.  Dilated Residual Networks , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[90]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[91]  Devis Tuia,et al.  Detecting Mammals in UAV Images: Best Practices to address a substantially Imbalanced Dataset with Deep Learning , 2018, Remote Sensing of Environment.