Neural hierarchical models of ecological populations.

Neural networks are increasingly being used in science to infer hidden dynamics of natural systems from noisy observations, a task typically handled by hierarchical models in ecology. This article describes a class of hierarchical models parameterised by neural networks - neural hierarchical models. The derivation of such models analogises the relationship between regression and neural networks. A case study is developed for a neural dynamic occupancy model of North American bird populations, trained on millions of detection/non-detection time series for hundreds of species, providing insights into colonisation and extinction at a continental scale. Flexible models are increasingly needed that scale to large data and represent ecological processes. Neural hierarchical models satisfy this need, providing a bridge between deep learning and ecological modelling that combines the function representation power of neural networks with the inferential capacity of hierarchical models.

[1]  Christopher K. Wikle,et al.  Comparison of Deep Neural Networks and Deep Hierarchical Models for Spatio-Temporal Data , 2019, Journal of Agricultural, Biological and Environmental Statistics.

[2]  Andrew J. Meade,et al.  The numerical solution of linear ordinary differential equations by feedforward neural networks , 1994 .

[3]  George Cybenko,et al.  Approximation by superpositions of a sigmoidal function , 1992, Math. Control. Signals Syst..

[4]  Julian D. Olden,et al.  Illuminating the “black box”: a randomization approach for understanding variable contributions in artificial neural networks , 2002 .

[5]  J Andrew Royle,et al.  Generalized site occupancy models allowing for false positive and false negative errors. , 2006, Ecology.

[6]  Christopher K. Wikle,et al.  An ensemble quadratic echo state network for non‐linear spatio‐temporal forecasting , 2017, 1708.05094.

[7]  Jiqiang Guo,et al.  Stan: A Probabilistic Programming Language. , 2017, Journal of statistical software.

[8]  H. Preisler,et al.  DEVELOPING PROBABILISTIC MODELS TO PREDICT AMPHIBIAN SITE OCCUPANCY IN A PATCHY LANDSCAPE , 2003 .

[9]  A. Hastings,et al.  Species are not most abundant in the centre of their geographic range or climatic niche. , 2017, Ecology letters.

[10]  M. Austin Spatial prediction of species distribution: an interface between ecological theory and statistical modelling , 2002 .

[11]  Hans J. Skaug,et al.  Spatial factor analysis: a new tool for estimating joint species distributions and correlations in species range , 2015 .

[12]  Leslie Ries,et al.  Joint dynamic species distribution models: a tool for community ordination and spatio‐temporal monitoring , 2016 .

[13]  Toby A Patterson,et al.  Classifying movement behaviour in relation to environmental conditions using hidden Markov models. , 2009, The Journal of animal ecology.

[14]  Roland Langrock,et al.  Flexible and practical modeling of animal telemetry data: hidden Markov models and extensions. , 2012, Ecology.

[15]  Daniel R. Schrider,et al.  The Unreasonable Effectiveness of Convolutional Neural Networks in Population Genetic Inference , 2018, bioRxiv.

[16]  E. N. Zwane,et al.  Semiparametric models for capture-recapture studies with covariates , 2004, Comput. Stat. Data Anal..

[17]  S. Gaines,et al.  The ‘abundant centre’ distribution: to what extent is it a biogeographical rule? , 2002 .

[18]  Andrew J. Viterbi,et al.  Error bounds for convolutional codes and an asymptotically optimum decoding algorithm , 1967, IEEE Trans. Inf. Theory.

[19]  M. Gevrey,et al.  Review and comparison of methods to study the contribution of variables in artificial neural network models , 2003 .

[20]  Alan Hastings,et al.  FITTING POPULATION MODELS INCORPORATING PROCESS NOISE AND OBSERVATION ERROR , 2002 .

[21]  Janet Franklin,et al.  A Convolutional Neural Network Classifier Identifies Tree Species in Mixed-Conifer Forest from Hyperspectral Imagery , 2019, Remote. Sens..

[22]  Carsten F. Dormann,et al.  Cross-validation strategies for data with temporal, spatial, hierarchical, or phylogenetic structure , 2017 .

[23]  Kevin Chen-Chuan Chang,et al.  A Comprehensive Survey of Graph Embedding: Problems, Techniques, and Applications , 2017, IEEE Transactions on Knowledge and Data Engineering.

[24]  Daniel W. Davies,et al.  Machine learning for molecular and materials science , 2018, Nature.

[25]  J. Andrew Royle N‐Mixture Models for Estimating Population Size from Spatially Replicated Counts , 2004, Biometrics.

[26]  Eric Hervet,et al.  Applications for deep learning in ecology , 2018, bioRxiv.

[27]  Francis K. C. Hui,et al.  Joint species distribution models with species correlations and imperfect detection. , 2019, Ecology.

[28]  Zhi-Yong Ran,et al.  Parameter Identifiability in Statistical Machine Learning: A Review , 2017, Neural Computation.

[29]  Francis K. C. Hui,et al.  So Many Variables: Joint Modeling in Community Ecology. , 2015, Trends in ecology & evolution.

[30]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[31]  Margaret Kosmala,et al.  Automatically identifying, counting, and describing wild animals in camera-trap images with deep learning , 2017, Proceedings of the National Academy of Sciences.

[32]  James T Thorson,et al.  Integrating count and detection-nondetection data to model population dynamics. , 2017, Ecology.

[33]  Catherine A Calder,et al.  Accounting for uncertainty in ecological analysis: the strengths and limitations of hierarchical statistical modeling. , 2009, Ecological applications : a publication of the Ecological Society of America.

[34]  Ribana Roscher,et al.  Explainable Machine Learning for Scientific Insights and Discoveries , 2019, IEEE Access.

[35]  Krishna Pacifici,et al.  Integrating multiple data sources in species distribution modeling: a framework for data fusion. , 2017, Ecology.

[36]  Cesare Furlanello,et al.  Phylogenetic convolutional neural networks in metagenomics , 2017, BMC Bioinformatics.

[37]  Joachim Denzler,et al.  Deep learning and process understanding for data-driven Earth system science , 2019, Nature.

[38]  Subhransu Maji,et al.  MistNet: Measuring historical bird migration in the US using archived weather radar data and convolutional neural networks , 2019, Methods in Ecology and Evolution.

[39]  David J. Harris Generating realistic assemblages with a joint species distribution model , 2015 .

[40]  Yoshua Bengio,et al.  Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling , 2014, ArXiv.

[41]  Benjamin Nachman,et al.  Accelerating Science with Generative Adversarial Networks: An Application to 3D Particle Showers in Multilayer Calorimeters. , 2017, Physical review letters.

[42]  Kurt Hornik,et al.  Approximation capabilities of multilayer feedforward networks , 1991, Neural Networks.

[43]  Anuj Karpatne,et al.  Physics-guided Neural Networks (PGNN): An Application in Lake Temperature Modeling , 2017, ArXiv.

[44]  Philippe Desjardins-Proulx,et al.  Artificial Intelligence for Ecological and Evolutionary Synthesis , 2019, Front. Ecol. Evol..

[45]  William A. Link,et al.  The North American Breeding Bird Survey 1966–2011: Summary Analysis and Species Accounts , 2013 .

[46]  J. Andrew Royle,et al.  A Bayesian state-space formulation of dynamic occupancy models. , 2007, Ecology.

[47]  Len Thomas,et al.  Spatial models for distance sampling data: recent developments and future directions , 2013 .

[48]  Expanding the North American Breeding Bird Survey Analysis to Include Additional Species and Regions , 2017 .

[49]  Aki Vehtari,et al.  Understanding predictive information criteria for Bayesian models , 2013, Statistics and Computing.

[50]  Andrew Thomas,et al.  WinBUGS - A Bayesian modelling framework: Concepts, structure, and extensibility , 2000, Stat. Comput..

[51]  Friedrich Recknagel,et al.  Prediction and elucidation of phytoplankton dynamics in the Nakdong River (Korea) by means of a recurrent artificial neural network , 2001 .

[52]  J Andrew Royle,et al.  Models for inference in dynamic metacommunity systems. , 2010, Ecology.

[53]  J. Andrew Royle,et al.  ESTIMATING SITE OCCUPANCY RATES WHEN DETECTION PROBABILITIES ARE LESS THAN ONE , 2002, Ecology.

[54]  G. Jolly EXPLICIT ESTIMATES FROM CAPTURE-RECAPTURE DATA WITH BOTH DEATH AND IMMIGRATION-STOCHASTIC MODEL. , 1965, Biometrika.

[55]  Nick Golding,et al.  Fast and flexible Bayesian species distribution modelling using Gaussian processes , 2016 .

[56]  Pastora Vega,et al.  State space neural network. Properties and application , 1998, Neural Networks.

[57]  Appropriate application of information from biodiversity databases is critical when investigating species distributions and diversity: a comment on Dallas et al. (). , 2018, Ecology letters.

[58]  Jay M Ver Hoef,et al.  A Model‐Based Approach for Making Ecological Inference from Distance Sampling Data , 2010, Biometrics.

[59]  D. Dunson,et al.  Using joint species distribution models for evaluating how species‐to‐species associations depend on the environmental context , 2017 .

[60]  Roland Langrock,et al.  Statistical modelling of individual animal movement: an overview of key methods and a discussion of practical challenges , 2016, AStA Advances in Statistical Analysis.

[61]  Matthew Webb,et al.  Location matters: Using spatially explicit occupancy models to predict the distribution of the highly mobile, endangered swift parrot , 2014 .

[62]  Devin S Johnson,et al.  A General Framework for the Analysis of Animal Resource Selection from Telemetry Data , 2008, Biometrics.

[63]  M. Huijbregts,et al.  Global patterns of current and future road infrastructure , 2018 .

[64]  C. Wikle Hierarchical Models in Environmental Science , 2003 .

[65]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[66]  Mevin B Hooten,et al.  The basis function approach for modeling autocorrelation in ecological data. , 2016, Ecology.

[67]  G. Guillera‐Arroita Modelling of species distributions, range dynamics and communities under imperfect detection: advances, challenges and opportunities , 2017 .

[68]  Christopher K. Wikle,et al.  Deep echo state networks with uncertainty quantification for spatio‐temporal forecasting , 2018, Environmetrics.

[69]  Can Ozan Tan,et al.  Methodological issues in building, training, and testing artificial neural networks in ecological applications , 2005, q-bio/0510017.

[70]  R. Holt,et al.  Alternative causes for range limits: a metapopulation perspective , 2000 .

[71]  Paulo E. Rauber,et al.  Visualizing the Hidden Activity of Artificial Neural Networks , 2017, IEEE Transactions on Visualization and Computer Graphics.

[72]  Yexiang Xue,et al.  Deep Multi-species Embedding , 2016, IJCAI.

[73]  Ping Jiang,et al.  Displacement prediction of landslide based on generalized regression neural networks with K-fold cross-validation , 2016, Neurocomputing.

[74]  Steven R. Hare,et al.  Neural network and fuzzy logic models for pacific halibut recruitment analysis , 2006 .

[75]  J. Oksanen,et al.  Continuum theory revisited: what shape are species responses along ecological gradients? , 2002 .

[76]  David B. Roy,et al.  Uncovering hidden spatial structure in species communities with spatially explicit joint species distribution models , 2016 .

[77]  W. Link,et al.  ESTIMATING POPULATION CHANGE FROM COUNT DATA: APPLICATION TO THE NORTH AMERICAN BREEDING BIRD SURVEY , 1998 .

[78]  Young-Seuk Park,et al.  Patterning and short-term predictions of benthic macroinvertebrate community dynamics by using a recurrent artificial neural network , 2001 .

[79]  Ian D. Jonsen,et al.  A hierarchical Bayesian approach to multi-state mark―recapture: simulations and applications , 2009 .

[80]  J. Nichols,et al.  Dynamic occupancy models for analyzing species' range dynamics across large geographic scales , 2013, Ecology and evolution.

[81]  Rushil Anirudh,et al.  Understanding Deep Neural Networks through Input Uncertainties , 2018, ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[82]  Zenghui Wang,et al.  Deep Convolutional Neural Networks for Image Classification: A Comprehensive Review , 2017, Neural Computation.

[83]  K. Jeong,et al.  Non-linear autoregressive modelling by Temporal Recurrent Neural Networks for the prediction of freshwater phytoplankton dynamics , 2008 .

[84]  Naftali Tishby,et al.  Machine learning and the physical sciences , 2019, Reviews of Modern Physics.

[85]  Silvio Savarese,et al.  Structural-RNN: Deep Learning on Spatio-Temporal Graphs , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[86]  Marie-Josée Fortin,et al.  Species' geographic ranges and distributional limits: pattern analysis and statistical issues , 2005 .

[87]  Anne E Carpenter,et al.  Opportunities and obstacles for deep learning in biology and medicine , 2017, bioRxiv.

[88]  L. Mark Berliner,et al.  Hierarchical Bayesian Time Series Models , 1996 .

[89]  Michael A. Tabak,et al.  Machine learning to classify animal species in camera trap images: applications in ecology , 2018, bioRxiv.

[90]  James D. Nichols,et al.  Local extinction and turnover rates at the edge and interior of species' ranges , 2003 .

[91]  J. Nichols,et al.  ESTIMATING SITE OCCUPANCY, COLONIZATION, AND LOCAL EXTINCTION WHEN A SPECIES IS DETECTED IMPERFECTLY , 2003 .

[92]  William A. Link,et al.  Analysis of the North American Breeding Bird Survey Using Hierarchical Models , 2011 .

[93]  Jeffrey Dean,et al.  Efficient Estimation of Word Representations in Vector Space , 2013, ICLR.

[94]  Dimitrios I. Fotiadis,et al.  Artificial neural networks for solving ordinary and partial differential equations , 1997, IEEE Trans. Neural Networks.

[95]  Brian L. Sullivan,et al.  eBird: A citizen-based bird observation network in the biological sciences , 2009 .

[96]  S. P. Hoogendoorn,et al.  Freeway Travel Time Prediction with State-Space Neural Networks: Modeling State-Space Dynamics with Recurrent Neural Networks , 2002 .

[97]  Stephen E. Fick,et al.  WorldClim 2: new 1‐km spatial resolution climate surfaces for global land areas , 2017 .

[98]  Lyle H. Ungar,et al.  A hybrid neural network‐first principles approach to process modeling , 1992 .

[99]  A M Latimer,et al.  Hierarchical models facilitate spatial analysis of large data sets: a case study on invasive plant species in the northeastern United States. , 2009, Ecology letters.

[100]  J Andrew Royle,et al.  Hierarchical modeling of an invasive spread: the Eurasian Collared-Dove Streptopelia decaocto in the United States. , 2011, Ecological applications : a publication of the Ecological Society of America.

[101]  Wojciech Samek,et al.  Methods for interpreting and understanding deep neural networks , 2017, Digit. Signal Process..

[102]  D. Mehlman CHANGE IN AVIAN ABUNDANCE ACROSS THE GEOGRAPHIC RANGE IN RESPONSE TO ENVIRONMENTAL CHANGE , 1997 .

[103]  A. Salleh,et al.  Applying artificial neural network theory to exploring diatom abundance at tropical Putrajaya Lake, Malaysia , 2012 .

[104]  S. Manel,et al.  Comparing discriminant analysis, neural networks and logistic regression for predicting species distributions: a case study with a Himalayan river bird , 1999 .

[105]  Erin M. Bayne,et al.  REVIEW: Wildlife camera trapping: a review and recommendations for linking surveys to ecological processes , 2015 .

[106]  I. Dimopoulos,et al.  Application of neural networks to modelling nonlinear relationships in ecology , 1996 .

[107]  S. Carpenter,et al.  Ecological forecasts: an emerging imperative. , 2001, Science.

[108]  H. Rue,et al.  Point process models for spatio-temporal distance sampling data , 2016, 1604.06013.

[109]  Kazuhiro Terao,et al.  Machine learning at the energy and intensity frontiers of particle physics , 2018, Nature.