Assessing the reliability of species distribution projections in climate change research

Aim Forecasting changes in species distribution under future scenarios is one of the most prolific areas of application for species distribution models (SDMs). However, no consensus yet exists on the reliability of such models for drawing conclusions on species distribution response to changing climate. In this study we provide an overview of common modelling practices in the field and assess model predictions reliability using a virtual species approach. Location Global Methods We first provide an overview of common modelling practices in the field by reviewing the papers published in the last 5 years. Then, we use a virtual species approach and three commonly applied SDM algorithms (GLM, MaxEnt and Random Forest) to assess the estimated (cross-validated) and actual predictive performance of models parameterized with different modelling settings and violations of modelling assumptions. Results Our literature review shows that most papers that model species distribution under climate change rely on single models (65%) and small samples (< 50 presence points, 62%), use presence-only data (85%), and binarize models’ output to estimate range shift, contraction or expansion (74%). Our virtual species approach reveals that the estimated predictive performance tends to be over-optimistic compared to the real predictive performance. Further, the binarization of predicted probabilities of presence reduces models’ predictive ability considerably. Sample size is one of the main predictors of real accuracy, but has little influence on estimated accuracy. Finally, the inclusion of irrelevant predictors and the violation of modelling assumptions increases estimated accuracy but decreases real accuracy of model projections, leading to biased estimates of range contraction and expansion. Main conclusions Our study calls for extreme caution in the application and interpretation of SDMs in the context of biodiversity conservation and climate change research, especially when modelling a large number of species where species-specific model settings become impracticable.

[1]  Miroslav Dudík,et al.  Modeling of species distributions with Maxent: new extensions and a comprehensive evaluation , 2008 .

[2]  L. Santini,et al.  Human pressures predict species’ geographic range size better than biological traits , 2015, Global change biology.

[3]  Robert P. Anderson,et al.  The effect of the extent of the study region on GIS models of species geographic distributions and estimates of niche evolution: preliminary tests with montane rodents (genus Nephelomys) in Venezuela , 2010 .

[4]  G. Moisen,et al.  PresenceAbsence: An R Package for Presence Absence Analysis , 2008 .

[5]  Brenna R. Forester,et al.  Considering adaptive genetic variation in climate change vulnerability assessment reduces species range loss projections , 2019, Proceedings of the National Academy of Sciences.

[6]  David B. Dunson,et al.  A comprehensive evaluation of predictive performance of 33 species distribution models at species and community levels , 2019, Ecological Monographs.

[7]  J. Brashares,et al.  The effects of small sample size and sample bias on threshold selection and accuracy assessment of species distribution models , 2012 .

[8]  Damaris Zurell,et al.  The virtual ecologist approach: simulating data and observers , 2010 .

[9]  Céline Bellard,et al.  Without quality presence–absence data, discrimination metrics such as TSS can be misleading measures of model performance , 2018, Journal of Biogeography.

[10]  Kerrie Mengersen,et al.  Transferring biodiversity models for conservation: Opportunities and challenges , 2018 .

[11]  D. Nogues‐Bravo,et al.  Applications of species distribution modeling to paleobiology , 2011 .

[12]  Jane Elith,et al.  What do we gain from simplicity versus complexity in species distribution models , 2014 .

[13]  Olaf Conrad,et al.  Climatologies at high resolution for the earth’s land surface areas , 2016, Scientific Data.

[14]  David I. Warton,et al.  Model-Based Control of Observer Bias for the Analysis of Presence-Only Data in Ecology , 2013, PloS one.

[15]  C. Dormann.  Promising the future? Global change projections of species distributions , 2007 .

[16]  A. Townsend Peterson,et al.  The influence of spatial errors in species occurrence data used in distribution models , 2007 .

[17]  S. Richards,et al.  Prevalence, thresholds and the performance of presence–absence models , 2014 .

[18]  Andy Liaw,et al.  Classification and Regression by randomForest , 2007 .

[19]  Alain F. Zuur,et al.  A protocol for data exploration to avoid common statistical problems , 2010 .

[20]  Carrie A. Schloss,et al.  Dispersal will limit ability of mammals to track climate change in the Western Hemisphere , 2012, Proceedings of the National Academy of Sciences.

[21]  A. Peterson,et al.  Effects of sample size on the performance of species distribution models , 2008 .

[22]  M. Araújo,et al.  Anthropogenic range contractions bias species climate change forecasts , 2018, Nature Climate Change.

[23]  A. Townsend Peterson,et al.  Constraints on interpretation of ecological niche models by limited environmental ranges on calibration areas , 2012 .

[24]  Jorge Soberón,et al.  Creating individual accessible area hypotheses improves stacked species distribution model performance , 2018 .

[25]  C. Bellard,et al.  virtualspecies, an R package to generate virtual species distributions , 2016 .

[26]  Jason Lowe,et al.  Quantifying the benefit of early climate change mitigation in avoiding biodiversity loss , 2013 .

[27]  M. Araújo,et al.  Validation of species–climate impact models under climate change , 2005 .

[28]  M. Araújo,et al.  Equilibrium of species’ distributions with climate , 2005 .

[29]  Steven J. Phillips,et al.  The art of modelling range‐shifting species , 2010 .

[30]  C. Graham,et al.  Within‐taxon niche structure: niche conservatism, divergence and predicted effects of climate change , 2010 .

[31]  L. Maiorano,et al.  Performance tradeoffs in target‐group bias correction for species distribution models , 2017 .

[32]  M. Pandit,et al.  Identifying conservation priorities for plant species in the Himalaya in current and future climates: A case study from Sikkim Himalaya, India , 2019, Biological Conservation.

[33]  Jian Zhang,et al.  Assessing the effectiveness of China's protected areas to conserve current and future amphibian diversity , 2017 .

[34]  Omri Allouche,et al.  Assessing the accuracy of species distribution models: prevalence, kappa and the true skill statistic (TSS) , 2006 .

[35]  Jeremy VanDerWal,et al.  Abundance and the Environmental Niche: Environmental Suitability Estimated from Niche Models Predicts the Upper Limit of Local Abundance , 2009, The American Naturalist.

[36]  R. Halvorsen,et al.  Bunching up the background betters bias in species distribution models , 2019, Ecography.

[37]  A. Nicotra,et al.  The effects of phenotypic plasticity and local adaptation on forecasts of species range shifts under climate change. , 2014, Ecology letters.

[38]  R Core Team,et al.  R: A language and environment for statistical computing. , 2014 .

[39]  M. Austin,et al.  Improving species distribution models for climate change studies: variable selection and scale , 2011 .

[40]  Brendan A. Wintle,et al.  Evaluating 318 continental-scale species distribution models over a 60-year prediction horizon: what factors influence the reliability of predictions? , 2017 .

[41]  Colin W. Rundel,et al.  Interface to Geometry Engine - Open Source (GEOS) , 2015 .

[42]  Robert P. Anderson,et al.  Maximum entropy modeling of species geographic distributions , 2006 .

[43]  Antoine Guisan,et al.  Building the niche through time: using 13,000 years of data to predict the effects of climate change on three tree species in Europe , 2013 .

[44]  Jaime Hernández,et al.  Projected distribution shifts and protected area coverage of range-restricted Andean birds under climate change , 2015 .

[45]  Mathieu Marmion,et al.  Does the interpolation accuracy of species distribution models come at the expense of transferability , 2012 .

[46]  D. Nogues‐Bravo,et al.  Predicting the past distribution of species climatic niches. , 2009 .

[47]  J. Andrew Royle,et al.  Presence‐only modelling using MAXENT: when can we trust the inferences? , 2013 .

[48]  David R. B. Stockwell,et al.  Effects of sample size on accuracy of species distribution models , 2002 .

[49]  MinasnyBudiman,et al.  A conditioned Latin hypercube method for sampling in the presence of ancillary information , 2006 .

[50]  Climatic tolerance or geographic breadth: what are we measuring? , 2016, Global change biology.

[51]  M. Araújo,et al.  Five (or so) challenges for species distribution modelling , 2006 .

[52]  Andreas Ziegler,et al.  ranger: A Fast Implementation of Random Forests for High Dimensional Data in C++ and R , 2015, 1508.04409.

[53]  Rui F. Fernandes,et al.  Effects of simulated observation errors on the performance of species distribution models , 2018, Diversity and Distributions.

[54]  A. Hastings,et al.  Habitat suitability estimated by niche models is largely unrelated to species abundance , 2018, Global Ecology and Biogeography.

[55]  J A Swets,et al.  Measuring the accuracy of diagnostic systems. , 1988, Science.

[56]  F. Jiguet,et al.  Selecting pseudo‐absences for species distribution models: how, where and how many? , 2012 .

[57]  Nicholas W. Synes,et al.  Choice of predictor variables as a source of uncertainty in continental‐scale species distribution modelling under climate change , 2011 .

[58]  X. Morin,et al.  Correlative climatic niche models predict real and virtual species distributions equally well. , 2019, Ecology.

[59]  Sam Veloz,et al.  Spatially autocorrelated sampling falsely inflates measures of accuracy for presence‐only niche models , 2009 .

[60]  Jon R. Wilson,et al.  Structural basis of oncogenic histone H3K27M inhibition of human polycomb repressive complex 2 , 2016, Nature Communications.

[61]  Anni Arponen,et al.  Projecting Global Biodiversity Indicators under Future Development Scenarios , 2016 .

[62]  Catherine S. Jarnevich,et al.  Misleading prioritizations from modelling range shifts under climate change , 2018 .

[63]  G. Tang,et al.  Indian Hedgehog: A Mechanotransduction Mediator in Condylar Cartilage , 2004, Journal of dental research.

[64]  Xavier Robin,et al.  pROC: an open-source package for R and S+ to analyze and compare ROC curves , 2011, BMC Bioinformatics.

[65]  K. Walker,et al.  Climatic Associations of British Species Distributions Show Good Transferability in Time but Low Predictive Accuracy for Range Change , 2012, PloS one.

[66]  Carsten Meyer,et al.  Multidimensional biases, gaps and uncertainties in global plant occurrence information. , 2016, Ecology letters.

[67]  Brendan A. Wintle,et al.  Is my species distribution model fit for purpose? Matching data and models to applications , 2015 .

[68]  Carsten F. Dormann,et al.  Cross-validation strategies for data with temporal, spatial, hierarchical, or phylogenetic structure , 2017 .

[69]  J. C. Brito,et al.  Predicting species distribution at range margins: testing the effects of study area extent, resolution and threshold selection in the Sahara–Sahel transition zone , 2014 .

[70]  J. Svenning,et al.  Historic and prehistoric human‐driven extinctions have reshaped global mammal diversity patterns , 2015 .

[71]  P. Hernandez,et al.  The effect of sample size and species characteristics on performance of different species distribution modeling methods , 2006 .

[72]  Wilfried Thuiller,et al.  Consequences of climate change on the tree of life in Europe , 2011, Nature.

[73]  R. Warren,et al.  The projected effect on insects, vertebrates, and plants of limiting global warming to 1.5°C rather than 2°C , 2018, Science.

[74]  Eric R. Dougherty,et al.  Parasite biodiversity faces extinction and redistribution in a changing climate , 2017, Science Advances.

[75]  Robert P. Anderson,et al.  Standards for distribution models in biodiversity assessments , 2019, Science Advances.

[76]  A. Peterson,et al.  The crucial role of the accessible area in ecological niche modeling and species distribution modeling , 2011 .

[77]  T. Newbold.  Future effects of climate and land-use change on terrestrial vertebrate community diversity under different scenarios , 2018, Proceedings of the Royal Society B: Biological Sciences.

[78]  P. De Marco,et al.  Model uncertainties do not affect observed patterns of species richness in the Amazon , 2017, PloS one.

[79]  J. Lobo,et al.  Is current climatic equilibrium a guarantee for the transferability of distribution model predictions? A case study of the spotted hyena , 2009 .

[80]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[81]  Brandon M. Greenwell.  pdp: An R Package for Constructing Partial Dependence Plots , 2017, R J..

[82]  Ramirez-Villegas.  Quantifying the benefit of early climate change mitigation in avoiding 1 biodiversity loss 2 3 , 2022 .

[83]  Xiao Feng,et al.  A standard protocol for reporting species distribution models , 2020, Ecography.

[84]  M. White,et al.  The effect of sample size on the accuracy of species distribution models: considering both presences and pseudo‐absences or background sites , 2019 .

[85]  A. Peterson,et al.  Major challenges for correlational ecological niche model projections to future climate conditions , 2018, Annals of the New York Academy of Sciences.

[86]  Wilfried Thuiller,et al.  Uncertainty in ensembles of global biodiversity scenarios , 2019, Nature Communications.

[87]  B. McGill,et al.  Testing the predictive performance of distribution models , 2013 .

[88]  Pedro Tarroso,et al.  Contemporary niche contraction affects climate change predictions for elephants and giraffes , 2016 .

[89]  N. Matzke,et al.  Evaluating presence‐only species distribution models with discrimination accuracy is uninformative for many applications , 2019, Journal of Biogeography.

[90]  Robin K. S. Hankin,et al.  Set partitions in R , 2007 .

[91]  Matthew J. Smith,et al.  Protected areas network is not adequate to protect a critically endangered East Africa Chelonian: Modelling distribution of pancake tortoise, Malacochersus tornieri under current and future climates , 2013, bioRxiv.

[92]  Graham J. Williams,et al.  wsrf: An R Package for Classification with Scalable Weighted Subspace Random Forests , 2017 .

[93]  A. Peterson,et al.  INTERPRETATION OF MODELS OF FUNDAMENTAL ECOLOGICAL NICHES AND SPECIES' DISTRIBUTIONAL AREAS , 2005 .

[94]  A. Hoffmann,et al.  Climate change and evolutionary adaptation , 2011, Nature.

[95]  Niels Raes,et al.  Minimum required number of specimen records to develop accurate species distribution models , 2016 .

[96]  Catherine S. Jarnevich,et al.  Modelling invasion for a habitat generalist and a specialist plant species , 2008 .

[97]  B. Leroy,et al.  Testing methods in species distribution modelling using virtual species: what have we learnt and what are we missing? , 2019, Ecography.

[98]  Thiago F. Rangel,et al.  Evaluating, partitioning, and mapping the spatial autocorrelation component in ecological niche modeling: a new approach based on environmentally equidistant records , 2014 .

[99]  J. R. Landis,et al.  The measurement of observer agreement for categorical data. , 1977, Biometrics.

[100]  Kendall R. Jones,et al.  Sixteen years of change in the global terrestrial human footprint and implications for biodiversity conservation , 2016, Nature Communications.

[101]  Julian D. Olden,et al.  Assessing transferability of ecological models: an underappreciated aspect of statistical validation , 2012 .

[102]  Yoan Fourcade,et al.  Paintings predict the distribution of species, or the challenge of selecting environmental predictors and evaluation statistics , 2018 .