Temporal validation plots: quantifying how well correlative species distribution models predict species' range changes over time

The use of data documenting how species' distributions have changed over time is crucial for testing how well correlative species distribution models (SDMs) predict species' range changes. So far, however, little attention has been given to developing a reliable methodological framework for using such data. We develop a new tool – the temporal validation (TV) plot – specifically aimed at making use of species' distribution records at two times for a comprehensive assessment of the prediction accuracy of SDMs over time. We extend existing presence–absence calibration plots to make use of distribution records from two time periods. TV plots visualize the agreement between change in modelled probabilities of presence and the probability of observing sites gained or lost between time periods. We then present three measures of prediction accuracy that can be easily calculated from TV plots. We present our methodological framework using a virtual species in a simplified landscape and then provide a real‐world case study using distribution records for two species of breeding birds from two time periods of intensive recording effort across Great Britain. Together with existing approaches, TV plots and their associated measures offer a simple tool for testing how well SDMs model species' observed range changes – perhaps the best way available to assess their ability to predict likely future changes.

[1]  R. Levy.  An Integrated Model , 2016 .

[2]  S. Richards,et al.  Prevalence, thresholds and the performance of presence–absence models , 2014 .

[3]  R Core Team,et al.  R: A language and environment for statistical computing. , 2014 .

[4]  Colin Robertson,et al.  Assessing Quality of Spatial Models Using the Structural Similarity Index and Posterior Predictive Checks , 2014 .

[5]  S. Mohapatra,et al.  Binary Logistic Regression , 2014 .

[6]  C. Moritz,et al.  Evaluation of species distribution models by resampling of sites surveyed a century ago by Joseph Grinnell , 2013 .

[7]  Scott K. Robinson,et al.  Exploring the role of physiology and biotic interactions in determining elevational ranges of tropical animals , 2013 .

[8]  K. Walker,et al.  Climatic Associations of British Species Distributions Show Good Transferability in Time but Low Predictive Accuracy for Range Change , 2012, PloS one.

[9]  F. Schurr,et al.  Forecasting species ranges by statistical estimation of ecological niches and spatial population dynamics , 2012 .

[10]  J. Nichols,et al.  An integrated model of habitat and species occurrence dynamics , 2011 .

[11]  Joshua Drew,et al.  The Role of Natural History Institutions and Bioinformatics in Conservation Biology , 2011, Conservation biology : the journal of the Society for Conservation Biology.

[12]  R. Pontius,et al.  Death to Kappa: birth of quantity disagreement and allocation disagreement for accuracy assessment , 2011 .

[13]  S. Dobrowski,et al.  Modeling plant ranges over 75 years of climate change in California, USA: temporal transferability and species traits , 2011 .

[14]  J. L. Parra,et al.  The role of climate, habitat, and species co‐occurrence as drivers of change in small mammal distributions over the past century , 2011 .

[15]  Truly Santika.  Assessing the effect of prevalence on the predictive performance of species distribution models using simulated data , 2011 .

[16]  Steven J. Phillips,et al.  The art of modelling range‐shifting species , 2010 .

[17]  Hugh P Possingham,et al.  Regional avian species declines estimated from volunteer-collected long-term data using List Length Analysis. , 2010, Ecological applications : a publication of the Ecological Society of America.

[18]  Jane Elith,et al.  POC plots: calibrating species distribution models with presence-only data. , 2010, Ecology.

[19]  P. Ehrlich,et al.  Biological collections and ecological/environmental research: a review, some observations and a look to the future , 2010, Biological reviews of the Cambridge Philosophical Society.

[20]  Steven R. Beissinger,et al.  Birds track their Grinnellian niche through a century of climate change , 2009, Proceedings of the National Academy of Sciences.

[21]  S. Beissinger,et al.  Detecting range shifts from historical species occurrences: new perspectives on old data. , 2009, Trends in ecology & evolution.

[22]  J. Kerr,et al.  Historically calibrated predictions of butterfly species' range shift using global change as a pseudo-experiment. , 2009, Ecology.

[23]  W. Hargrove,et al.  The projection of species distribution models and the problem of non-analog climate , 2009, Biodiversity and Conservation.

[24]  J. Elith,et al.  Species Distribution Models: Ecological Explanation and Prediction Across Space and Time , 2009 .

[25]  Res Altwegg,et al.  Climate and the range dynamics of species with imperfect detection , 2008, Biology Letters.

[26]  J Elith,et al.  A working guide to boosted regression trees. , 2008, The Journal of animal ecology.

[27]  Rhys E. Green,et al.  Using simple species lists to monitor trends in animal populations: new methods and a comparison with independent data , 2007 .

[28]  A. Hirzel,et al.  Evaluating the ability of habitat suitability models to predict species presences , 2006 .

[29]  A. Townsend Peterson,et al.  Novel methods improve prediction of species' distributions from occurrence data , 2006 .

[30]  M. Araújo,et al.  Validation of species–climate impact models under climate change , 2005 .

[31]  M. Araújo,et al.  Reducing uncertainty in projections of extinction risk from climate change , 2005 .

[32]  T. D. Mitchell,et al.  An improved method of constructing a database of monthly climate observations and associated high‐resolution grids , 2005 .

[33]  D. Rogers,et al.  The effects of species’ range sizes on the accuracy of distribution models: ecological phenomenon or statistical artefact? , 2004 .

[34]  N. Aebischer,et al.  Habitat availability and use by Turtle Doves Streptopelia turtur between 1965 and 1995: an analysis of Common Birds Census data , 2004 .

[35]  Barry Boots,et al.  Toward Comparing Maps as Spatial Processes , 2004, SDH.

[36]  M. Boyce,et al.  Evaluating resource selection functions , 2002 .

[37]  J. Andrew Royle,et al.  ESTIMATING SITE OCCUPANCY RATES WHEN DETECTION PROBABILITIES ARE LESS THAN ONE , 2002, Ecology.

[38]  Simon Ferrier,et al.  Evaluating the predictive performance of habitat models developed using logistic regression , 2000 .

[39]  P. Jones,et al.  Representing Twentieth-Century Space-Time Climate Variability. Part II: Development of 1901-96 Monthly Grids of Terrestrial Surface Climate , 2000 .

[40]  Humphrey Q. P. Crick,et al.  Climate change related to egg-laying trends , 1999, Nature.

[41]  P. Jones,et al.  Representing Twentieth-Century Space–Time Climate Variability. Part I: Development of a 1961–90 Mean Monthly Terrestrial Climatology , 1999 .

[42]  G. Ridgeway.  The State of Boosting ∗ , 1999 .

[43]  F. Harrell,et al.  Prognostic/Clinical Prediction Models: Multivariable Prognostic Models: Issues in Developing Models, Evaluating Assumptions and Adequacy, and Measuring and Reducing Errors , 2005 .

[44]  Stephen R. Baillie,et al.  Population declines and range contractions among lowland farmland birds in Britain , 1995 .

[45]  Gareth Jones,et al.  The new atlas of breeding birds in Britain and Ireland 1988-1991 , 1993 .

[46]  S L Hui,et al.  Validation techniques for logistic regression models. , 1991, Statistics in medicine.

[47]  J. Hanley,et al.  The meaning and use of the area under a receiver operating characteristic (ROC) curve. , 1982, Radiology.

[48]  C. Mason,et al.  Breeding biology of the Pied and Yellow Wagtails , 1980 .

[49]  J. T.R. Sharrock,et al.  The Atlas of Breeding Birds in Britain and Ireland , 1980 .

[50]  V. Hamilton,et al.  Reducing Uncertainty , 1967, Nature.