Complexity is costly: a meta‐analysis of parametric and non‐parametric methods for short‐term population forecasting

Short-term forecasts based on time series of counts or survey data are widely used in population biology to provide advice concerning the management, harvest and conservation of natural populations. A common approach to produce these forecasts uses time-series models, of different types, fit to time series of counts. Similar time-series models are used in many other disciplines, however relative to the data available in these other disciplines, population data are often unusually short and noisy and models that perform well for data from other disciplines may not be appropriate for population data. In order to study the performance of time-series forecasting models for natural animal population data, we assembled 2379 time series of vertebrate population indices from actual surveys. Our data were comprised of three vastly different types: highly variable (marine fish productivity), strongly cyclic (adult salmon counts), and small variance but long-memory (bird and mammal counts). We tested the predictive performance of 49 different forecasting models grouped into three broad classes: autoregressive time-series models, non-linear regression-type models and non-parametric time-series models. Low-dimensional parametric autoregressive models gave the most accurate forecasts across a wide range of taxa; the most accurate model was one that simply treated the most recent observation as the forecast. More complex parametric and non-parametric models performed worse, except when applied to highly cyclic species. Across taxa, certain life history characteristics were correlated with lower forecast error; specifically, we found that better forecasts were correlated with attributes of slow growing species: large maximum age and size for fishes and high trophic level for birds. © 2014 Nordic Society Oikos.

[1]  J. Stock,et al.  A Comparison of Linear and Nonlinear Univariate Models for Forecasting Macroeconomic Time Series , 1998 .

[2]  Morton E. Gurtin,et al.  Non-linear age-dependent population dynamics , 1974 .

[3]  Non-linear age-dependent population growth , 1980, Journal of mathematical biology.

[4]  Perry de Valpine,et al.  Are patterns of density dependence in the Global Population Dynamics Database driven by uncertainty about population abundance? , 2012, Ecology letters.

[5]  Ray Hilborn,et al.  Standing on the Shoulders of Giants: Learning from Experience in Fisheries , 1998, Reviews in Fish Biology and Fisheries.

[6]  G. Coco,et al.  Complex Positive Connections between Functional Groups Are Revealed by Neural Network Analysis of Ecological Time Series , 2008, The American Naturalist.

[7]  E. Toth,et al.  Comparison of short-term rainfall prediction models for real-time flood forecasting , 2000 .

[8]  George Sugihara,et al.  Detecting and forecasting complex nonlinear dynamics in spatially structured catch-per-unit- effort time series for North Pacific albacore (Thunnus alalunga) , 2011 .

[9]  B T Grenfell,et al.  Noisy Clockwork: Time Series Analysis of Population Fluctuations in Animals , 2001, Science.

[10]  Brian Dennis,et al.  Estimation of Growth and Extinction Parameters for Endangered Species , 1991 .

[11]  Michael J. Ford,et al.  Status review update for Pacific salmon and steelhead listed under the Endangered Species Act : Pacific Northwest , 2011 .

[12]  I. Dimopoulos,et al.  Application of neural networks to modelling nonlinear relationships in ecology , 1996 .

[13]  A. Raftery,et al.  Using Bayesian Model Averaging to Calibrate Forecast Ensembles , 2005 .

[14]  David R. Anderson,et al.  Model Selection and Multimodel Inference , 2003 .

[15]  C. S. Robbins,et al.  The breeding bird survey , 1986 .

[16]  Elizabeth E. Holmes,et al.  Inferring spatial structure from time‐series data: using multivariate state‐space models to detect metapopulation structure of California sea lions in the Gulf of California, Mexico , 2010 .

[17]  Stephen T. Buckland,et al.  State-space models for the dynamics of wild animal populations , 2004 .

[18]  George Sugihara,et al.  Model-free forecasting outperforms the correct mechanistic model for simulated and experimental data , 2013, Proceedings of the National Academy of Sciences.

[19]  G Sugihara,et al.  Distinguishing error from chaos in ecological time series. , 1990, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[20]  Chih-hao Hsieh,et al.  Extending Nonlinear Analysis to Short Ecological Time Series , 2007, The American Naturalist.

[21]  S. Wood Generalized Additive Models: An Introduction with R , 2006 .

[22]  S. Lele,et al.  ESTIMATING DENSITY DEPENDENCE, PROCESS NOISE, AND OBSERVATION ERROR , 2006 .

[23]  E. E. Holmes,et al.  Estimating risks in declining populations with poor data , 2001, Proceedings of the National Academy of Sciences of the United States of America.

[24]  Rainer Froese,et al.  FishBase 2000: Concepts, designs and data sources , 2000 .

[25]  Rob J Hyndman,et al.  25 years of time series forecasting , 2006 .

[26]  C. Granger,et al.  Experience with Forecasting Univariate Time Series and the Combination of Forecasts , 1974 .

[27]  Cóilín Minto,et al.  Examining the knowledge base and status of commercially exploited marine species with the RAM Legacy Stock Assessment Database , 2012 .

[28]  Konstantinos I. Stergiou,et al.  Modelling and forecasting annual fisheries catches: comparison of regression, univariate and multivariate time series methods , 1997 .

[29]  Ben Collen,et al.  Monitoring Change in Vertebrate Abundance: the Living Planet Index , 2009, Conservation biology : the journal of the Society for Conservation Biology.

[30]  W. Link,et al.  The North American Breeding Bird Survey Results and Analysis , 1997 .

[31]  Rob J Hyndman,et al.  Another look at measures of forecast accuracy , 2006 .

[32]  S. T. Bucklanda,et al.  State-space models for the dynamics of wild animal populations , 2003 .

[33]  D. R. Cutler,et al.  Utah State University From the SelectedWorks of , 2017 .

[34]  R. Peterman,et al.  Magnitude and Trends in Abundance of Hatchery and Wild Pink Salmon, Chum Salmon, and Sockeye Salmon in the North Pacific Ocean , 2010 .

[35]  Rob J Hyndman,et al.  A state space framework for automatic forecasting using exponential smoothing methods , 2002 .

[36]  S. Lindley,et al.  STATUS REVIEW UPDATE FOR PACIFIC SALMON AND STEELHEAD LISTED UNDER THE ENDANGERED SPECIES ACT: SOUTHWEST , 2011 .

[37]  Karen C. Abbott,et al.  Analysis of ecological time series with ARMA(p,q) models. , 2010, Ecology.

[38]  John Geweke,et al.  Comparing alternative tests of causality in temporal systems: Analytic results and experimental evidence☆ , 1983 .

[39]  R. May,et al.  VARIABLE PARASITOID SEX RATIOS AND THEIR EFFECT ON HOST-PARASITOID DYNAMICS , 1983 .

[40]  R. Peterman,et al.  Historical trends in productivity of 120 Pacific pink, chum, and sockeye salmon stocks reconstructed by using a Kalman filter , 2008 .

[41]  George Sugihara,et al.  Nonlinear forecasting for the classification of natural time series , 1994, Philosophical Transactions of the Royal Society of London. Series A: Physical and Engineering Sciences.

[42]  S. Lindley,et al.  ESTIMATION OF POPULATION GROWTH AND EXTINCTION PARAMETERS FROM NOISY DATA , 2003 .

[43]  George Sugihara,et al.  Nonlinear forecasting as a way of distinguishing chaos from measurement error in time series , 1990, Nature.

[44]  Steven V. Viscido,et al.  A statistical approach to quasi-extinction forecasting. , 2007, Ecology letters.

[45]  R. May Thresholds and breakpoints in ecosystems with a multiplicity of stable states , 1977, Nature.

[46]  A. Hastings,et al.  Stochastic Dynamics and Deterministic Skeletons: Population Behavior of Dungeness Crab , 1997 .

[47]  W. Cleveland,et al.  Locally Weighted Regression: An Approach to Regression Analysis by Local Fitting , 1988 .

[48]  J. Randers,et al.  The Living Planet Index: using species population time series to track trends in biodiversity , 2005, Philosophical Transactions of the Royal Society B: Biological Sciences.

[49]  S. T. Buckland,et al.  Hidden process models for animal population dynamics. , 2006, Ecological applications : a publication of the Ecological Society of America.

[50]  Elizabeth E. Holmes,et al.  VALIDATING POPULATION VIABILITY ANALYSIS FOR CORRUPTED DATA SETS , 2002 .