Model application niche analysis: Assessing the transferability and generalizability of ecological models.

The use of models by ecologists and environmental managers, to inform environmental management and decision-making, has grown exponentially in the past 50 years. Due to logistical, economical, and theoretical benefits, model users frequently transfer preexisting models to new sites where data are scarce. Modelers have made significant progress in understanding how to improve model generalizability during model development. However, models are always imperfect representations of systems and are constrained by the contextual frameworks used during their development. Thus, model users need better ways to evaluate the possibility of unintentional misapplication when transferring models to new sites. We propose a method of describing a model's application niche for use during the model selection process. Using this method, model users synthesize information from databases, past studies, and/or past model transfers to create model performance curves and heat maps. We demonstrated this method using an empirical model developed to predict the ecological condition of plant communities in riverine wetlands of the Appalachian Highland physiographic region, U.S.A. We assessed this model's transferability and generalizability across (1) riverine wetlands in the contiguous U.S.A., (2) wetland types in the Appalachian Highland physiographic region, and (3) wetland types in the contiguous U.S.A. With this methodology and a discussion of its critical steps, we set the stage for further inquiries into the development of consistent and transparent practices for model selection when transferring a model.

[1]  Cory P. McDonald,et al.  Using a model selection criterion to identify appropriate complexity in aquatic biogeochemical models , 2010 .

[2]  Robert P. Brooks,et al.  A plant-based index of biological integrity (IBI) for headwater wetlands in central Pennsylvania , 2006 .

[3]  R. Brooks,et al.  Hydric Soils Across Pennsylvania Reference, Disturbed, and Mitigated Wetlands , 2013 .

[4]  C. Meynard,et al.  Predicting species distributions: a critical comparison of the most common statistical models using artificial species , 2007 .

[5]  Hugh P. Possingham,et al.  Evaluating model transferability for a threatened species to adjacent areas: Implications for rock-wallaby conservation , 2011 .

[6]  Jianguo Wu,et al.  CONCEPTS OF SCALE AND SCALING , 2006 .

[7]  J. Pellet,et al.  The transferability of distribution models across regions: an amphibian case study , 2009 .

[8]  Mark T. Brown,et al.  LANDSCAPE DEVELOPMENT INTENSITY INDEX , 2005, Environmental monitoring and assessment.

[9]  A. G. Endress,et al.  Trajectories of vegetation-based indicators used to assess wetland restoration progress. , 2009, Ecological applications : a publication of the Ecological Society of America.

[10]  A. Vatn,et al.  Transferring environmental value estimates: Issues and alternatives , 2006 .

[11]  Anthony C. Davison,et al.  Bootstrap Methods and Their Application , 1998 .

[12]  Barbara A. Adams-Vanharn,et al.  Evaluation of the current state of mechanistic aquatic biogeochemical modeling: citation analysis and future perspectives. , 2006, Environmental science & technology.

[13]  Jason T. Bried,et al.  Floristic quality assessment signals human disturbance over natural variability in a wetland system , 2013 .

[14]  Steven J. Phillips,et al.  The art of modelling range‐shifting species , 2010 .

[15]  C. Stow,et al.  Skill Assessment for Coupled Biological/Physical Models of Marine Systems. , 2009, Journal of marine systems : journal of the European Association of Marine Sciences and Techniques.

[16]  Stephen J. Burges,et al.  Assessment of soil-based and calibrated parameters of the Sacramento model and parameter transferability , 2006 .

[17]  H. Jenny Factors of Soil Formation: A System of Quantitative Pedology , 2011 .

[18]  J. Krebs,et al.  Should conservation strategies consider spatial generality? Farmland birds show regional not national patterns of habitat association. , 2007, Ecology letters.

[19]  Paul L. Angermeier,et al.  Factors influencing behavior and transferability of habitat models for a benthic stream fish , 1997 .

[20]  H. Van Dyck,et al.  Transferability of Species Distribution Models: a Functional Habitat Approach for Two Regionally Threatened Butterflies , 2007, Conservation biology : the journal of the Society for Conservation Biology.

[21]  Sarah J. Miller,et al.  Adapting the floristic quality assessment index to indicate anthropogenic disturbance in central Pennsylvania wetlands , 2006 .

[22]  Hydrogeomorphology, environment, and vegetation associations across a latitudinal gradient in highland wetlands of the northeastern USA , 2009, Plant Ecology.

[23]  J. Visser,et al.  Development and use of a floristic quality index for coastal Louisiana marshes , 2012, Environmental Monitoring and Assessment.

[24]  J. Elith,et al.  Species Distribution Models: Ecological Explanation and Prediction Across Space and Time , 2009 .

[25]  S. Walker,et al.  The use of mechanistic descriptions of algal growth and zooplankton grazing in an estuarine eutrophication model , 2003 .

[26]  J. Wiens Spatial Scaling in Ecology , 1989 .

[27]  Christoph Lange,et al.  Context-oriented model validation of individual-based models in ecology: A hierarchically structured approach to validate qualitative, compositional and quantitative characteristics , 2015 .

[28]  Mark D. Anderson,et al.  Trade-offs between specificity and regional generality in habitat association models: a case study of two species of African vulture , 2009 .

[29]  Stephen E. Fick,et al.  WorldClim 2: new 1‐km spatial resolution climate surfaces for global land areas , 2017 .

[30]  D. Mladenoff,et al.  A forest growth and biomass module for a landscape simulation model, LANDIS: design, validation, and application , 2004 .

[31]  D. Goffaux,et al.  Assessing river biotic condition at a continental scale: a European approach using functional metrics and fish assemblages , 2006 .

[32]  R. Brooks,et al.  Testing a rapid Floristic Quality Index on headwater wetlands in central Pennsylvania, USA , 2016 .

[33]  Hadley Wickham,et al.  ggplot2 - Elegant Graphics for Data Analysis (2nd Edition) , 2017 .

[34]  K. Taylor,et al.  Restoration of vegetation communities of created depressional marshes in Ohio and Colorado (USA): The importance of initial effort for mitigation success , 2009 .

[35]  R Core Team,et al.  R: A language and environment for statistical computing. , 2014 .

[36]  M. Fortin,et al.  Spatial pattern and ecological analysis , 1989, Vegetatio.

[37]  Raghavan Srinivasan,et al.  INTEGRATION OF A BASIN‐SCALE WATER QUALITY MODEL WITH GIS , 1994 .

[38]  J. R. Landis,et al.  The measurement of observer agreement for categorical data. , 1977, Biometrics.

[39]  Julian D. Olden,et al.  Assessing transferability of ecological models: an underappreciated aspect of statistical validation , 2012 .

[40]  C. Ahn,et al.  Soil Properties Predict Plant Community Development of Mitigation Wetlands Created in the Virginia Piedmont, USA , 2012, Environmental Management.

[41]  Edward B. Rastetter,et al.  Validating models of ecosystem response to global change , 1996 .

[42]  Sven Erik Jørgensen,et al.  Overview of the model types available for development of ecological models , 2008 .

[43]  H. Possingham,et al.  Can multiscale models of species’ distribution be generalized from region to region? A case study of the koala , 2008 .

[44]  Robert P. Brooks,et al.  Soil properties of reference wetlands and wetland creation projects in Pennsylvania , 1996, Wetlands.

[45]  Tatsuya Amano,et al.  Unravelling the dynamics of organisms in a changing world using ecological modelling , 2012, Ecological Research.

[46]  Soroosh Sorooshian,et al.  Model Parameter Estimation Experiment (MOPEX): An overview of science strategy and major results from the second and third workshops , 2006 .

[47]  M. Brauer,et al.  Intercity transferability of land use regression models for estimating ambient concentrations of nitrogen dioxide , 2009, Journal of Exposure Science and Environmental Epidemiology.

[48]  Joseph H. A. Guillaume,et al.  Characterising performance of environmental models , 2013, Environ. Model. Softw..

[49]  K. Reiss Florida Wetland Condition Index for depressional forested wetlands , 2006 .

[50]  Steve Frolking,et al.  A model of nitrous oxide evolution from soil driven by rainfall events: 2. Model applications , 1992 .

[51]  Raghavan Srinivasan,et al.  SWAT: Model Use, Calibration, and Validation , 2012 .

[52]  L. M. Cowardin,et al.  Classification of Wetlands and Deepwater Habitats of the United States , 2017 .

[53]  H. Jenny,et al.  Factors of Soil Formation , 1941 .

[54]  P. Bohlen,et al.  Land-use and isolation interact to affect wetland plant assemblages , 2010 .

[55]  K. Winemiller,et al.  Ecoregional, catchment, and reach-scale environmental factors shape functional-trait structure of stream fish assemblages , 2015, Hydrobiologia.

[56]  K. Mellert,et al.  Species distribution models as a tool for forest management planning under climate change: risk evaluation of Abies alba in Bavaria , 2011 .

[57]  Michael Brauer,et al.  The transferability of NO and NO2 land use regression models between cities and pollutants , 2011 .

[58]  B. McCune,et al.  Analysis of Ecological Communities , 2002 .

[59]  J. Kerr,et al.  Historically calibrated predictions of butterfly species' range shift using global change as a pseudo-experiment. , 2009, Ecology.

[60]  Randall S. Rosenberger,et al.  Measurement, generalization, and publication: Sources of error in benefit transfers and their management , 2006 .

[61]  Edward J. Rykiel,et al.  Testing ecological models: the meaning of validation , 1996 .

[62]  W. Tobler A Computer Movie Simulating Urban Growth in the Detroit Region , 1970 .

[63]  Randall S. Rosenberger,et al.  Correspondence and Convergence in Benefit Transfer Accuracy: Meta-Analytic Review of the Literature , 2007 .

[64]  Brian D. Ripley,et al.  Modern Applied Statistics with S Fourth edition , 2002 .

[65]  David B. Smith,et al.  Geochemical and mineralogical data for soils of the conterminous United States , 2013 .

[66]  S. Chamberlain,et al.  Developing coefficients of conservatism to advance floristic quality assessment in the Mid-Atlantic region1 , 2012 .

[67]  M. Brinson A Hydrogeomorphic Classification for Wetlands , 1993 .

[68]  Kazuaki Hiramatsu,et al.  Prediction ability and sensitivity of artificial intelligence-based habitat preference models for predicting spatial distribution of Japanese medaka (Oryzias latipes) , 2008 .

[69]  Raghavan Srinivasan,et al.  Nonpoint Source (NPS) Pollution Modeling Using Models Integrated with Geographic Information Systems (GIS) , 1993 .

[70]  C. R. Lane,et al.  Isolated Wetlands of the Southeastern United States: Abundance and Expected Condition , 2012, Wetlands.

[71]  Robert P. Anderson,et al.  Species-specific tuning increases robustness to sampling bias in models of species distributions: An implementation with Maxent , 2011 .

[72]  Bruce L. Webber,et al.  Here be dragons: a tool for quantifying novelty due to covariate range and correlation change when projecting species distribution models , 2014 .

[73]  John R. Williams,et al.  LARGE AREA HYDROLOGIC MODELING AND ASSESSMENT PART I: MODEL DEVELOPMENT 1 , 1998 .

[74]  M. Araújo,et al.  Validation of species–climate impact models under climate change , 2005 .

[75]  S. Nichols Floristic quality assessment of Wisconsin lake plant communities with example applications , 1999 .

[76]  Suming Jin,et al.  Completion of the 2011 National Land Cover Database for the Conterminous United States – Representing a Decade of Land Cover Change Information , 2015 .

[77]  Antoine Guisan,et al.  Are niche-based species distribution models transferable in space? , 2006 .

[78]  R. Brooks,et al.  The Wetland Disturbance Index: Links with Soil and Water Nitrate Concentrations , 2011, Wetlands.

[79]  W. McNab,et al.  Ecological zones in the Southern Appalachians: first approximation , 2005 .

[80]  J. Diniz‐Filho,et al.  Spatial analysis improves species distribution modelling during range expansion , 2008, Biology Letters.

[81]  R. Brooks,et al.  Comparison of Hydrology of Wetlands in Pennsylvania and Oregon (USA) as an Indicator of Transferability of Hydrogeomorphic (HGM) Functional Models Between Regions , 2002, Environmental management.

[82]  Marvin N. Wright,et al.  SoilGrids250m: Global gridded soil information based on machine learning , 2017, PloS one.

[83]  R. Lopez,et al.  TESTING THE FLORISTIC QUALITY ASSESSMENT INDEX AS AN INDICATOR OF WETLAND CONDITION , 2002 .

[84]  Damaris Zurell,et al.  Predicting to new environments: tools for visualizing model behaviour and impacts on mapped distributions , 2012 .

[85]  JAMES R. MILLER,et al.  Spatial Extrapolation: The Science of Predicting Ecological Patterns and Processes , 2004 .

[86]  J. Aber,et al.  A generalized, lumped-parameter model of photosynthesis, evapotranspiration and net primary production in temperate and boreal forest ecosystems , 1992, Oecologia.

[87]  A. Townsend Peterson,et al.  Constraints on interpretation of ecological niche models by limited environmental ranges on calibration areas , 2012 .

[88]  N. Rao,et al.  12.04 – Environmental Benefit Transfers of Ecosystem Service Valuation , 2011 .

[89]  Deok-Kyun Yun,et al.  Clustering Categorical and Numerical Data: A New Procedure Using Multidimensional Scaling , 2003, Int. J. Inf. Technol. Decis. Mak..

[90]  Robert P. Anderson,et al.  The effect of the extent of the study region on GIS models of species geographic distributions and estimates of niche evolution: preliminary tests with montane rodents (genus Nephelomys) in Venezuela , 2010 .

[91]  Larry Kapustka,et al.  Using ecological production functions to link ecological processes to ecosystem services , 2017, Integrated environmental assessment and management.

[92]  M. Brittingham,et al.  Terrestrial liming benefits birds in an acidified forest in the northeast. , 2007, Ecological applications : a publication of the Ecological Society of America.

[93]  H. Shugart,et al.  Predictive distribution modeling with enhanced remote sensing and multiple validation techniques to support mountain bongo antelope recovery , 2011 .

[94]  John M. Norman,et al.  Application of Geostatistics to Characterize Leaf Area Index (LAI) from Flux Tower to Landscape Scales Using a Cyclic Sampling Design , 2002, Ecosystems.

[95]  W. Parton,et al.  Analysis of factors controlling soil organic matter levels in Great Plains grasslands , 1987 .

[96]  Sven Erik Jørgensen,et al.  Tool boxes for an integrated ecological and environmental management , 2012 .

[97]  Mathieu Marmion,et al.  Does the interpolation accuracy of species distribution models come at the expense of transferability , 2012 .

[98]  William N. Venables,et al.  Modern Applied Statistics with S , 2010 .

[99]  Changsheng Li,et al.  A model of nitrous oxide evolution from soil driven by rainfall events: 1. Model structure and sensitivity , 1992 .