Three way k-fold cross-validation of resource selection functions

A resource selection function (RSF) yields a prediction that is proportional to the probability of use of a resource unit by an organism. Because many apparently adequate models fail in new areas or time periods we developed a method for model selection and evaluation based on the model’s ability to predict generally, spatially, and temporally. This work is an extension of previous work using k-fold cross-validation to evaluate models developed using presence-only study designs. A RSF model’s utility is its ability to predict, so this method is applicable to any RSF model regardless of study design. The use and application of our proposed 3-way evaluation using the RSF Plot Index (RPI) statistic is illustrated using survey data of grassland birds, Landsat imagery, soil data, and a Digital Elevation Model from the Canadian Forces Base Suffield in southeastern Alberta. The sensitivity of the RPI statistic to the number and placement of bins is addressed and a method is presented to ameliorate this problem. The 3-way method provides the means to not only select the model with the best predictive power, but to understand the limitations of all models under consideration. Test results of best models using an independent field season are presented.

[1]  Jack J. Lennon,et al.  Red-shifts and red herrings in geographical ecology , 2000 .

[2]  K. Price,et al.  Optimal Landsat TM band combinations and vegetation indices for discrimination of six grassland types in eastern Kansas , 2002 .

[3]  Douglas H. Johnson,et al.  Patch Size and Landscape Effects on Density and Nesting Success of Grassland Birds , 2006 .

[4]  R. A. Owens,et al.  Effects of agriculture upon populations of native passerine birds of an Alberta fescue grassland , 1973 .

[5]  Lennon.  Resource selection functions: taking space seriously? , 1999, Trends in ecology & evolution.

[6]  J. A. Schell,et al.  Monitoring vegetation systems in the great plains with ERTS , 1973 .

[7]  Limin Yang,et al.  Derivation of a tasselled cap transformation based on Landsat 7 at-satellite reflectance , 2002 .

[8]  FRED S. GUTHERY,et al.  INVITED PAPER: INFORMATION THEORY IN WILDLIFE SCIENCE: CRITIQUE AND VIEWPOINT , 2005 .

[9]  B. Manly,et al.  Resource selection by animals: statistical design and analysis for field studies. , 1994 .

[10]  M. Schemper,et al.  A solution to the problem of separation in logistic regression , 2002, Statistics in medicine.

[11]  J. Diniz‐Filho,et al.  Spatial autocorrelation and red herrings in geographical ecology , 2003 .

[12]  Peter E. Lowther,et al.  Baird's Sparrow (Ammodramus bairdii) , 2002 .

[13]  Mark S. Boyce,et al.  Modelling distribution and abundance with presence‐only data , 2006 .

[14]  Brenda C. Dale,et al.  Sprague's Pipit (Anthus spragueii) , 1999 .

[15]  Yoshua Bengio,et al.  No Unbiased Estimator of the Variance of K-Fold Cross-Validation , 2003, J. Mach. Learn. Res..

[16]  R. Kauth,et al.  The tasselled cap - A graphic description of the spectral-temporal development of agricultural crops as seen by Landsat , 1976 .

[17]  Martin L. Cody,et al.  On the Methods of Resource Division in Grassland Bird Communities , 1968, The American Naturalist.

[18]  M. Skeel,et al.  Distribution and habitat associations of three endemic grassland songbirds in southern Saskatchewan , 1999 .

[19]  J. C. van Houwelingen,et al.  A goodness-of-fit test for binary regression models, based on smoothing methods , 1991 .

[20]  Alberta.,et al.  Range plant communities and range health assessment guidelines for the dry mixedgrass natural subregion of Alberta , 2005 .

[21]  William K. Lauenroth,et al.  Changes in grassland canopy structure across a precipitation gradient , 2000 .

[22]  M. Boyce,et al.  Evaluating resource selection functions , 2002 .

[23]  M. Oosterveld,et al.  Empirical relations between laboratory determinations of soil texture and moisture retention. , 1980 .

[24]  S. Weisberg,et al.  Residuals and Influence in Regression , 1982 .

[25]  R. Macarthur,et al.  On Bird Species Diversity , 1961 .

[26]  John A. Wiens,et al.  An approach to the study of ecological relationships among grassland birds , 1969 .

[27]  C. Lisa Mahon.  Habitat selection and detectability of Baird's sparrows in southwestern Alberta , 1995 .

[28]  Shay Howlin,et al.  A VALIDATION TECHNIQUE FOR ASSESSING PREDICTIVE ABILITIES OF RESOURCE SELECTION FUNCTIONS , 2003 .

[29]  A. Huete,et al.  A Modified Soil Adjusted Vegetation Index , 1994 .

[30]  Paul E. Gessler,et al.  Modeling Soil–Landscape and Ecosystem Properties Using Terrain Attributes , 2000 .

[31]  Osvaldo E. Sala,et al.  Inter-annual variation in primary production of a semi-arid grassland related to previous-year production , 2001 .

[32]  Geo. W. Robertson,et al.  ESTIMATING MOST PROBABLE PRAIRIE WHEAT PRODUCTION FROM PRECIPITATION DATA , 1965 .

[33]  A. Poole,et al.  Lark Bunting (Calamospiza melanocorys) , 2000 .

[34]  Paul Hendricks,et al.  A Fixed-Radius Point Count Method for Nonbreeding and Breeding-Season Use , 1986 .

[35]  Chris J. Johnson,et al.  Resource Selection Functions Based on Use–Availability Data: Theoretical Motivation and Evaluation Methods , 2006 .

[36]  R. De Jong,et al.  EMPIRICAL RELATIONS BETWEEN SOIL COMPONENTS AND WATER RETENTION AT 1/3 and 15 ATMOSPHERES , 1982 .

[37]  N. Koper,et al.  A multi-scaled analysis of avian response to habitat amount and fragmentation in the Canadian dry mixed-grass prairie , 2006, Landscape Ecology.

[38]  K. Klement,et al.  Interactive Effects of Drought and Grazing on Northern Great Plains Rangelands , 2005 .

[39]  S. K. Davis,et al.  AREA SENSITIVITY IN GRASSLAND PASSERINES: EFFECTS OF PATCH SIZE, PATCH SHAPE, AND VEGETATION STRUCTURE ON BIRD ABUNDANCE AND OCCURRENCE IN SOUTHERN SASKATCHEWAN , 2004 .

[40]  D. McMaster,et al.  AN EVALUATION OF CANADA'S PERMANENT COVER PROGRAM: HABITAT FOR GRASSLAND BIRDS? , 2001 .

[41]  David R. Anderson,et al.  Model selection and multimodel inference : a practical information-theoretic approach , 2003 .

[42]  Geoffrey A. J. Scott.  Canada's Vegetation: A World Perspective , 1995 .

[43]  A. Hirzel,et al.  Evaluating the ability of habitat suitability models to predict species presences , 2006 .

[44]  Lawrence B. Flanagan,et al.  Seasonal and interannual variation in carbon dioxide exchange and carbon balance in a northern temperate grassland , 2002 .

[45]  D. Hosmer,et al.  A comparison of goodness-of-fit tests for the logistic regression model. , 1997, Statistics in medicine.

[46]  Robert C. Beason,et al.  Horned Lark (Eremophila alpestris) , 1995 .

[47]  S. Weisberg,et al.  Residuals and Influence in Regression , 1982 .

[48]  F. Samson,et al.  Prairie conservation: preserving North America's most endangered ecosystem. , 1997 .

[49]  Charlotte H. Mason,et al.  Collinearity, power, and interpretation of multiple regression analysis. , 1991 .

[50]  S. Baird,et al.  The birds of North America , 1974 .

[51]  G. Stenhouse,et al.  Modeling grizzly bear habitats in the Yellowhead ecosystem of Alberta: taking autocorrelation seriously , 2002 .