Land-cover change model validation by an ROC method for the Ipswich watershed, Massachusetts, USA

Scientists need a better and larger set of tools to validate land-use change models, because it is essential to know a model’s prediction accuracy. This paper describes how to use the relative operating characteristic (ROC) as a quantitative measurement to validate a land-cover change model. Typically, a crucial component of a spatially explicit simulation model of land-cover change is a map of suitability for land-cover change, for example a map of probability of deforestation. The model usually selects locations for new land-cover change at locations that have relatively high suitability. The ROC can compare a map of actual change to maps of modeled suitability for land-cover change. ROC is a summary statistic derived from several two-by-two contingency tables, where each contingency table corresponds to a different simulated scenario of future land-cover change. The categories in each contingency table are actual change and actual non-change versus simulated change and simulated non-change. This paper applies the theoretical concepts to a model of deforestation in the Ipswich watershed, USA. © 2001 Elsevier Science B.V. All rights reserved.

[1]  R. Costanza.  MODEL GOODNESS OF FIT: A MULTIPLE RESOLUTION PROCEDURE , 1989 .

[2]  D F Wagner,et al.  Cellular Automata and Geographic Information Systems , 1997 .

[3]  N. Bockstael,et al.  Spatial landscape indices in a hedonic framework: an ecological economics analysis using GIS , 1997 .

[4]  Russell G. Congalton,et al.  Assessing the accuracy of remotely sensed data : principles and practices , 1998 .

[5]  Eric F. Lambin,et al.  Modelling and monitoring land-cover change processes in tropical regions , 1997 .

[6]  J. C. Ogilvie,et al.  Maximum-likelihood estimation of receiver operating characteristic curve parameters , 1968 .

[7]  Eastman J. Ronald,et al.  RASTER PROCEDURES FOR MULTI-CRITERIA/MULTI-OBJECTIVE DECISIONS , 1995 .

[8]  Russell G. Congalton,et al.  A review of assessing the accuracy of classifications of remotely sensed data , 1991 .

[9]  G. Fischer,et al.  Land-use and land-cover change. Science/research plan , 1995 .

[10]  F. Wu,et al.  Simulation of Land Development through the Integration of Cellular Automata and Multicriteria Evaluation , 1998 .

[11]  J A Swets,et al.  Form of empirical ROCs in discrimination and diagnostic tasks: implications for theory and measurement of performance. , 1986, Psychological bulletin.

[12]  John T. Finn,et al.  A spatial model of land use and forest regeneration in the Ituri forest of northeastern zaire , 1988 .

[13]  Laurence W. Carstensen,et al.  A Measure of Similarity for Cellular Maps , 1987 .

[14]  J. Swets.  Indices of discrimination or diagnostic accuracy: their ROCs and implied models. , 1986, Psychological bulletin.

[15]  R. G. Pontius,et al.  Modeling land-use change in the Ipswich watershed, Massachusetts, USA , 2001 .

[16]  Robert Gilmore Pontius,et al.  Modeling Tropical Land Use Change and Assessing Policies to Reduce Carbon Dioxide Release from Africa , 1994 .

[17]  James P. Egan,et al.  Signal detection theory and ROC analysis , 1975 .

[18]  Monica G. Turner,et al.  Methods to evaluate the performance of spatial simulation models , 1989 .

[19]  J A Swets,et al.  Measuring the accuracy of diagnostic systems. , 1988, Science.

[20]  Lucien Duckstein,et al.  A Multiple Criteria Decision-Making Approach to GIS-Based Land Suitability Evaluation , 1993, Int. J. Geogr. Inf. Sci..

[21]  Hanqin Tian,et al.  Modelling spatial and temporal patterns of tropical land use change , 1995 .

[22]  A. Veldkamp,et al.  CLUE-CR: An integrated multi-scale model to simulate land use change scenarios in Costa Rica , 1996 .

[23]  C. Metz.  Basic principles of ROC analysis. , 1978, Seminars in nuclear medicine.

[24]  Eric F. Lambin,et al.  Spatial modelling of deforestation in southern Cameroon - Spatial disaggregation of diverse deforestation processes , 1997 .