A large‐area, spatially continuous assessment of land cover map error and its impact on downstream analyses

Land cover maps increasingly underlie research into socioeconomic and environmental patterns and processes, including global change. It is known that map errors impact our understanding of these phenomena, but quantifying these impacts is difficult because many areas lack adequate reference data. We used a highly accurate, high-resolution map of South African cropland to assess (1) the magnitude of error in several current generation land cover maps, and (2) how these errors propagate in downstream studies. We first quantified pixel-wise errors in the cropland classes of four widely used land cover maps at resolutions ranging from 1 to 100 km, and then calculated errors in several representative "downstream" (map-based) analyses, including assessments of vegetative carbon stocks, evapotranspiration, crop production, and household food security. We also evaluated maps' spatial accuracy based on how precisely they could be used to locate specific landscape features. We found that cropland maps can have substantial biases and poor accuracy at all resolutions (e.g., at 1 km resolution, up to ∼45% underestimates of cropland (bias) and nearly 50% mean absolute error (MAE, describing accuracy); at 100 km, up to 15% underestimates and nearly 20% MAE). National-scale maps derived from higher-resolution imagery were most accurate, followed by multi-map fusion products. Constraining mapped values to match survey statistics may be effective at minimizing bias (provided the statistics are accurate). Errors in downstream analyses could be substantially amplified or muted, depending on the values ascribed to cropland-adjacent covers (e.g., with forest as adjacent cover, carbon map error was 200%-500% greater than in input cropland maps, but ∼40% less for sparse cover types). The average locational error was 6 km (600%). These findings provide deeper insight into the causes and potential consequences of land cover map error, and suggest several recommendations for land cover map users.

[1]  Thomas J. Fuchs,et al.  A generalized computer vision approach to mapping crop fields in heterogeneous agricultural landscapes , 2016 .

[2]  Thomas Blaschke,et al.  A comparison of three image-object methods for the multiscale analysis of landscape structure , 2003 .

[3]  D. Lettenmaier,et al.  A simple hydrologically based model of land surface water and energy fluxes for general circulation models , 1994 .

[4]  Kathleen Neumann,et al.  Challenges in using land use and land cover data for global change studies , 2011 .

[5]  Steffen Fritz,et al.  The Need for Improved Maps of Global Cropland , 2013 .

[6]  Hankui K. Zhang,et al.  Meta-discoveries from a synthesis of satellite-based land-cover mapping research , 2014 .

[7]  E. Hand.  Startup liftoff. , 2015, Science.

[8]  Steffen Fritz,et al.  Identifying and quantifying uncertainty and spatial disagreement in the comparison of Global Land Cover for different applications , 2008 .

[9]  Miska Luoto,et al.  PREDICTING BIRD SPECIES RICHNESS USING REMOTE SENSING IN BOREAL AGRICULTURAL‐FOREST MOSAICS , 2004 .

[10]  Kelly K. Caylor,et al.  Reconciling agriculture, carbon and biodiversity in a savannah transformation frontier , 2016, Philosophical Transactions of the Royal Society B: Biological Sciences.

[11]  Philip K. Thornton,et al.  High carbon and biodiversity costs from converting Africa’s wet savannahs to cropland , 2015 .

[12]  K. Baylis,et al.  Goldilocks and the Raster Grid: Selecting Scale when Evaluating Conservation Programs , 2016, PloS one.

[13]  Ashley E. Larsen,et al.  Taken as a given: evaluating the accuracy of remotely sensed crop data in the USA. , 2015 .

[14]  Jianjun Ge,et al.  Impacts of land use/cover classification accuracy on regional climate simulations , 2007 .

[15]  Sean Sloan,et al.  Major atmospheric emissions from peat fires in Southeast Asia during non-drought years: evidence from the 2013 Sumatran fires , 2014, Scientific Reports.

[16]  Mao Ning Tuanmu,et al.  A global 1‐km consensus land‐cover product for biodiversity and ecosystem modelling , 2014 .

[17]  Calogero Carletto,et al.  The Emperor has no Data ! Agricultural Statistics in Sub-Saharan Africa , 2013 .

[18]  Tom Evans,et al.  Mapping Cropland in Smallholder-Dominated Savannas: Integrating Remote Sensing Techniques and Probabilistic Modeling , 2015, Remote. Sens..

[19]  E. Wood,et al.  A Drought Monitoring and Forecasting System for Sub-Sahara African Water Resources and Food Security , 2014 .

[20]  Laurence C. Smith,et al.  How well do we know northern land cover? Comparison of four global vegetation and wetland products with a new ground‐truth database for West Siberia , 2007 .

[21]  Luis Roman Carrasco,et al.  Agricultural intensification escalates future conservation costs , 2013, Proceedings of the National Academy of Sciences.

[22]  James S. Gerber,et al.  Subnational distribution of average farm size and smallholder contributions to global food production , 2016 .

[23]  N. Ramankutty,et al.  Farming the planet: 2. Geographic distribution of crop areas, yields, physiological types, and net primary production in the year 2000 , 2008 .

[24]  Steffen Fritz,et al.  A Unified Cropland Layer at 250 m for Global Agriculture Monitoring , 2016, Data.

[25]  Mark Rounsevell,et al.  Please Scroll down for Article International Journal of Geographical Information Science Exploring Spatial Data Uncertainties in Land-use Change Scenarios Exploring Spatial Data Uncertainties in Land-use Change Scenarios , 2022 .

[26]  M. Hardy,et al.  Rainfed Farming Systems in South Africa , 2011 .

[27]  Debra K. Meyer,et al.  Completion of the National Land Cover Database (NLCD) 1992–2001 Land Cover Change Retrofit product , 2009 .

[28]  Grégoire Dubois,et al.  Monitoring land cover changes in African protected areas in the 21st century , 2013, Ecol. Informatics.

[29]  E. Lambin,et al.  Dynamics of Land-Use and Land-Cover Change in Tropical Regions , 2003 .

[30]  C. Woodcock,et al.  Resolution dependent errors in remote sensing of cultivated areas , 2006 .

[31]  S. Openshaw.  A million or so correlation coefficients : three experiments on the modifiable areal unit problem , 1979 .

[32]  N. Ramankutty,et al.  Farming the planet: 1. Geographic distribution of global agricultural lands in the year 2000 , 2008 .

[33]  Pinki Mondal,et al.  Mapping cropping intensity of smallholder farms: A comparison of methods using multiple sensors , 2013 .

[34]  Atul K. Jain,et al.  CO2 emissions from land‐use change affected more by nitrogen cycle, than by the choice of land‐cover data , 2013, Global change biology.

[35]  C. Woodcock,et al.  Making better use of accuracy data in land change studies: Estimating accuracy and area and quantifying uncertainty using stratified estimation , 2013 .

[36]  Navin Ramankutty,et al.  Mind the gap: how do climate and agricultural management explain the ‘yield gap’ of croplands around the world? , 2010 .

[37]  Gorm E. Shackelford,et al.  landscapes: hotspots of conflict between agriculture and nature , 2015 .

[38]  Steffen Fritz,et al.  Geo-Wiki: An online platform for improving global land cover , 2012, Environ. Model. Softw..

[39]  Tyler J. Lark,et al.  Cropland expansion outpaces agricultural and biofuel policies in the United States , 2015 .

[40]  Steffen Fritz,et al.  Cropland for sub‐Saharan Africa: A synergistic approach using five land cover data sets , 2011 .

[41]  Catherine Linard,et al.  Assessing the use of global land cover data for guiding large area population distribution modelling , 2010, GeoJournal.

[42]  Steffen Fritz,et al.  Improved global cropland data as an essential ingredient for food security , 2015 .

[43]  Eric F. Lambin,et al.  Modelling and monitoring land-cover change processes in tropical regions , 1997 .

[44]  J. Anderies,et al.  Governance and the Capacity to Manage Resilience in Regional Social-Ecological Systems , 2006 .

[45]  Tom Evans,et al.  Dependency Provenance in Agent Based Modeling , 2013, 2013 IEEE 9th International Conference on e-Science.

[46]  S. Fritz,et al.  Comparison of global and regional land cover maps with statistical information for the agricultural domain in Africa , 2010 .

[47]  Gerard B M Heuvelink,et al.  Effect of land cover data on nitrous oxide inventory in fen meadows. , 2008, Journal of environmental quality.

[48]  Holly K. Gibbs,et al.  New IPCC Tier-1 Global Biomass Carbon Map for the Year 2000 , 2008 .

[49]  S. Dark,et al.  The modifiable areal unit problem (MAUP) in physical geography , 2007 .

[50]  Ben Collen,et al.  Global effects of land use on local terrestrial biodiversity , 2015, Nature.

[51]  G. Hay,et al.  Remote Sensing Contributions to the Scale Issue , 1999 .

[52]  Matthias Drusch,et al.  Sentinel-2: ESA's Optical High-Resolution Mission for GMES Operational Services , 2012 .

[53]  T. Kuemmerle,et al.  The Emerging Soybean Production Frontier in Southern Africa: Conservation Challenges and the Role of South‐South Telecouplings , 2016 .

[54]  Giles M. Foody,et al.  Good practices for estimating area and assessing accuracy of land change , 2014 .

[55]  L. D. Estes,et al.  A platform for crowdsourcing the creation of representative, accurate landcover maps , 2016, Environ. Model. Softw..

[56]  Jonah Busch,et al.  On international equity in reducing emissions from deforestation , 2010 .

[57]  Damien Sulla-Menashe,et al.  MODIS Collection 5 global land cover: Algorithm refinements and characterization of new datasets , 2010 .

[58]  R. Tibshirani,et al.  Generalized Additive Models , 1986 .

[59]  Rick Mueller,et al.  Mapping global cropland and field size , 2015, Global change biology.

[60]  Giles M. Foody,et al.  Status of land cover classification accuracy assessment , 2002 .

[61]  Sydney Gourlay,et al.  From Guesstimates to Gpstimates: Land Area Measurement and Implications for Agricultural Analysis , 2013 .

[62]  Seth Pettie,et al.  Mind the gap , 2006, Nature Reviews Drug Discovery.

[63]  Mark Rounsevell,et al.  The limitations of spatial land use data in environmental analysis , 2006 .

[64]  D. Marceau.  The Scale Issue in the Social and Natural Sciences , 1999 .

[65]  Damien Sulla-Menashe,et al.  A global land-cover validation data set, part I: fundamental design principles , 2012 .

[66]  Aaron Moody,et al.  The influence of scale and the spatial characteristics of landscapes on land-cover mapping using remote sensing , 1995, Landscape Ecology.

[67]  Steffen Fritz,et al.  Mapping Priorities to Focus Cropland Mapping Activities: Fitness Assessment of Existing Global, Regional and National Cropland Maps , 2015, Remote. Sens..

[68]  M. Wimberly,et al.  Recent land use change in the Western Corn Belt threatens grasslands and wetlands , 2013, Proceedings of the National Academy of Sciences.

[69]  Steffen Fritz,et al.  Highlighting continued uncertainty in global land cover maps for the user community , 2011 .

[70]  Mark New,et al.  Ensemble forecasting of species distributions. , 2007, Trends in ecology & evolution.

[71]  Thomas Berger,et al.  Research, part of a Special Feature on Empirical based agent-based modeling Creating Agents and Landscapes for Multiagent Systems from Random Samples , 2006 .

[72]  Patrick Hostert,et al.  Challenges and opportunities in mapping land use intensity globally , 2013, Current opinion in environmental sustainability.

[73]  Michael Oppenheimer,et al.  Projected climate impacts to South African maize and wheat production in 2055: a comparison of empirical and mechanistic modeling approaches , 2013, Global change biology.

[74]  Chengquan Huang,et al.  Conservation policy and the measurement of forests , 2016 .

[75]  Thomas G. Dietterich,et al.  Map Misclassification Can Cause Large Errors in Landscape Pattern Indices: Examples from Habitat Fragmentation , 2006, Ecosystems.

[76]  Eric F. Lambin,et al.  Estimating the world's potentially available cropland using a bottom-up approach , 2013 .

[77]  Martin Herold,et al.  A global land-cover validation data set, II: augmenting a stratified sampling design to estimate accuracy by region and land-cover class , 2012 .

[78]  James W. Jones,et al.  Uncertainty in Simulating Wheat Yields Under Climate Change , 2013 .

[79]  Alfons Balmann,et al.  Post‐Soviet cropland abandonment and carbon sequestration in European Russia, Ukraine, and Belarus , 2013 .

[80]  Mathias Disney,et al.  Impact of land cover uncertainties on estimates of biospheric carbon fluxes , 2008 .

[81]  F. Giorgi,et al.  Calculation of average, uncertainty range, and reliability of regional climate changes from AOGCM simulations via the reliability ensemble averaging (REA) method , 2002 .

[82]  Curtis E. Woodcock,et al.  The effect of spatial resolution on the ability to monitor the status of agricultural lands , 1997 .

[83]  Maria Cristina Rulli,et al.  Global land and water grabbing , 2013, Proceedings of the National Academy of Sciences.

[84]  Jin Chen,et al.  Global land cover mapping at 30 m resolution: A POK-based operational approach , 2015 .

[85]  S. Carpenter,et al.  Solutions for a cultivated planet , 2011, Nature.

[86]  S. Goetz,et al.  Mapping and monitoring carbon stocks with satellite observations: a comparison of methods , 2009, Carbon balance and management.

[87]  G. Powell,et al.  High-resolution forest carbon stocks and emissions in the Amazon , 2010, Proceedings of the National Academy of Sciences.