Using Volunteered Data in Land Cover Map Validation: Mapping West African Forests

Accuracy assessment should be a fundamental part of a programme that maps land cover from remotely sensed imagery but this activity is often constrained by the lack of high quality ground reference data. Here, two sources of volunteered data are used to illustrate the potential of amateur or neogeographical activity in map validation. Ground based photographs acquired from an internet-based collaborative project and interpreted by a set of four further volunteers provided the reference data to support evaluation of the Globcover map's representation of tropical forest in West Africa. Although the results highlight some concerns with volunteered data, notably the low levels of inter-volunteer agreement they also show that such imperfect data may be used to derive credible estimates on accuracy from both a site and non-site specific perspective. Specifically, the estimates of the producer's accuracy of forest and of forest extent derived from the free and volunteered data using a latent class model were of comparable magnitude to those derived in a formal validation by experts; the estimate of forest extent was within 1.38-9.08% of reference estimates while the difference in estimated producer's accuracy from that derived in an authoritative assessment of map accuracy was 2.82% and 0.34% for the forest and non-forest classes respectively.

[1]  C. Kleinn,et al.  Uncertainties of forest area estimates caused by the minimum crown cover criterion , 2012, Environmental Monitoring and Assessment.

[2]  Damien Sulla-Menashe,et al.  A global land-cover validation data set, part I: fundamental design principles , 2012 .

[3]  Giles M. Foody,et al.  Status of land cover classification accuracy assessment , 2002 .

[4]  Steffen Fritz,et al.  Geo-Wiki: An online platform for improving global land cover , 2012, Environ. Model. Softw..

[5]  Giles M. Foody,et al.  Sample size determination for image classification accuracy assessment and comparison , 2009 .

[6]  Giles M. Foody,et al.  Local characterization of thematic classification accuracy through spatially constrained confusion matrices , 2005 .

[7]  Jacob Cohen A Coefficient of Agreement for Nominal Scales , 1960 .

[8]  Giles M. Foody,et al.  On the compensation for chance agreement in image classification accuracy assessment, Photogram , 1992 .

[9]  Y. Yamagata,et al.  Validating land cover maps with Degree Confluence Project information , 2006 .

[10]  Michael F. Goodchild,et al.  Please Scroll down for Article International Journal of Digital Earth Crowdsourcing Geographic Information for Disaster Response: a Research Frontier Crowdsourcing Geographic Information for Disaster Response: a Research Frontier , 2022 .

[11]  Patrick Weber,et al.  The Rise of the GIS Volunteer , 2012, Trans. GIS.

[12]  J. Cihlar Land cover mapping of large areas from satellites: Status and research priorities , 2000 .

[13]  I Yang,et al.  Latent variable modeling of diagnostic accuracy. , 1997, Biometrics.

[14]  D. H. Card Using known map category marginal frequencies to improve estimates of thematic map accuracy , 1982 .

[15]  Jeroen K. Vermunt,et al.  Log-Linear Models for Event Histories , 1997 .

[16]  Richard A. Wadsworth,et al.  What is Land Cover? , 2005 .

[17]  M. Hansen,et al.  The DISCover validation image interpretation process , 1999 .

[18]  C. Woodcock,et al.  Making better use of accuracy data in land change studies: Estimating accuracy and area and quantifying uncertainty using stratified estimation , 2013 .

[19]  Ronald R Rindfuss,et al.  Developing a science of land change: challenges and methodological issues. , 2004, Proceedings of the National Academy of Sciences of the United States of America.

[20]  G. Foody Assessing the accuracy of land cover change with imperfect ground reference data , 2010 .

[21]  Ling Bai Comparison and validation of five land cover products over the African continent , 2010 .

[22]  Mark J. Carlotto,et al.  Effect of errors in ground truth on classification accuracy , 2009 .

[23]  F. J. Gallego Remote sensing and land cover area estimation , 2004 .

[24]  M. Goodchild Citizens as sensors: the world of volunteered geography , 2007 .

[25]  Stephen V. Stehman,et al.  Sampling designs for accuracy assessment of land cover , 2009 .

[26]  Giles M. Foody,et al.  Harshness in image classification accuracy assessment , 2008 .

[27]  Ronald E. McRoberts,et al.  Satellite image-based maps: Scientific inference or pretty pictures? , 2011 .

[28]  Lorenzo Bruzzone,et al.  Quality assessment of classification and cluster maps without ground truth knowledge , 2005, IEEE Transactions on Geoscience and Remote Sensing.

[29]  Giles M. Foody,et al.  Latent Class Modeling for Site- and Non-Site-Specific Classification Accuracy Assessment Without Ground Data , 2012, IEEE Transactions on Geoscience and Remote Sensing.

[30]  R. Pontius,et al.  Death to Kappa: birth of quantity disagreement and allocation disagreement for accuracy assessment , 2011 .

[31]  Robert Gilmore Pontius,et al.  Assessing a predictive model of land change using uncertain data , 2010, Environ. Model. Softw..

[32]  Ola Ahlqvist,et al.  Extending post-classification change detection using semantic similarity metrics to overcome class heterogeneity: A study of 1992 and 2001 U.S. National Land Cover Database changes , 2008 .

[33]  David L.B. Jupp,et al.  The stability of global estimates from confusion matrices , 1989 .

[34]  Russell G. Congalton,et al.  Assessing the accuracy of remotely sensed data : principles and practices , 1998 .