Sub-pixel confusion-uncertainty matrix for assessing soft classifications

Abstract The prevailing concerns on ecological and environmental issues, occurring especially at regional to global scales, have prompted significant advances on the use of remote sensing data for the estimation of land cover information at sub-pixel level. However, the quality of such classification products, as well as the performance of the classification protocol employed, are difficult to quantify. This paper had the objectives of 1) reviewing the existing alternatives, while identifying major drawbacks and desirable properties, for sub-pixel accuracy assessment based on cross-comparison matrices, and 2) developing theoretical grounds, for a more general accuracy assessment of soft classifications, that account for the sub-pixel class distribution uncertainty. It was found that, for a sub-pixel confusion matrix to exhibit a diagonalization characteristic that allows identifying a perfect matching case, the agreement measure must be constrained at pixel level, whereas a disagreement measure can take into account the sub-pixel distribution uncertainty, leading to an underspecified problem termed the sub-pixel area allocation problem . It was demonstrated that the sub-pixel area allocation problem admits a unique solution if, and only if, no more than one class is either over- or underestimated at each pixel. In this case, the sub-pixel confusion can be uniquely determined. When no unique solution exists, the space of feasible solutions can be represented by confusion intervals. A new cross-comparison matrix that reports the confusion intervals in the form of a center value plus–minus maximum error was proposed to account for the sub-pixel distribution uncertainty. The new matrix is referred to as sub-pixel confusion–uncertainty matrix (SCM). Sub-pixel accuracy measures were also derived from this matrix. The practical use of the SCM and derived indices was demonstrated in assessing an invasive species detection method and a fuzzy classification of urban land use/land cover through remote sensing procedures.

[1]  Giles M. Foody,et al.  Cross-entropy for the evaluation of the accuracy of a fuzzy land cover classification with fuzzy ground data , 1995 .

[2]  Russell G. Congalton,et al.  An Error Matrix Approach to Fuzzy Accuracy Assessment: The NIMA Geocover Project , 2004 .

[3]  G. Gutman,et al.  The derivation of the green vegetation fraction from NOAA/AVHRR data for use in numerical weather prediction models , 1998 .

[4]  D. Roberts,et al.  Green vegetation, nonphotosynthetic vegetation, and soils in AVIRIS data , 1993 .

[5]  R. G. Pontius,et al.  Categorical Coefficients of Agreement for Assessing Soft-Classified Maps at Multiple Resolutions , 2004 .

[6]  C. Woodcock,et al.  Theory and methods for accuracy assessment of thematic maps using fuzzy sets , 1994 .

[7]  B. R. Baum,et al.  Introduced and naturalized tamarisks in the United States and Canada (Tamaricaceae) , 1967 .

[8]  Hugh G. Lewis,et al.  A generalized confusion matrix for assessing area estimates from remotely sensed data , 2001 .

[9]  R. G. Pontius.  Statistical Methods to Partition Effects of Quantity and Location During Comparison of Categorical Maps at Multiple Resolutions , 2002 .

[10]  Zhuping Sheng,et al.  Saltcedar control and water salvage on the Pecos river, Texas, 1999-2003. , 2005, Journal of environmental management.

[11]  Jacob Cohen.  A Coefficient of Agreement for Nominal Scales , 1960 .

[12]  John A. Richards,et al.  Remote Sensing Digital Image Analysis , 1986 .

[13]  Arnon Karnieli,et al.  Methods for fuzzy classification and accuracy assessment of historical aerial photographs for vegetation change analyses. Part I: Algorithm development , 2006 .

[14]  J. Settle,et al.  Subpixel measurement of tropical forest cover using AVHRR data , 1991 .

[15]  Robert Gilmore Pontius,et al.  A generalized cross‐tabulation matrix to compare soft‐classified maps at multiple resolutions , 2006, Int. J. Geogr. Inf. Sci..

[16]  Gail A. Carpenter,et al.  A Neural Network Method for Mixture Estimation for Vegetation Mapping , 1999 .

[17]  Philip A. Townsend,et al.  A Quantitative Fuzzy Approach to Assess Mapped Vegetation Classifications for Ecological Applications , 2000 .

[18]  Russell G. Congalton,et al.  Assessing the accuracy of remotely sensed data : principles and practices , 1998 .

[19]  J. H. Everitt,et al.  Remote Sensing of Chinese Tamarisk (Tamarix chinensis) and Associated Vegetation , 1990, Weed Science.

[20]  Robert Gilmore Pontius,et al.  Expanding the conceptual , mathematical and practical methods for map comparison , 2007 .

[21]  C. Woodcock,et al.  Resolution dependent errors in remote sensing of cultivated areas , 2006 .

[22]  Kazuyoshi Z. Nanjo,et al.  A model of damage mechanics for the deformation of the continental crust , 2005 .

[23]  Edward J. Milton,et al.  Image Endmembers and the Scene Model , 1999 .

[24]  Peter F. Fisher,et al.  The evaluation of fuzzy membership of land cover classes in the suburban zone , 1990 .

[25]  Russell G. Congalton,et al.  A review of assessing the accuracy of classifications of remotely sensed data , 1991 .

[26]  Stephen V. Stehman,et al.  Design and Analysis for Thematic Map Accuracy Assessment: Fundamental Principles , 1998 .

[27]  Sucharita Gopal,et al.  Fuzzy set theory and thematic maps: accuracy assessment and area estimation , 2000, Int. J. Geogr. Inf. Sci..

[28]  Rasim Latifovic,et al.  Accuracy assessment using sub-pixel fractional error matrices of global land cover products derived from satellite data , 2004 .

[29]  Ranga B. Myneni,et al.  Subpixel burn detection in Moderate Resolution Imaging Spectroradiometer 500‐m data with ARTMAP neural networks , 2005 .

[30]  Elisabetta Binaghi,et al.  A fuzzy set-based accuracy assessment of soft classification , 1999, Pattern Recognit. Lett..

[31]  Christopher Small,et al.  The Landsat ETM+ spectral mixing space , 2004 .

[32]  Giles M. Foody,et al.  Status of land cover classification accuracy assessment , 2002 .