Use of the Weighted Kappa Coefficient in Classification Error Assessment of Thematic Maps

Abstract The weighted Kappa coefficient is applied to the comparison of thematic maps. Weighted Kappa is a useful measure of accuracy when the map classes are ordered, or when the relative seriousness of the different possible errors may vary. The calculation and interpretation of weighted Kappa are demonstrated by two examples from forest surveys. First, the accuracy of thematic site quality maps classified according to an ordinal scale is assessed. Error matrices are derived from map overlays, and two different sets of agreement weights are used for the calculation. Weighted Kappa ranges from 0.34 to 0.55, but it does not differ significantly between two separate areas. Secondly, weighted Kappa is calculated for a tree species cover classified according to a nominal scale. Weights reflecting the economic loss for the forest owner due to erroneous data are used for the computation. The value of weighted Kappa is 0.56.

[1]  H. Braastad,et al.  Growth model computer program for Pinus sylvestris. , 1980 .

[2]  R. G. Oderwald,et al.  Assessing Landsat classification accuracy using discrete multivariate analysis statistical techniques. , 1983 .

[3]  A. Agresti,et al.  Categorical Data Analysis , 1991, International Encyclopedia of Statistical Science.

[4]  T. Allison,et al.  A New Procedure for Assessing Reliability of Scoring EEG Sleep Recordings , 1971 .

[5]  Jacob Cohen.  A Coefficient of Agreement for Nominal Scales , 1960 .

[6]  G. H. Rosenfield,et al.  A coefficient of agreement as a measure of thematic classification accuracy. , 1986 .

[7]  Russell G. Congalton,et al.  Using thematic mapper imagery to examine forest understory , 1990 .

[8]  P. A. Agbu,et al.  Comparisons between spectral mapping units derived from SPOT imager texture and field soil map units , 1991 .

[9]  R. Congalton.  A Quantitative Method to Test for Consistency and Correctness in Photointerpretation , 1983 .

[10]  Dale J. Prediger,et al.  Coefficient Kappa: Some Uses, Misuses, and Alternatives , 1981 .

[11]  J. K. Ord,et al.  The Comparison of Means When Samples Consist of Spatially Autocorrelated Observations , 1975 .

[12]  Giles M. Foody,et al.  On the compensation for chance agreement in image classification accuracy assessment, Photogram , 1992 .

[13]  J. R. Landis,et al.  The measurement of observer agreement for categorical data. , 1977, Biometrics.

[14]  H. Braastad.  Tilvekstmodellprogram for bjørk , 1977 .

[15]  B. Everitt,et al.  Large sample standard errors of kappa and weighted kappa. , 1969 .

[16]  Peng Gong,et al.  An assessment of some factors influencing multispectral land-cover classification , 1990 .

[17]  John T. Finn,et al.  Use of the Average Mutual Information Index in Evaluating Classification Error and Consistency , 1993, Int. J. Geogr. Inf. Sci..

[18]  S. V. Stehman,et al.  Comparison of systematic and random sampling for estimating the accuracy of maps generated from remotely sensed data , 1992 .

[19]  Jacob Cohen,et al.  Weighted kappa: Nominal scale agreement provision for scaled disagreement or partial credit. , 1968 .

[20]  Russell G. Congalton,et al.  A review of assessing the accuracy of classifications of remotely sensed data , 1991 .