Unbiased estimates of class proportions from thematic maps

A statistical overview is presented (or estimating various components related to map accuracy assessment. The em- phasis is on estimation of the true proportions of each map class under several common sampling designs. A complete system is presented for relating alternative approaches and estimators using standard rules of probability theory. Covari- ance matrices for estimates of true class proportions are de- rived in the Appendices for each of the sampling designs discussed. Introduction The traditional approach to assessing map accuracy is based on the assumption that each pixel within the map has a cor- rect classification, but errors in classification can be made. Unbiased estimation of true class proportions is based on a subsample where the true classification is obtained by an in- fallible and expensive method so that both the true and map classifications are known. The subsample results are then ex- trapolated to the entire map utilizing statistical procedures that are discussed below. Tenenbein (1972) gives an early development based on a double sampling scheme. Card (1982) recognizes that appli- cations to thematic maps are privy to complete knowledge of the map class marginal proportions, and he suggests appro- priate alterations to Tenenbein's method. Grassia and Sund- berg (1982) present a method for calibrating sorting machines that can also be applied to thematic maps. While Card's method requires a subsample after the map is made, Grassia and Sundberg's approach could utilize the training data ac- quired to calibrate the classification algorithm. Bauer et al. (1978) and Hay (1988) are examples from the remote sensing literature that are similar to Grassia and Sundberg's method. Czaplewski and Catts (1992) compare Grassia and Sundberg's (1982) estimator and Tenenbein's (1972) estimator in a simulation study. While this compari- son is valid, it does not recognize the improvement made to Tenenbein's method by Card (1982) for the special circum- stances of thematic map accuracy assessment. Green et al. (1993) and Card (1982) present a Bayesian derivation of the producer's risk, a term used (Aronoff, 1982; Aronoff, 1985) to describe the conditional probability of the map class given the true class, say p(m I t). Prisley and Smith (1987) and Story and Congalton (1986) both state that the user's accu- racy, which is analogous to the consumer's risk and p(t I m), is commonly estimated by dividing the number of sample observations correctly classified as category X by the total number of category X ground samples. Story and Congalton (1986) state that an alternative method is to divide the num- ber of correctly classified samples of category X by the tolal number of samples classified as category X. This paper also looks at alternative methods for comput-