Using Additional Criteria for Measuring the Quality of Predictions and Their Uncertainties in a Digital Soil Mapping Framework

In this paper we introduce additional criteria to assess the quality of digital soil property maps. Soil map quality is estimated on the basis of validating both the accuracy of the predictions and their uncertainties (which are expressed as a prediction interval [PI]). The first criterion is an accuracy measure that is different in form to the usual mean square error (MSE) because it accounts also for the prediction uncertainties. This measure is the spatial average of the statistical expectation of the mean square error of a simulated random value (MSES). The second criterion addresses the quality of the uncertainties which is estimated as the total proportion of the study area where the (1-a)–PI covers the true value. Ideally, this areal proportion equals the nominal value (1 - a). In the Lower Hunter Valley, NSW, Australia, we used both criteria to validate a soil pH map using additional units collected from a probability sample at five depth intervals: 0 to 5, 5 to 15, 15 to 30, 30 to 60, and 60 to 100 cm. For the first depth interval (0–5 cm) in 96% of the area, the 95% PI of pH covered the true value. The root mean squared simulation error (RMSES) at this depth was 1.0 pH units. Generally, the discrepancy between the nominal value and the areal proportion in addition to the RMSES increased with soil depth, indicating largely a growing imprecision of the map and underestimation of the uncertainty with increasing soil depth. In exploring this result, conventional map quality indicators emphasized a combination of bias and imprecision particularly with increasing soil depth. There is great value in coupling conventional map quality indicators with those which we propose in this study as they target the decision making process for improving the precision of maps and their uncertainties. For our study area we discuss options for improving on our results in addition to determining the possibility of extending a similar sampling approach for which multiple soil property maps can be validated concurrently

[1]  G. E. Rayment,et al.  Australian laboratory handbook of soil and water chemical methods. , 1992 .

[2]  D. J. Brus,et al.  Sampling for Natural Resource Monitoring , 2006 .

[3]  Gerard B. M. Heuvelink,et al.  Assessing uncertainty propagation through physically based models of soil water flow and solute transport , 2006 .

[4]  P. A. Burrough,et al.  THE RELATION BETWEEN COST AND UTILITY IN SOIL SURVEY (I–III)1 , 1971 .

[5]  Dimitri Solomatine,et al.  A novel method to estimate model uncertainty using machine learning techniques , 2009 .

[6]  P. A. Burrough,et al.  THE RELATION BETWEEN COST AND UTILITY IN SOIL SURVEY , 1971 .

[7]  Budiman Minasny,et al.  Mapping continuous depth functions of soil carbon storage and available water capacity , 2009 .

[8]  R. M. Lark,et al.  Components of accuracy of maps with special reference to discriminant analysis on remote sensor data , 1995 .

[9]  Peter Finke,et al.  Chapter 39 Quality Assessment of Digital Soil Maps: Producers and Users Perspectives , 2004 .

[10]  F. Bastida,et al.  Past, present and future of soil quality indices : A biological perspective , 2008 .

[11]  Budiman Minasny,et al.  Generation of kth-order random toposequences , 2008, Comput. Geosci..

[12]  Budiman Minasny,et al.  Empirical estimates of uncertainty for mapping continuous depth functions of soil attributes , 2011 .

[13]  P. Beckett,et al.  Quality and Usefulness of Soil Maps , 1968, Nature.

[14]  Robert E. White,et al.  Soils for Fine Wines , 2003 .

[15]  L. Lin,et al.  A concordance correlation coefficient to evaluate reproducibility. , 1989, Biometrics.

[16]  A. McBratney,et al.  A continuum approach to soil classification by modified fuzzy k‐means with extragrades , 1992 .

[17]  Stephen A. J. Bell.  Vegetation of Werakata National Park, Hunter Valley, New South Wales , 2004 .

[18]  Budiman Minasny,et al.  Estimating Pedotransfer Function Prediction Limits Using Fuzzy k-Means with Extragrades , 2010 .

[19]  Robert Tibshirani,et al.  The Elements of Statistical Learning: Data Mining, Inference, and Prediction, 2nd Edition , 2001, Springer Series in Statistics.

[20]  D. J. Brus,et al.  A sampling scheme for estimating the mean extractable phosphorus concentration of fields for environmental regulation , 1999 .

[21]  Brett Whelan,et al.  Measuring the quality of digital soil maps using information criteria , 2001 .

[22]  Budiman Minasny,et al.  A conditioned Latin hypercube method for sampling in the presence of ancillary information , 2006, Comput. Geosci..

[23]  Sabine Grunwald,et al.  Multi-criteria characterization of recent digital soil mapping and modeling approaches , 2009 .

[24]  B. Minasny,et al.  Bottom-up digital soil mapping. II. Soil series classes , 2011 .

[25]  Durga L. Shrestha,et al.  Machine learning approaches for estimation of prediction interval for the model output , 2006, Neural Networks.

[26]  D. J. Brus,et al.  Random sampling or geostatistical modelling? Choosing between design-based and model-based sampling strategies for soil (with discussion) , 1997 .

[27]  Budiman Minasny,et al.  Uncertainty analysis for pedotransfer functions , 2002 .

[28]  Gerard B. M. Heuvelink,et al.  Sampling for validation of digital soil maps , 2011 .