Accounting for the effects of water and the environment on proximally sensed vis–NIR soil spectra and their calibrations

Summary Visible–near infrared (vis–NIR) spectroscopy can be used to estimate soil properties effectively using spectroscopic calibrations derived from data contained in spectroscopic databases. However, these calibrations cannot be used with proximally sensed (field) spectra because the spectra in these databases are recorded in the laboratory and are different to field spectra. Environmental factors, such as the amount of water in the soil, ambient light, temperature and the condition of the soil surface, cause the differences. Here, we investigated the use of direct standardization (DS) to remove those environmental factors from field spectra. We selected 104 sensing (sampling) sites from nine paddy fields in Zhejiang province, China. At each site, vis–NIR spectra were recorded with a portable spectrometer. The soils were also sampled to record their spectra under laboratory conditions and to measure their soil organic matter (SOM) content. The resulting data were divided into training and validation sets. A subset of the corresponding field and laboratory spectra in the training set (the transfer set) was used to derive the DS transfer matrix, which characterizes the differences between the field and laboratory spectra. Using DS, we transferred the field spectra of the validation samples so that they acquired the characteristics of spectra that were measured in the laboratory. A partial least squares regression (PLSR) of SOM on the laboratory spectra of the training set was then used to predict both the original field spectra and the DS-transferred field spectra. The assessment statistics of the predictions were improved from R2 = 0.25 and RPD = 0.35 to R2 = 0.69 and RPD = 1.61. We also performed independent predictions of SOM on the DS-transferred field spectra with a PLSR derived using the Chinese soil spectroscopic database (CSSD), which was developed in the laboratory. The R2 and RPD values of these predictions were 0.70 and 1.79, respectively. Predictions of SOM with the DS-transferred field spectra were more accurate than those treated with external parameter orthogonalisation (EPO), and more accurate than predictions made by spiking. Our results show that DS can effectively account for the effects of water and environmental factors on field spectra and improve predictions of SOM. DS is conceptually straightforward and allows the use of calibrations made with laboratory-measured spectra to predict soil properties from proximally sensed (field) spectra, without needing to recalibrate the models.