Predictive performance of mobile vis-near infrared spectroscopy for key soil properties at different geographical scales by using spiking and data mining techniques

Abstract The development of accurate visible and near infrared (vis-NIR) spectroscopy calibration models for selected soil properties based on mobile measurements is essential for site specific soil management at fine sampling scale. The objective of the present study was to compare the mobile and laboratory prediction performance of vis-NIR spectroscopy for total nitrogen (TN), total carbon (TC) and soil moisture content (MC) of field soil samples based on single field (SFD), two-field dataset (TFD), UK national dataset (UND) and European continental dataset (ECD) calibration models developed with linear and nonlinear data mining techniques including spiking. Fresh soil samples collected from fields in the UK, Czech Republic, Germany, Denmark and the Netherlands were scanned with a fibre-type vis-NIR spectrophotometer (tec5 Technology for Spectroscopy, Germany), with a spectral range of 305–2200 nm. After dividing spectra into calibration (75%) and validation (25%) sets, spectra in the calibration set were subjected to three multivariate calibration models, including the partial least squares regression (PLSR), multivariate adaptive regression splines (MARS) and support vector machines (SVM), with leave-one-out cross-validation to establish calibration models of TN, TC and MC. Results showed that the best model performance in cross-validation was obtained with MARS methods for the majority of dataset scales used, whereas the lowest model performance was obtained with the SFD. The effect of spiking was significant and the best model performance in general term was obtained when local samples collected from two target fields in the UK were spiked with the ECD, with coefficients of determination (R2) values of 0.96, 0.98 and 0.93, root mean square error (RMSE) of 0.01, 0.1 and 1.75, and ratio of performance to interquartile distance (RPIQ) of 7.46, 6.57 and 3.98, for TC, TN and MC, respectively. Therefore, these results suggest that ECD vis-NIR MARS calibration models can be successfully used to predict TN, TC and MC under both laboratory and mobile measurement conditions.

[1]  Gabriele Buttafuoco,et al.  Studying the relationship between water-induced soil erosion and soil organic matter using Vis–NIR spectroscopy and geomorphological analysis: A case study in southern Italy , 2013 .

[2]  R. A. Potyrailo.  On-line Measurement , 2001 .

[3]  W. Hively,et al.  Visible-near infrared reflectance spectroscopy for assessment of soil properties in a semi-arid area of Turkey , 2010 .

[4]  M. Vohland,et al.  Estimation accuracies of near infrared spectroscopy for general soil properties and enzyme activities for two forest sites along three transects , 2017 .

[5]  M. R. Malekia,et al.  On-the-go variable-rate phosphorus fertilisation based on a visible and near-infrared soil sensor , 2007 .

[6]  M. Forina,et al.  Multivariate calibration. , 2007, Journal of chromatography. A.

[7]  Charlie Chen,et al.  Digitally mapping the information content of visible–near infrared spectra of surficial Australian soils , 2011 .

[8]  Yiyun Chen,et al.  Estimating Soil Organic Carbon Using VIS/NIR Spectroscopy with SVMR and SPA Methods , 2014, Remote. Sens..

[9]  Sabine Grunwald,et al.  Comparison of multivariate methods for inferential modeling of soil carbon using visible/near-infrared spectra , 2008 .

[10]  David J. Brown.  Using a global VNIR soil-spectral library for local soil characterization and landscape modeling in a 2nd-order Uganda watershed , 2007 .

[11]  César Guerrero,et al.  Spiking of NIR regional models using samples from target sites: effect of model size on prediction accuracy. , 2010 .

[12]  Pijush Samui,et al.  Multivariate Adaptive Regression Spline (Mars) for Prediction of Elastic Modulus of Jointed Rock Mass , 2012, Geotechnical and Geological Engineering.

[13]  M. Chodak,et al.  Near infrared spectroscopy—A tool for chemical properties and organic matter assessment of afforested mine soils , 2014 .

[14]  H. Ramon,et al.  On-line measurement of some selected soil properties using a VIS–NIR sensor , 2007 .

[15]  D. Ruppert.  The Elements of Statistical Learning: Data Mining, Inference, and Prediction , 2004 .

[16]  A. Mouazen,et al.  Calibration of visible and near infrared spectroscopy for soil analysis at the field scale on three European farms , 2011 .

[17]  K. Shepherd,et al.  Development of Reflectance Spectral Libraries for Characterization of Soil Properties , 2002 .

[18]  Budiman Minasny,et al.  Evaluating near infrared spectroscopy for field prediction of soil properties. , 2009 .

[19]  H. Ramon,et al.  Towards development of on-line soil moisture content sensor using a fibre-type NIR spectrophotometer , 2005 .

[20]  Abdul Mounem Mouazen,et al.  Influence of the number of samples on prediction error of visible and near infrared spectroscopy of selected soil properties at the farm scale , 2011 .

[21]  B. Stenberg,et al.  Near‐infrared spectroscopy for within‐field soil characterization: small local calibrations compared with national libraries spiked with local samples , 2010 .

[22]  A. McBratney,et al.  Critical review of chemometric indicators commonly used for assessing the quality of the prediction of soil attributes by NIR spectroscopy , 2010 .

[23]  Annamaria Castrignanò,et al.  Laboratory-based Vis–NIR spectroscopy and partial least square regression with spatially correlated errors for predicting spatial variation of soil organic matter content , 2015 .

[24]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[25]  Xiang Yu,et al.  Evaluation of MLSR and PLSR for estimating soil element contents using visible/near-infrared spectroscopy in apple orchards on the Jiaodong peninsula , 2016 .

[26]  Abdul Mounem Mouazen,et al.  Effect of spiking strategy and ratio on calibration of on-line visible and near infrared soil sensor for measurement in European farms , 2013 .

[27]  Henning Buddenbaum,et al.  Estimation of soil salinity using three quantitative methods based on visible and near-infrared reflectance spectroscopy: a case study from Egypt , 2015, Arabian Journal of Geosciences.

[28]  R. V. Rossel,et al.  In situ measurements of soil colour, mineral composition and clay content by vis–NIR spectroscopy , 2009 .

[29]  Á. Felicísimo,et al.  Mapping landslide susceptibility with logistic regression, multiple adaptive regression splines, classification and regression trees, and maximum entropy methods: a comparative study , 2013, Landslides.

[30]  Tereza Zádorová,et al.  Uncertainty propagation in VNIR reflectance spectroscopy soil organic carbon mapping , 2013 .

[31]  J. Friedman.  Multivariate adaptive regression splines , 1990 .

[32]  Kurt Hornik,et al.  Misc Functions of the Department of Statistics (e1071), TU Wien , 2014 .

[33]  Thomas Martinetz,et al.  Reliability of Cross-Validation for SVMs in High-Dimensional, Low Sample Size Scenarios , 2008, ICANN.

[34]  Johan A. K. Suykens,et al.  Least Squares Support Vector Machine Classifiers , 1999, Neural Processing Letters.

[35]  M. Vohland,et al.  Comparing different multivariate calibration methods for the determination of soil organic carbon pools with visible to near infrared spectroscopy , 2011 .

[36]  Abdul Mounem Mouazen,et al.  Comparison between artificial neural network and partial least squares for on-line visible and near infrared spectroscopy measurement of soil organic carbon, pH and clay content , 2015 .

[37]  Cristine L. S. Morgan,et al.  In Situ Characterization of Soil Clay Content with Visible Near‐Infrared Diffuse Reflectance Spectroscopy , 2007 .

[38]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[39]  Abdul Mounem Mouazen,et al.  On-line visible and near infrared spectroscopy for in-field phosphorous management , 2016 .

[40]  G. Ayoko,et al.  The use of reflectance visible-NIR spectroscopy to predict seasonal change of trace metals in suspended solids of Changjiang River , 2013 .

[41]  Yufeng Ge,et al.  Moisture insensitive prediction of soil properties from VNIR reflectance spectra based on external parameter orthogonalization , 2016 .

[42]  Alexandros Karatzoglou,et al.  Kernel-based machine learning for fast text mining in R , 2010, Comput. Stat. Data Anal..

[43]  J. Lowenberg‐DeBoer,et al.  Precision Agriculture and Sustainability , 2004, Precision Agriculture.

[44]  Bo Stenberg,et al.  Improving the prediction performance of a large tropical vis‐NIR spectroscopic soil library from Brazil by clustering into smaller subsets or use of data mining calibration techniques , 2014 .

[45]  Ramón Fernández Astudillo,et al.  Uncertainty Propagation , 2011, Robust Speech Recognition of Uncertain or Missing Data.

[46]  Soheil Mehralian,et al.  Comparison between Artificial Neural Network and neuro-fuzzy for gold price prediction , 2013, 2013 13th Iranian Conference on Fuzzy Systems (IFSC).

[47]  Yufeng Ge,et al.  Comparison and detection of total and available soil carbon fractions using visible/near infrared diffuse reflectance spectroscopy. , 2011 .

[48]  K. Shepherd,et al.  Global soil characterization with VNIR diffuse reflectance spectroscopy , 2006 .

[49]  Henning Buddenbaum,et al.  Estimating the soil clay content and organic matter by means of different calibration methods of vis-NIR diffuse reflectance spectroscopy , 2016 .

[50]  Keith D. Shepherd,et al.  The prediction of soil carbon fractions using mid-infrared-partial least square analysis , 2007 .

[51]  Rebecca L. Whetton,et al.  Machine learning based prediction of soil total nitrogen, organic carbon and moisture content by using VIS-NIR spectroscopy , 2016 .

[52]  Mats Söderström,et al.  Increased sample point density in farm soil mapping by local calibration of visible and near infrared prediction models , 2010 .

[53]  Kurt Hornik,et al.  Misc Functions of the Department of Statistics, ProbabilityTheory Group (Formerly: E1071), TU Wien , 2015 .

[54]  Abdul Mounem Mouazen,et al.  Site-specific land management of cereal crops based on management zone delineation by proximal soil sensing , 2013 .

[55]  Abdul Mounem Mouazen,et al.  Assessment of soil organic carbon at local scale with spiked NIR calibrations: effects of selection and extra-weighting on the spiking subset , 2013 .

[56]  Mapping landslide susceptibility in The Faroe Islands , 2008 .

[57]  K. Sudduth,et al.  Geographic Operating Range Evaluation of a NIR Soil Sensor , 1996 .

[58]  R. V. Rossel,et al.  Using data mining to model and interpret soil diffuse reflectance spectra. , 2010 .

[59]  L. Miska,et al.  Evaluation of current statistical approaches for predictive geomorphological mapping , 2005 .

[60]  Robert Tibshirani,et al.  The Elements of Statistical Learning: Data Mining, Inference, and Prediction, 2nd Edition , 2001, Springer Series in Statistics.

[61]  Rick L. Lawrence,et al.  Comparing local vs. global visible and near-infrared (VisNIR) diffuse reflectance spectroscopy (DRS) calibrations for the prediction of soil clay, organic C and inorganic C , 2008 .

[62]  R. V. Rossel,et al.  Visible and near infrared spectroscopy in soil science , 2010 .

[63]  H. Ramon,et al.  Comparison among principal component, partial least squares and back propagation neural network analyses for accuracy of measurement of selected soil properties with visible and near infrared spectroscopy , 2010 .