Comparing CalReg performance with other multivariate methods for estimating selected soil properties from Moroccan agricultural regions using NIR spectroscopy

Abstract Nowadays, near-infrared spectroscopy (NIRS) is widely used as an alternative method for analysing soil attributes. The successful determination of soil attributes by this method depends on the selection of an appropriate multivariate calibration technique. In this study, four multivariate techniques, including partial least squares regression (PLSR), memory-based learning (MBL), backpropagation neural network (BPNN) and a new proposed method named ‘combined classification & regression method’ (CalReg) were compared in terms of accuracy in predicting soil properties, including extractable calcium (Caex), extractable iron (Feex), zinc (Zn), extractable magnesium (Mgex), extractable potassium (Kex), phosphorus extracted by Olson method (P-Olson), calcium carbonate (CaCO3), soil organic carbon (SOC), total nitrogen (TN), clay, silt, sand, cation exchange capacity (CEC), and pH. A total of 660 soil samples collected from six Moroccan agricultural regions were used as the dataset for the laboratory’s calibration and validation procedure. Prediction accuracy was quantified using four statistical metrics of accuracy including the coefficient of determination (R2), root mean squared error (RMSE), the ratio of performance to interquartile distance (RPIQ) and residual prediction deviation (RPD). Both calibration and data test sets showed that the CalReg model outperformed the other tested models for SOC, TN, Caex, CEC, P-Olson, Kex and pH predictions. At the same time, MBL outperformed the other models for Feex, Zn, silt, sand and clay prediction. Moreover, BPNN outperformed the other models for Mgex and CaCO3 predictions. The best predictions were obtained by the CalReg model for SOC (R2ts = 0.92; RMSEts = 0.29; RPD = 4.73, RPIQ =5.71), TN (R2ts = 0.90; RMSEts = 364.80; RPD = 3.39, RPIQ =3.16), Caex (R2ts = 0.88; RMSEts = 3.25; RPD = 3.33, RPIQ = 4.40), CaCO3 (R2ts = 0.89; RMSEts = 3.86; RPD = 3.18, RPIQ = 3.95), and clay (R2ts = 0.88; RMSEts = 3.95; RPD = 3.25, RPIQ =4.81). Furthermore, the results showed that CalReg helps improve the accuracy of soil property prediction. NIRS combined with CalReg has great potential to determine accurately the selected soil properties in the laboratory.

[1]  Jean-Philippe Gras,et al.  Best practices for obtaining and processing field visible and near infrared (VNIR) spectra of topsoils , 2014 .

[2]  K. Norris,et al.  Techniques for obtaining absorption spectra on intact biological samples. , 1961, IRE transactions on bio-medical electronics.

[3]  Mohammadmehdi Saberioon,et al.  A Memory-Based Learning Approach as Compared to Other Data Mining Algorithms for the Prediction of Soil Texture Using Diffuse Reflectance Spectra , 2016, Remote. Sens..

[4]  A. Gholizadeh,et al.  Visible, Near-Infrared, and Mid-Infrared Spectroscopy Applications for Soil Assessment with Emphasis on Soil Organic Matter Content and Quality: State-of-the-Art and Key Issues , 2013, Applied spectroscopy.

[5]  H. Ramon,et al.  Comparison among principal component, partial least squares and back propagation neural network analyses for accuracy of measurement of selected soil properties with visible and near infrared spectroscopy , 2010 .

[6]  Frans van den Berg,et al.  Review of the most common pre-processing techniques for near-infrared spectra , 2009 .

[7]  Xueguang Shao,et al.  Multivariate calibration methods in near infrared spectroscopic analysis , 2010 .

[8]  Thomas Scholten,et al.  The spectrum-based learner: A new local approach for modeling soil vis–NIR spectra of complex datasets , 2013 .

[9]  Eyal Ben-Dor,et al.  A memory-based learning approach utilizing combined spectral sources and geographical proximity for improved VIS-NIR-SWIR soil properties estimation , 2019, Geoderma.

[10]  Budiman Minasny,et al.  Convolutional neural network for simultaneous prediction of several soil properties using visible/near-infrared, mid-infrared, and their combined spectra , 2019, Geoderma.

[11]  L. Buydens,et al.  Comparing support vector machines to PLS for spectral regression applications , 2004 .

[12]  S. Wold,et al.  The multivariate calibration problem in chemistry solved by the PLS method , 1983 .

[13]  A. Saleh,et al.  Application of near-infrared reflectance for quantitative assessment of soil properties , 2017 .

[14]  G. McCarty,et al.  The potential of diffuse reflectance spectroscopy for the determination of carbon inventories in soils. , 2002, Environmental pollution.

[15]  Abdul Mounem Mouazen,et al.  Comparison between artificial neural network and partial least squares for on-line visible and near infrared spectroscopy measurement of soil organic carbon, pH and clay content , 2015 .

[16]  G. Matteucci,et al.  Using laboratory Vis-NIR spectroscopy for monitoring some forest soil properties , 2018, Journal of Soils and Sediments.

[17]  K. Shepherd,et al.  Development of Reflectance Spectral Libraries for Characterization of Soil Properties , 2002 .

[18]  A. McBratney,et al.  Critical review of chemometric indicators commonly used for assessing the quality of the prediction of soil attributes by NIR spectroscopy , 2010 .

[19]  Richard Webster,et al.  Predicting soil properties from the Australian soil visible–near infrared spectroscopic database , 2012 .

[20]  Luiz Antonio Martinelli,et al.  Influence of soil texture on carbon dynamics and storage potential in tropical forest soils of Amazonia , 2003 .

[21]  J. Baerdemaeker,et al.  Vis/NIR spectroscopic measurement of selected soil fertility parameters of Cuban agricultural Cambisols , 2014 .

[22]  M. Vohland,et al.  Comparing different multivariate calibration methods for the determination of soil organic carbon pools with visible to near infrared spectroscopy , 2011 .

[23]  Weixing Cao,et al.  Laboratory assessment of three quantitative methods for estimating the organic matter content of soils in China based on visible/near-infrared reflectance spectra , 2013 .

[24]  Keith D. Shepherd,et al.  Sensing landscape level change in soil fertility following deforestation and conversion in the highlands of Madagascar using Vis-NIR spectroscopy , 2006 .

[25]  Sabine Grunwald,et al.  Modeling of Soil Organic Carbon Fractions Using Visible–Near‐Infrared Spectroscopy , 2009 .

[26]  Geoffrey E. Hinton,et al.  Learning representations by back-propagating errors , 1986, Nature.

[27]  Richard Webster,et al.  Discrimination of Australian soil horizons and classes from their visible–near infrared spectra , 2011 .

[28]  P. Lagacherie,et al.  Regional predictions of eight common soil properties and their spatial structures from hyperspectral Vis–NIR data , 2012 .

[29]  Budiman Minasny,et al.  Transfer learning to localise a continental soil vis-NIR calibration model , 2019, Geoderma.

[30]  H. Ramon,et al.  On-line measurement of some selected soil properties using a VIS–NIR sensor , 2007 .

[31]  Abdul Mounem Mouazen,et al.  On-line visible and near infrared spectroscopy for in-field phosphorous management , 2016 .

[32]  Shinichi Morishita,et al.  On Classification and Regression , 1998, Discovery Science.

[33]  Lakssir Brahim,et al.  A comparative study between a new method and other machine learning algorithms for soil organic carbon and total nitrogen prediction using near infrared spectroscopy , 2019 .

[34]  Guofeng Wu,et al.  Visible and near-infrared reflectance spectroscopy-an alternative for monitoring soil contamination by heavy metals. , 2014, Journal of hazardous materials.

[35]  André Carnieletto Dotto,et al.  Two preprocessing techniques to reduce model covariables in soil property predictions by Vis-NIR spectroscopy , 2017 .

[36]  R. V. Rossel,et al.  Using data mining to model and interpret soil diffuse reflectance spectra. , 2010 .

[37]  Gaëtan F. Tremblay,et al.  Predicting Soil Phosphorus‐Related Properties Using Near‐Infrared Reflectance Spectroscopy , 2012 .

[38]  Abdul Mounem Mouazen,et al.  On-line vis-NIR spectroscopy prediction of soil organic carbon using machine learning , 2019, Soil and Tillage Research.

[39]  Kang He,et al.  Assessment of important soil properties related to Chinese Soil Taxonomy based on vis-NIR reflectance spectroscopy , 2018, Comput. Electron. Agric..

[40]  R. Sui,et al.  SOIL REFLECTANCE SENSING FOR DETERMINING SOIL PROPERTIES IN PRECISION AGRICULTURE , 2001 .

[41]  P. Williams,et al.  Near-Infrared Technology in the Agricultural and Food Industries , 1987 .

[42]  José Alexandre Melo Demattê,et al.  Spectral pedology: A new perspective on evaluation of soils along pedogenetic alterations , 2014 .

[43]  Wouter Saeys,et al.  Potential for Onsite and Online Analysis of Pig Manure using Visible and Near Infrared Reflectance Spectroscopy , 2005 .

[44]  Eyal Ben-Dor,et al.  Modelling Diverse Soil Attributes with Visible to Longwave Infrared Spectroscopy Using PLSR Employed by an Automatic Modelling Engine , 2017, Remote. Sens..

[45]  Keith D. Shepherd,et al.  Soil Spectroscopy: An Alternative to Wet Chemistry for Soil Monitoring , 2015 .

[46]  Rebecca L. Whetton,et al.  Machine learning based prediction of soil total nitrogen, organic carbon and moisture content by using VIS-NIR spectroscopy , 2016 .

[47]  A. Walkley,et al.  AN EXAMINATION OF THE DEGTJAREFF METHOD FOR DETERMINING SOIL ORGANIC MATTER, AND A PROPOSED MODIFICATION OF THE CHROMIC ACID TITRATION METHOD , 1934 .

[48]  Sabine Grunwald,et al.  Comparison of multivariate methods for inferential modeling of soil carbon using visible/near-infrared spectra , 2008 .

[49]  José Alexandre Melo Demattê,et al.  Chemometric soil analysis on the determination of specific bands for the detection of magnesium and potassium by spectroscopy , 2017 .

[50]  Alex B. McBratney,et al.  Simultaneous estimation of several soil properties by ultra-violet, visible, and near-infrared reflectance spectroscopy , 2003 .

[51]  Andrew Rawson,et al.  Rapid Prediction of Soil Water Retention using Mid Infrared Spectroscopy , 2007 .

[52]  M. D. Dyar,et al.  Reflectance and emission spectroscopy study of four groups of phyllosilicates: smectites, kaolinite-serpentines, chlorites and micas , 2008, Clay Minerals.

[53]  R. Poppi,et al.  Determination of organic matter in soils using radial basis function networks and near infrared spectroscopy , 2002 .

[54]  Budiman Minasny,et al.  From pedotransfer functions to soil inference systems , 2002 .

[55]  C. Isendahl,et al.  Determining soil properties in Amazonian Dark Earths by reflectance spectroscopy , 2015 .

[56]  Tashpolat Tiyip,et al.  Study on Soil Salinization Information in Arid Region Using Remote Sensing Technique , 2011 .

[57]  R. V. Rossel,et al.  Visible, near infrared, mid infrared or combined diffuse reflectance spectroscopy for simultaneous assessment of various soil properties , 2006 .

[58]  J. Stafford,et al.  Assessment of two reflectance techniques for the quantification of the within-field spatial variability of soil organic carbon. , 2003 .

[59]  Helaina I. J. Black,et al.  Digital RGB photography and visible-range spectroscopy for soil composition analysis , 2018 .

[60]  J. De Baerdemaeker,et al.  Optimisation of soil VIS–NIR sensor-based variable rate application system of soil phosphorus , 2007 .

[61]  Antonio Delgado,et al.  Soil fertility assessment by Vis-NIR spectroscopy: Predicting soil functioning rather than availability indices , 2019, Geoderma.

[62]  José Alexandre Melo Demattê,et al.  Determining soil water status and other soil characteristics by spectral proximal sensing , 2006 .

[63]  Michael Vohland,et al.  Usefulness of near-infrared spectroscopy for the prediction of chemical and biological soil properties in different long-term experiments , 2013 .

[64]  A. Savitzky,et al.  Smoothing and Differentiation of Data by Simplified Least Squares Procedures. , 1964 .

[65]  James B. Reeves,et al.  Near Infrared Reflectance Spectroscopy for the Analysis of Agricultural Soils , 1999 .

[66]  Desire L. Massart,et al.  Comparison of multivariate methods based on latent vectors and methods based on wavelength selection for the analysis of near-infrared spectroscopic data , 1995 .

[67]  Dominique Arrouays,et al.  Extrapolating regional soil landscapes from an existing soil map: Sampling intensity, validation procedures, and integration of spatial context , 2008 .

[68]  Hang Cheng,et al.  Combination of fractional order derivative and memory-based learning algorithm to improve the estimation accuracy of soil organic matter by visible and near-infrared spectroscopy , 2019, CATENA.

[69]  Mario Minacapilli,et al.  Prediction of Soil Texture Distributions Using VNIR-SWIR Reflectance Spectroscopy , 2013 .

[70]  C. Hurburgh,et al.  Near-Infrared Reflectance Spectroscopy–Principal Components Regression Analyses of Soil Properties , 2001 .

[71]  Jiancang Xie,et al.  Assessing heavy metal concentrations in earth-cumulic-orthic-anthrosols soils using Vis-NIR spectroscopy transform coupled with chemometrics. , 2020, Spectrochimica acta. Part A, Molecular and biomolecular spectroscopy.

[72]  Sabine Grunwald,et al.  A systematic study on the application of scatter-corrective and spectral-derivative preprocessing for multivariate prediction of soil organic carbon by Vis-NIR spectra , 2018 .

[73]  J. M. Soriano-Disla,et al.  The Performance of Visible, Near-, and Mid-Infrared Reflectance Spectroscopy for Prediction of Soil Physical, Chemical, and Biological Properties , 2014 .

[74]  E. Ben-Dor.  Quantitative remote sensing of soil properties , 2002 .

[75]  Robin Gebbers,et al.  Performance of Automated Near‐Infrared Reflectance Spectrometry for Continuous in Situ Mapping of Soil Fertility at Field Scale , 2013 .

[76]  B. Stenberg,et al.  Near‐infrared spectroscopy for within‐field soil characterization: small local calibrations compared with national libraries spiked with local samples , 2010 .

[77]  Michael Vohland,et al.  Determination of soil properties with visible to near- and mid-infrared spectroscopy: Effects of spectral variable selection , 2014 .

[78]  J. Stewart,et al.  Changes in Inorganic and Organic Soil Phosphorus Fractions Induced by Cultivation Practices and by Laboratory Incubations1 , 1982 .

[79]  R. V. Rossel,et al.  In situ measurements of soil colour, mineral composition and clay content by vis–NIR spectroscopy , 2009 .

[80]  J. Steward,et al.  Gel chromatography of soil organic phosphorus , 1971 .

[81]  Henning Buddenbaum,et al.  Estimating the soil clay content and organic matter by means of different calibration methods of vis-NIR diffuse reflectance spectroscopy , 2016 .

[82]  F. H. Merkle,et al.  Controlled-Potential Coulometric Analysis of N-Substituted Phenothiazine Derivatives. , 1964 .

[83]  R. Henry,et al.  Simultaneous Determination of Moisture, Organic Carbon, and Total Nitrogen by Near Infrared Reflectance Spectrophotometry , 1986 .

[84]  Meiyan Wang,et al.  Comparison of multivariate methods for estimating selected soil properties from intact soil cores of paddy fields by Vis–NIR spectroscopy , 2018 .

[85]  José Alexandre Melo Demattê,et al.  Visible–NIR reflectance: a new approach on soil evaluation , 2004 .

[86]  Salah Eddine Itqiq,et al.  Predicting soil phosphorus and studying the effect of texture on the prediction accuracy using machine learning combined with near-infrared spectroscopy. , 2020, Spectrochimica acta. Part A, Molecular and biomolecular spectroscopy.

[87]  Cheng-Wen Chang,et al.  NEAR-INFRARED REFLECTANCE SPECTROSCOPIC ANALYSIS OF SOIL C AND N , 2002 .

[88]  Zhou Shi,et al.  Accounting for the effects of water and the environment on proximally sensed vis–NIR soil spectra and their calibrations , 2015 .