Evaluating Calibration and Spectral Variable Selection Methods for Predicting Three Soil Nutrients Using Vis-NIR Spectroscopy

Soil nutrients, including soil available potassium (SAK), soil available phosphorous (SAP), and soil organic matter (SOM), play an important role in farmland soil productivity, food security, and agricultural management. Spectroscopic analysis has proven to be a rapid, nondestructive, and effective technique for predicting soil properties in general and potassium, phosphorous, and organic matter in particular. However, the successful estimation of soil nutrient content by visible and near-infrared (Vis-NIR) reflectance spectroscopy depends on proper calibration methods (including preprocessing transformation methods and multivariate methods for regression analysis) and the selection of appropriate variable selection techniques. In this study, raw spectrum and 13 preprocessing transformations combined with 2 variable selection methods (competitive adaptive reweighted sampling (CARS) and the successive projections algorithm (SPA)) and 2 regression algorithms (support vector machine (SVM) and partial least squares regression (PLSR)), for a total of 56 calibration methods, were investigated for modeling and predicting the above three soil nutrients using hyperspectral Vis-NIR data (400–2450 nm). The results show that first-order derivatives based on logarithmic and inverse transformations (FD-LGRs) can provide better predictions of soil available potassium and phosphorous, and the best form of soil organic matter transformation is SG+MSC. CARS was superior to the SPA in selecting effective variables, and the PLSR model outperformed the SVM models. The best estimation accuracies (R2, RMSE) for soil available potassium, phosphorous, and organic matter were 0.7532, 32.3090 mg/kg; 0.7440, 6.6910 mg/kg; and 0.9009, 3.2103 g/kg, respectively, and their corresponding calibration methods were (FD-LGR)/SPA/PLSR, (FD-LGR)/SPA/PLSR, and SG+MSC/CARS/SVM, respectively. Overall, for the prediction of the soil nutrient content, organic matter was superior to available phosphorous, followed by available potassium. It was concluded that the application of hyperspectral images (Vis-NIR data) was an efficient method for mapping and monitoring soil nutrients at the regional scale, thus contributing to the development of precision agriculture.

[1]  Henry Lin,et al.  Advancing the frontiers of soil science towards a geoscience , 2006 .

[2]  J. Demattê,et al.  Stratification of a local VIS-NIR-SWIR spectral library by homogeneity criteria yields more accurate soil organic carbon predictions , 2019, Geoderma.

[3]  Guangxing Wang,et al.  Integrating a Hybrid Back Propagation Neural Network and Particle Swarm Optimization for Estimating Soil Heavy Metal Contents Using Hyperspectral Data , 2019, Sustainability.

[4]  Zou Xiaobo,et al.  Variables selection methods in near-infrared spectroscopy. , 2010, Analytica chimica acta.

[5]  Hidetoshi Asai,et al.  Laboratory Visible and Near-Infrared Spectroscopy with Genetic Algorithm-Based Partial Least Squares Regression for Assessing the Soil Phosphorus Content of Upland and Lowland Rice Fields in Madagascar , 2019, Remote. Sens..

[6]  Xixi Liu,et al.  Water-based measured-value fuzzification improves the estimation accuracy of soil organic matter by visible and near-infrared spectroscopy. , 2020, The Science of the total environment.

[7]  A. Mouazen,et al.  Comparing laboratory and airborne hyperspectral data for the estimation and mapping of topsoil organic carbon: Feature selection coupled with random forest , 2020 .

[8]  Jingming Ning,et al.  Rapid evaluation of soil fertility in tea plantation based on near-infrared spectroscopy , 2018, Spectroscopy Letters.

[9]  Weixing Cao,et al.  Laboratory assessment of three quantitative methods for estimating the organic matter content of soils in China based on visible/near-infrared reflectance spectra , 2013 .

[10]  Guoqiang Wang,et al.  The Application of Discrete Wavelet Transform with Improved Partial Least-Squares Method for the Estimation of Soil Properties with Visible and Near-Infrared Spectral Data , 2018, Remote. Sens..

[11]  Guofeng Wu,et al.  Comparison of multivariate methods for estimating soil total nitrogen with visible/near-infrared spectroscopy , 2012, Plant and Soil.

[12]  G. Matteucci,et al.  Using laboratory Vis-NIR spectroscopy for monitoring some forest soil properties , 2018, Journal of Soils and Sediments.

[13]  A. McBratney,et al.  Pedometrics timeline , 2019, Geoderma.

[14]  Shi Zhou,et al.  In Situ Measurement of Some Soil Properties in Paddy Soil Using Visible and Near-Infrared Spectroscopy , 2014, PloS one.

[15]  M. Cho,et al.  Non-linear partial least square regression increases the estimation accuracy of grass nitrogen and phosphorus using in situ hyperspectral and environmental data , 2013 .

[16]  Tormod Næs,et al.  Related versions of the multiplicative scatter correction method for preprocessing spectroscopic data , 1995 .

[17]  M. C. U. Araújo,et al.  The successive projections algorithm for variable selection in spectroscopic multicomponent analysis , 2001 .

[18]  Jean-Philippe Gras,et al.  Best practices for obtaining and processing field visible and near infrared (VNIR) spectra of topsoils , 2014 .

[19]  Roberta E. Martin,et al.  Multi-method ensemble selection of spectral bands related to leaf biochemistry , 2015 .

[20]  Hang Cheng,et al.  Transferability of Vis-NIR models for Soil Organic Carbon Estimation between Two Study Areas by using Spiking , 2018, Soil Science Society of America Journal.

[21]  Jianli Ding,et al.  Combination of efficient signal pre-processing and optimal band combination algorithm to predict soil organic matter through visible and near-infrared spectra. , 2020, Spectrochimica acta. Part A, Molecular and biomolecular spectroscopy.

[22]  Hang Cheng,et al.  Combining multivariate method and spectral variable selection for soil total nitrogen estimation by Vis–NIR spectroscopy , 2020, Archives of Agronomy and Soil Science.

[23]  Yiyun Chen,et al.  Estimating Soil Organic Carbon Using VIS/NIR Spectroscopy with SVMR and SPA Methods , 2014, Remote. Sens..

[24]  Sabine Grunwald,et al.  Modeling of Soil Organic Carbon Fractions Using Visible–Near‐Infrared Spectroscopy , 2009 .

[25]  Peter Finke,et al.  Digital mapping of soil properties using multiple machine learning in a semi-arid region, central Iran , 2019, Geoderma.

[26]  Viacheslav I. Adamchuk,et al.  Simultaneous measurement of multiple soil properties through proximal sensor data fusion: A case study , 2019, Geoderma.

[27]  M. Vohland,et al.  Comparing different multivariate calibration methods for the determination of soil organic carbon pools with visible to near infrared spectroscopy , 2011 .

[28]  Sara E. Kuebbing,et al.  Evidence for the primacy of living root inputs, not root or shoot litter, in forming soil organic carbon. , 2018, The New phytologist.

[29]  A. Walkley,et al.  AN EXAMINATION OF THE DEGTJAREFF METHOD FOR DETERMINING SOIL ORGANIC MATTER, AND A PROPOSED MODIFICATION OF THE CHROMIC ACID TITRATION METHOD , 1934 .

[30]  Abdul Mounem Mouazen,et al.  Special issue ‘Diffuse reflectance spectroscopy in soil science and land resource assessment’ , 2010 .

[31]  Suhas P. Wani,et al.  Effects of image pansharpening on soil total nitrogen prediction models in South India , 2018, Geoderma.

[32]  Yong He,et al.  Nitrogen, phosphorus, and potassium prediction in soils, using infrared spectroscopy , 2011 .

[33]  A A Gowen,et al.  Characterisation of hydrogen bond perturbations in aqueous systems using aquaphotomics and multivariate curve resolution-alternating least squares. , 2013, Analytica chimica acta.

[34]  R. Webster,et al.  Discriminating between organic matter in soil from grass and forest by near‐infrared spectroscopy , 2010 .

[35]  S. Wold,et al.  The Collinearity Problem in Linear Regression. The Partial Least Squares (PLS) Approach to Generalized Inverses , 1984 .

[36]  G. Hunt SPECTRAL SIGNATURES OF PARTICULATE MINERALS IN THE VISIBLE AND NEAR INFRARED , 1977 .

[37]  Yong Zhang,et al.  Combining Fractional Order Derivative and Spectral Variable Selection for Organic Matter Estimation of Homogeneous Soil Samples by VIS-NIR Spectroscopy , 2018, Remote. Sens..

[38]  Sabine Grunwald,et al.  Comparison of multivariate methods for inferential modeling of soil carbon using visible/near-infrared spectra , 2008 .

[39]  Darrell Whitley,et al.  A genetic algorithm tutorial , 1994, Statistics and Computing.

[40]  Gerard Govers,et al.  Quantification of organic carbon concentrations and stocks of tidal marsh sediments via mid-infrared spectroscopy , 2019, Geoderma.

[41]  B. Kowalski,et al.  Partial least-squares regression: a tutorial , 1986 .

[42]  E. Ben-Dor,et al.  NEAR INFRARED ANALYSIS (NIRA) AS A METHOD TO SIMULTANEOUSLY EVALUATE SPECTRAL FEATURELESS CONSTITUENTS IN SOILS , 1995 .

[43]  Zhan-feng Yang,et al.  The Hongqiaocun Site: The earliest evidence of ancient flood sedimentation of the water conservancy facilities in the Chengdu Plain, China , 2020 .

[44]  Meiyan Wang,et al.  Comparison of multivariate methods for estimating selected soil properties from intact soil cores of paddy fields by Vis–NIR spectroscopy , 2018 .

[45]  H. Ramon,et al.  Comparison among principal component, partial least squares and back propagation neural network analyses for accuracy of measurement of selected soil properties with visible and near infrared spectroscopy , 2010 .

[46]  Sabine Chabrillat,et al.  A remote sensing adapted approach for soil organic carbon prediction based on the spectrally clustered LUCAS soil database , 2019, Geoderma.

[47]  Frans van den Berg,et al.  Review of the most common pre-processing techniques for near-infrared spectra , 2009 .

[48]  Xianzhang Pan,et al.  Evaluating the characteristics of soil vis-NIR spectra after the removal of moisture effect using external parameter orthogonalization , 2020 .

[49]  A. Savitzky,et al.  Smoothing and Differentiation of Data by Simplified Least Squares Procedures. , 1964 .

[50]  Bin Zhao,et al.  Using hyperspectral vegetation indices as a proxy to monitor soil salinity , 2010 .

[51]  Jing Li,et al.  SPA-Based Methods for the Quantitative Estimation of the Soil Salt Content in Saline-Alkali Land from Field Spectroscopy Data: A Case Study from the Yellow River Irrigation Regions , 2019, Remote. Sens..

[52]  R. V. Rossel,et al.  Visible–Near Infrared Spectra as a Proxy for Topsoil Texture and Glacial Boundaries , 2013 .

[53]  Christophe David,et al.  Nutrient recycling in organic farming is related to diversity in farm types at the local level , 2015 .

[54]  Hongwei Wang,et al.  Assessing Soil Organic Matter Content in a Coal Mining Area through Spectral Variables of Different Numbers of Dimensions , 2020, Sensors.

[55]  Shiwen Zhang,et al.  Hyperspectral indirect inversion of heavy-metal copper in reclaimed soil of iron ore area. , 2019, Spectrochimica acta. Part A, Molecular and biomolecular spectroscopy.

[56]  Jia Sun,et al.  Prediction of Soil-Available Potassium Content with Visible Near-Infrared Ray Spectroscopy of Different Pretreatment Transformations by the Boosting Algorithms , 2020, Applied Sciences.

[57]  R. Pullanagari,et al.  Quantification of dead vegetation fraction in mixed pastures using AisaFENIX imaging spectroscopy data , 2017, Int. J. Appl. Earth Obs. Geoinformation.

[58]  R. V. Rossel,et al.  Using data mining to model and interpret soil diffuse reflectance spectra. , 2010 .

[59]  Kang He,et al.  Assessment of important soil properties related to Chinese Soil Taxonomy based on vis-NIR reflectance spectroscopy , 2018, Comput. Electron. Agric..

[60]  A. Karnieli,et al.  A spectral soil quality index (SSQI) for characterizing soil function in areas of changed land use , 2014 .

[61]  F. Meer,et al.  Quantitative analysis of salt-affected soil reflectance spectra: A comparison of two adaptive methods (PLSR and ANN) , 2007 .

[62]  Qing-Song Xu,et al.  libPLS: An integrated library for partial least squares regression and linear discriminant analysis , 2018 .

[63]  A. Mouazen,et al.  Assessment of a soil fertility index using visible and near‐infrared spectroscopy in the rice paddy region of southern China , 2019, European Journal of Soil Science.

[64]  Jichang Han,et al.  Application of Vis-NIR spectroscopy for determination the content of organic matter in saline-alkali soils. , 2019, Spectrochimica acta. Part A, Molecular and biomolecular spectroscopy.

[65]  Shengyao Jia,et al.  Quantitative Analysis of Soil Total Nitrogen Using Hyperspectral Imaging Technology with Extreme Learning Machine , 2019, Sensors.

[66]  Tarin Paz-Kagan,et al.  Evaluating calibration methods for predicting soil available nutrients using hyperspectral VNIR data , 2018 .

[67]  L. Wilding,et al.  Spatial variability: its documentation, accommodation and implication to soil surveys , 1985 .

[68]  R. Barnes,et al.  Standard Normal Variate Transformation and De-Trending of Near-Infrared Diffuse Reflectance Spectra , 1989 .

[69]  R. V. Rossel,et al.  Spectral fusion by Outer Product Analysis (OPA) to improve predictions of soil organic C , 2019, Geoderma.

[70]  Hongdong Li,et al.  Key wavelengths screening using competitive adaptive reweighted sampling method for multivariate calibration. , 2009, Analytica chimica acta.

[71]  Bo Li,et al.  Predicting Spatial Variations in Soil Nutrients with Hyperspectral Remote Sensing at Regional Scale , 2018, Sensors.

[72]  C. Hurburgh,et al.  Near-Infrared Reflectance Spectroscopy–Principal Components Regression Analyses of Soil Properties , 2001 .

[73]  Sabine Grunwald,et al.  A systematic study on the application of scatter-corrective and spectral-derivative preprocessing for multivariate prediction of soil organic carbon by Vis-NIR spectra , 2018 .

[74]  Raphael A. Viscarra Rossel,et al.  ParLeS: Software for chemometric analysis of spectroscopic data , 2008 .

[75]  Hassan Ghassemian,et al.  A probabilistic SVM approach for hyperspectral image classification using spectral and texture features , 2017 .

[76]  L. S. Galvão,et al.  Role of organic matter in obliterating the effects of iron on spectral reflectance and colour of Brazilian tropical soils , 1998 .

[77]  Erik Karltun,et al.  Soil nutrient build-up, input interaction effects and plot level N and P balances under long-term addition of compost and NP fertilizer , 2016 .

[78]  Henning Buddenbaum,et al.  Estimating the soil clay content and organic matter by means of different calibration methods of vis-NIR diffuse reflectance spectroscopy , 2016 .

[79]  C. Du,et al.  Evaluation of soil fertility using infrared spectroscopy: a review , 2009 .

[80]  Qing Li,et al.  Hyperspectral Imaging Analysis for the Classification of Soil Types and the Determination of Soil Total Nitrogen , 2017, Sensors.

[81]  Jiansheng Wu,et al.  Soil moisture retrieving using hyperspectral data with the application of wavelet analysis , 2013, Environmental Earth Sciences.

[82]  Bernhard Schölkopf,et al.  A tutorial on support vector regression , 2004, Stat. Comput..

[83]  Wouter Saeys,et al.  Potential for Onsite and Online Analysis of Pig Manure using Visible and Near Infrared Reflectance Spectroscopy , 2005 .

[84]  Rebecca L. Whetton,et al.  Machine learning based prediction of soil total nitrogen, organic carbon and moisture content by using VIS-NIR spectroscopy , 2016 .