Hyperspectral Estimation of Soil Organic Matter Content using Different Spectral Preprocessing Techniques and PLSR Method

Soil organic matter (SOM) is the main source of soil nutrients, which are essential for the growth and development of agricultural crops. Hyperspectral remote sensing is one of the most efficient ways of estimating the SOM content. Visible, near infrared, and mid-infrared reflectance spectroscopy, combined with the partial least squares regression (PLSR) method is considered to be an effective way of determining soil properties. In this study, we used 54 different spectral pretreatments to preprocess soil spectral data. These spectral pretreatments were composed of three denoising methods, six data transformations, and three dimensionality reduction methods. The three denoising methods included no denoising (ND), Savitzky–Golay denoising (SGD), and wavelet packet denoising (WPD). The six data transformations included original spectral data, R; reciprocal, 1/R; logarithmic, log(R); reciprocal logarithmic, log(1/R); first derivative, R’; and first derivative of reciprocal, (1/R)’. The three dimensionality reduction methods included no dimensionality reduction (NDR), sensitive waveband dimensionality reduction (SWDR), and principal component analysis (PCA) dimensionality reduction (PCADR). The processed spectra were then employed to construct PLSR models for predicting the SOM content. The main results were as follows—(1) the wavelet packet denoising (WPD)-R’ and WPD-(1/R)’ data showed stronger correlations with the SOM content. Furthermore, these methods could effectively limit the correlation between the adjacent bands and, thus, prevent “overfitting”. (2) Of the 54 pretreatments investigated, WPD-(1/R)’-PCADR yielded the model with the highest accuracy and stability. (3) For the same denoising method and spectral transformation data, the accuracy of the SOM content estimation model based on SWDR was higher than that of the model based on NDR. Furthermore, the accuracy in the case of PCADR was higher than that for SWDR. (4) Dimensionality reduction was effective in preventing data overfitting. (5) The quality of the spectral data could be improved and the accuracy of the SOM content estimation model could be enhanced effectively, by using some appropriate preprocessing methods (one combining WPD and PCADR in this study).

[1]  Y. S. Shivay,et al.  Soil Organic Matter , 2021, Soil Organic Matter and Feeding the Future.

[2]  Dimitrios D. Alexakis,et al.  Integrated Use of Satellite Remote Sensing, Artificial Neural Networks, Field Spectroscopy, and GIS in Estimating Crucial Soil Parameters in Terms of Soil Erosion , 2019, Remote. Sens..

[3]  Shiwen Zhang,et al.  Hyperspectral inversion of heavy metal content in reclaimed soil from a mining wasteland based on different spectral transformation and modeling methods. , 2019, Spectrochimica acta. Part A, Molecular and biomolecular spectroscopy.

[4]  Hidetoshi Asai,et al.  Laboratory Visible and Near-Infrared Spectroscopy with Genetic Algorithm-Based Partial Least Squares Regression for Assessing the Soil Phosphorus Content of Upland and Lowland Rice Fields in Madagascar , 2019, Remote. Sens..

[5]  Junjie Wang,et al.  The Influence of Spectral Pretreatment on the Selection of Representative Calibration Samples for Soil Organic Matter Estimation Using Vis-NIR Reflectance Spectroscopy , 2019, Remote. Sens..

[6]  Fei Zhang,et al.  New methods for improving the remote sensing estimation of soil organic matter content (SOMC) in the Ebinur Lake Wetland National Nature Reserve (ELWNNR) in northwest China , 2018, Remote Sensing of Environment.

[7]  Eyal Ben-Dor,et al.  Examining the Performance of PARACUDA-II Data-Mining Engine versus Selected Techniques to Model Soil Carbon from Reflectance Spectra , 2018, Remote. Sens..

[8]  F. Mainardi Fractional Calculus , 2018, Fractional Calculus.

[9]  Jay Gao,et al.  Hyperspectral sensing of heavy metals in soil and vegetation: Feasibility and challenges , 2018 .

[10]  Susana Fernández,et al.  Prediction of Topsoil Organic Carbon Using Airborne and Satellite Hyperspectral Imagery , 2017, Remote. Sens..

[11]  Michael Vohland,et al.  Quantification of Soil Properties with Hyperspectral Data: Selecting Spectral Variables with Different Methods to Improve Accuracies and Analyze Prediction Mechanisms , 2017, Remote. Sens..

[12]  Gustau Camps-Valls,et al.  Hyperspectral dimensionality reduction for biophysical variable statistical retrieval , 2017 .

[13]  Meichen Feng,et al.  Hyperspectral estimation of soil organic matter based on different spectral preprocessing techniques , 2017 .

[14]  Eyal Ben-Dor,et al.  Modelling Diverse Soil Attributes with Visible to Longwave Infrared Spectroscopy Using PLSR Employed by an Automatic Modelling Engine , 2017, Remote. Sens..

[15]  Dongryeol Ryu,et al.  Estimation of Organic Matter Content in Coastal Soil Using Reflectance Spectroscopy , 2016 .

[16]  Bo Li,et al.  Adaptive fractional differential approach and its application to medical image enhancement , 2015, Comput. Electr. Eng..

[17]  Wouter Saeys,et al.  Multivariate calibration of NIR spectroscopic sensors for continuous glucose monitoring , 2015 .

[18]  Nicholas M. Holden,et al.  Evaluation of soil structural quality using VIS-NIR spectra , 2015 .

[19]  Edward G. Gregorich,et al.  Rapid determination of soil organic matter quality indicators using visible near infrared reflectance spectroscopy , 2014 .

[20]  J. Hook,et al.  Smoothing non-smooth systems with low-pass filters , 2014 .

[21]  Della Riccia Giacomo,et al.  A multivariate regression model for detection of fumonisins content in maize from near infrared spectra. , 2013, Food chemistry.

[22]  Jitendra Virmani,et al.  SVM-Based Characterization of Liver Ultrasound Images Using Wavelet Packet Texture Descriptors , 2013, Journal of Digital Imaging.

[23]  Huazhou Chen,et al.  The Combined Optimization of Savitzky-Golay Smoothing and Multiplicative Scatter Correction for FT-NIR PLS Models , 2013 .

[24]  Jing Wang,et al.  Mapping soil organic matter based on land degradation spectral response units using Hyperion images , 2010, Int. J. Appl. Earth Obs. Geoinformation.

[25]  J. Paruelo,et al.  How to evaluate models : Observed vs. predicted or predicted vs. observed? , 2008 .

[26]  H. Ramon,et al.  On-line measurement of some selected soil properties using a VIS–NIR sensor , 2007 .

[27]  R. V. Rossel,et al.  Visible, near infrared, mid infrared or combined diffuse reflectance spectroscopy for simultaneous assessment of various soil properties , 2006 .

[28]  Anne E. Carey,et al.  Organic matter sources and transport in an agriculturally dominated temperate watershed , 2004 .

[29]  Sabine Chabrillat,et al.  Use of hyperspectral images in the identification and mapping of expansive clay soils and the role of spatial resolution , 2002 .

[30]  C. Hurburgh,et al.  Near-Infrared Reflectance Spectroscopy–Principal Components Regression Analyses of Soil Properties , 2001 .

[31]  E. Ben-Dor,et al.  NEAR INFRARED ANALYSIS (NIRA) AS A METHOD TO SIMULTANEOUSLY EVALUATE SPECTRAL FEATURELESS CONSTITUENTS IN SOILS , 1995 .

[32]  Eyal Ben-Dor,et al.  Near-Infrared Analysis as a Rapid Method to Simultaneously Evaluate Several Soil Properties , 1995 .

[33]  R. Barnes,et al.  Standard Normal Variate Transformation and De-Trending of Near-Infrared Diffuse Reflectance Spectra , 1989 .

[34]  K. Sahrawat,et al.  Simple modification of the Walkley-Black method for simultaneous determination of organic carbon and potentially mineralizable nitrogen in tropical rice soils , 1982, Plant and Soil.

[35]  A. Savitzky,et al.  Smoothing and Differentiation of Data by Simplified Least Squares Procedures. , 1964 .

[36]  Karl Pearson F.R.S. LIII. On lines and planes of closest fit to systems of points in space , 1901 .

[37]  K. Moffett,et al.  Remote Sens , 2015 .

[38]  Panos Panagos,et al.  Prediction of soil organic carbon content by diffuse reflectance spectroscopy using a local partial least square regression approach , 2014 .

[39]  Wang Jianhua,et al.  Reconstruction of MODIS-EVI Time-Series Data with S-G Filter , 2009 .

[40]  Z. Weijun Spectral Characteristics of Soils in Hunan Province as Affected by Removal of Soil Organic Matter , 2006 .

[41]  E. R. Stoner,et al.  REFLECTANCE PROPERTIES OF SOILS , 1986 .

[42]  J. Steinier,et al.  Smoothing and differentiation of data by simplified least square procedure. , 1972, Analytical chemistry.