Local modeling approaches for estimating soil properties in selected Indian soils using diffuse reflectance data over visible to near-infrared region

Abstract Robust calibration algorithms are needed for the accurate assessment of soil properties in the diffuse reflectance spectroscopy (DRS) approach. Despite several studies on different calibration algorithms, the prediction accuracy of soil properties using DRS need to be improved. Specifically, the utility of local modeling approaches for small spectral libraries is less examined compared with global modeling approaches. In this study, we compared global modeling approaches such as partial-least-squares regression (PLSR), lasso, ridge regression with several locally-weighted PLSR (PLSR LW ) approaches. We also examined seven different distance measures: Euclidean distance, covariance-based distance, correlation-based distance, surface difference spectrum, information-based distance, optimized principal component Mahalanobis, and locally-linear embeddings used in the PLSR LW approach for their effectiveness in modeling soil properties using DRS. A total of 954 soil samples were collected from three different states of India: West Bengal, Odisha, and Karnataka. Five soil properties such as sand content, clay content, soil organic carbon (SOC) content, extractable iron (Fe) content and extractable zinc (Zn) content were predicted using reflectance spectra over 350–2500 nm. Root-mean-squared error (RMSE) and the coefficient of determination (R 2 ) were used as performance statistics. Among the global modeling approaches, lasso performed better than the PLSR although it is computationally more intensive than the PLSR. In general, the correlation-based PLSR LW performed significantly better than the global approaches. Specifically, the test R 2 values increased from 0.66 to 0.72 for prediction of sand content, from 0.59 to 0.70 for prediction of SOC content, and from 0.70 to 0.74 for prediction of Fe content by using the PLSR LW over PLSR. We also used depth of absorption peak of spectra at approximately 1400, 1900 and 2200 nm for mineralogical characterization of soil samples. The results suggested that the improvement in prediction accuracy of soil properties using the PLSR LW was achieved because calibration samples which had same mineralogy as the test sample were given higher weights. These results suggest that the prediction accuracy of soil properties may also be improved in small spectral libraries if an appropriate local modeling scheme is selected.

[1]  Viacheslav I. Adamchuk,et al.  A global spectral library to characterize the world’s soil , 2016 .

[2]  Martial Bernoux,et al.  National calibration of soil organic carbon concentration using diffuse infrared reflectance spectroscopy , 2016 .

[3]  W. Lindsay,et al.  Development of a DTPA soil test for zinc, iron, manganese and copper , 1978 .

[4]  Panos Panagos,et al.  Prediction of soil organic carbon content by diffuse reflectance spectroscopy using a local partial least square regression approach , 2014 .

[5]  K. M. Nair,et al.  Soils of India: historical perspective, classification and recent advances , 2013 .

[6]  Bernhard Schölkopf,et al.  A tutorial on support vector regression , 2004, Stat. Comput..

[7]  K. Krishna Agroecosystems: Soils, Climate, Crops, Nutrient Dynamics and Productivity , 2013 .

[8]  Trevor Hastie,et al.  The elements of statistical learning. 2001 , 2001 .

[9]  A. Walkley,et al.  AN EXAMINATION OF THE DEGTJAREFF METHOD FOR DETERMINING SOIL ORGANIC MATTER, AND A PROPOSED MODIFICATION OF THE CHROMIC ACID TITRATION METHOD , 1934 .

[10]  J. M. Soriano-Disla,et al.  The Performance of Visible, Near-, and Mid-Infrared Reflectance Spectroscopy for Prediction of Soil Physical, Chemical, and Biological Properties , 2014 .

[11]  G. Palumbo,et al.  Chemometric technique performances in predicting forest soil chemical and biological properties from UV-Vis-NIR reflectance spectra with small, high dimensional datasets , 2016 .

[12]  R. Tibshirani,et al.  Improvements on Cross-Validation: The 632+ Bootstrap Method , 1997 .

[13]  Bernard Barthès,et al.  Prediction of soil organic carbon stock using visible and near infrared reflectance spectroscopy (VNIRS) in the field , 2016 .

[14]  Sabine Grunwald,et al.  Soil total carbon analysis in Hawaiian soils with visible, near-infrared and mid-infrared diffuse reflectance spectroscopy , 2012 .

[15]  R. M. Lark,et al.  The relationship between diffuse spectral reflectance of the soil and its cation exchange capacity is scale-dependent , 2010 .

[16]  A. Savitzky,et al.  Smoothing and Differentiation of Data by Simplified Least Squares Procedures. , 1964 .

[17]  B. Kowalski,et al.  Partial least-squares regression: a tutorial , 1986 .

[18]  J. Friedman Multivariate adaptive regression splines , 1990 .

[19]  A. Shiferaw,et al.  Visible near infra-red (VisNIR) spectroscopy for predicting soil organic carbon in Ethiopia , 2014 .

[20]  Mohammadmehdi Saberioon,et al.  A Memory-Based Learning Approach as Compared to Other Data Mining Algorithms for the Prediction of Soil Texture Using Diffuse Reflectance Spectra , 2016, Remote. Sens..

[21]  J. Bezdek,et al.  FCM: The fuzzy c-means clustering algorithm , 1984 .

[22]  Thorsten Behrens,et al.  Distance and similarity-search metrics for use with soil vis-NIR spectra , 2013 .

[23]  Sabine Grunwald,et al.  Comparison of multivariate methods for inferential modeling of soil carbon using visible/near-infrared spectra , 2008 .

[24]  P. Lagacherie,et al.  Regional predictions of eight common soil properties and their spatial structures from hyperspectral Vis–NIR data , 2012 .

[25]  Manabu Kano,et al.  Covariance-based Locally Weighted Partial Least Squares for High- Performance Adaptive Modeling , 2015 .

[26]  Pierre Dardenne,et al.  Near Infrared Reflectance Spectroscopy for Estimating Soil Characteristics Valuable in the Diagnosis of Soil Fertility , 2011 .

[27]  Gilles Grandjean,et al.  Geometrical analysis of laboratory soil spectra in the short-wave infrared domain: Clay composition and estimation of the swelling potential , 2015 .

[28]  Thomas Scholten,et al.  The spectrum-based learner: A new local approach for modeling soil vis–NIR spectra of complex datasets , 2013 .

[29]  D. Cox,et al.  An Analysis of Transformations , 1964 .

[30]  Xia Zhang,et al.  Estimating soil zinc concentrations using reflectance spectroscopy , 2017, Int. J. Appl. Earth Obs. Geoinformation.

[31]  S.A. Dyer,et al.  Estimation of Soil Properties Using a Combination of Spectral and Scalar Sensor Data , 2006, 2006 IEEE Instrumentation and Measurement Technology Conference Proceedings.

[32]  Claudy Jolivet,et al.  Optimization criteria in sample selection step of local regression for quantitative analysis of large soil NIRS database , 2012 .

[33]  Manfred F. Buchroithner,et al.  Quantitative Retrieval of Organic Soil Properties from Visible Near-Infrared Shortwave Infrared (Vis-NIR-SWIR) Spectroscopy Using Fractal-Based Feature Extraction , 2016, Remote. Sens..

[34]  I C Edmundson,et al.  Particle size analysis , 2013 .

[35]  José Alexandre Melo Demattê,et al.  Spectral pedology: A new perspective on evaluation of soils along pedogenetic alterations , 2014 .

[36]  B. Mohanty,et al.  Rapid and Noninvasive Assessment of Atterberg Limits Using Diffuse Reflectance Spectroscopy , 2016 .

[37]  Manabu Kano,et al.  Estimation of active pharmaceutical ingredients content using locally weighted partial least squares and statistical wavelength selection. , 2011, International journal of pharmaceutics.

[38]  Rajeev Srivastava,et al.  Hyperspectral remote sensing: opportunities, status and challenges for rapid soil assessment in India , 2015 .

[39]  R. V. Rossel,et al.  Using data mining to model and interpret soil diffuse reflectance spectra. , 2010 .

[40]  Mario Minacapilli,et al.  Prediction of Soil Texture Distributions Using VNIR-SWIR Reflectance Spectroscopy , 2013 .

[41]  Jing Wang,et al.  Rapid identification and assay of crude oils based on moving-window correlation coefficient and near infrared spectral library , 2011 .

[42]  Sabine Grunwald,et al.  Spectroscopic models of soil organic carbon in Florida, USA. , 2010, Journal of environmental quality.

[43]  Suhas P. Wani,et al.  Variable indicators for optimum wavelength selection in diffuse reflectance spectroscopy of soils , 2016 .

[44]  B. Minasny,et al.  Removing the effect of soil moisture from NIR diffuse reflectance spectra for the prediction of soil organic carbon , 2011 .

[45]  Suhas P. Wani,et al.  Dependency measures for assessing the covariation of spectrally active and inactive soil properties in diffuse reflectance spectroscopy , 2014 .

[46]  R. V. Rossel,et al.  Visible, near infrared, mid infrared or combined diffuse reflectance spectroscopy for simultaneous assessment of various soil properties , 2006 .