Using rule-based regression models to predict and interpret soil properties from X-ray powder diffraction data

Abstract Data mining is often used to derive calibrations for soil property prediction from diffuse reflectance spectroscopy, facilitating inference of organic and mineral contributions to given properties. In contrast to spectroscopy, X-ray powder diffraction (XRPD) offers a more direct probe into the complexities of soil mineralogy. Here a national scale XRPD dataset of Scottish soils is used in combination with the rule-based regression algorithm ‘Cubist’ for prediction of eight soil properties (total carbon and nitrogen, cation exchange capacity, pH, aqua regia extractable potassium, and the sand, silt and clay size fractions), and interpretation of soil property–mineralogy relationships. Precision sample preparation methods prior to XRPD analysis eliminated effects of preferred orientation, creating reproducible data appropriate for data mining. For direct comparison, Cubist was also applied to an equivalent dataset of near infrared spectroscopy (NIRS) measurements. In terms of predictive performance, XRPD surpassed NIRS for prediction of six of the eight soil properties investigated. Notably, diffuse scattering from X-ray amorphous organic matter facilitated relatively accurate predictions of total carbon and nitrogen from XRPD. Aqua regia extractable potassium was predicted with substantial accuracy and confirmed to reflect the phyllosilicate potassium. The particle size fractions were predicted with moderate-substantial agreement using combinations of quartz, phyllosilicate and feldspar variables. This approach introduces the value of XRPD datasets in enhancing the understanding of soil mineralogy–property relationships whilst contributing to soil mineralogy's advance into the digital soil typing paradigm.

[1]  J. M. Soriano-Disla,et al.  The Performance of Visible, Near-, and Mid-Infrared Reflectance Spectroscopy for Prediction of Soil Physical, Chemical, and Biological Properties , 2014 .

[2]  R. V. Rossel,et al.  In situ measurements of soil colour, mineral composition and clay content by vis–NIR spectroscopy , 2009 .

[3]  R Core Team,et al.  R: A language and environment for statistical computing. , 2014 .

[4]  A. Newman The significance of clays in agriculture and soils , 1984, Philosophical Transactions of the Royal Society of London. Series A, Mathematical and Physical Sciences.

[5]  E. Fitzpatrick Soils: Their Formation, Classification, and Distribution , 1982 .

[6]  J. R. Quinlan Learning With Continuous Classes , 1992 .

[7]  R. V. Rossel,et al.  Using data mining to model and interpret soil diffuse reflectance spectra. , 2010 .

[8]  L. A. Stone,et al.  Computer Aided Design of Experiments , 1969 .

[9]  A. Edwards,et al.  Assessing potassium reserves in northern temperate grassland soils: a perspective based on quantitative mineralogical analysis and aqua-regia extractable potassium. , 2010 .

[10]  K. Lajtha,et al.  Depth trends of soil organic matter C:N and 15N natural abundance controlled by association with minerals , 2017, Biogeochemistry.

[11]  A. Robertson,et al.  Global and Local Calibrations to Predict Chemical and Physical Properties of a National Spatial Dataset of Scottish Soils from Their near Infrared Spectra , 2016 .

[12]  A. McBratney,et al.  Critical review of chemometric indicators commonly used for assessing the quality of the prediction of soil attributes by NIR spectroscopy , 2010 .

[13]  Richard Webster,et al.  Predicting soil properties from the Australian soil visible–near infrared spectroscopic database , 2012 .

[14]  A. Lilly,et al.  Comparison of soil carbon stocks in Scottish soils between 1978 and 2009 , 2013 .

[15]  James B. Reeves,et al.  The potential of mid- and near-infrared diffuse reflectance spectroscopy for determining major- and trace-element concentrations in soils from a geochemical survey of North America. , 2009 .

[16]  R. Parfitt,et al.  Contribution of organic matter and clay minerals to the cation exchange capacity of soils. , 1995 .

[17]  Kurt Hornik,et al.  Misc Functions of the Department of Statistics, ProbabilityTheory Group (Formerly: E1071), TU Wien , 2015 .

[18]  J. Huang,et al.  Field level digital mapping of soil mineralogy using proximal and remote‐sensed data , 2017 .

[19]  S. Hillier,et al.  Acid-extractable potassium in agricultural soils Source minerals assessed by differential and quantitative X-ray diffraction , 2013 .

[20]  S. McGrath,et al.  A simplified method for the extraction of the metals Fe, Zn, Cu, Ni, Cd, Pb, Cr, Co, and Mn from soils and sewage sludges. , 1985 .

[21]  Budiman Minasny,et al.  Synergistic Use of Vis-NIR, MIR, and XRF Spectroscopy for the Determination of Soil Geochemistry , 2016 .

[22]  J. Kirkegaard,et al.  Stable soil organic matter: A comparison of C:N:P:S ratios in Australian and other world soils , 2011 .

[23]  Robert C. Reynolds,et al.  X-Ray Diffraction and the Identification and Analysis of Clay Minerals , 1989 .

[24]  Steve P. McGrath Computerized quality control, statistics and regional mapping of the concentrations of trace and major elements in the soil of England and Wales , 1987 .

[25]  D. Mccarty,et al.  Some successful approaches to quantitative mineral analysis as revealed by the 3rd Reynolds Cup contest , 2006 .

[26]  Budiman Minasny,et al.  Digital soil mapping: A brief history and some lessons , 2016 .

[27]  B. Minasny,et al.  Regression rules as a tool for predicting soil properties from infrared reflectance spectroscopy , 2008 .

[28]  L. Lin,et al.  A concordance correlation coefficient to evaluate reproducibility. , 1989, Biometrics.

[29]  C. Cleveland,et al.  C:N:P stoichiometry in soil: is there a “Redfield ratio” for the microbial biomass? , 2007 .

[30]  Budiman Minasny,et al.  An assessment of model averaging to improve predictive power of portable vis-NIR and XRF for the determination of agronomic soil properties , 2016 .

[31]  S. Hillier,et al.  Use of an air brush to spray dry samples for X-ray powder diffraction , 1999, Clay Minerals.

[32]  Wei Dong,et al.  PolySNAP3: a computer program for analysing and visualizing high-throughput data from diffraction and spectroscopic sources , 2009 .

[33]  Alex B. McBratney,et al.  In Situ Analysis of Soil Mineral Composition Through Conjoint Use of Visible, Near-Infrared and X-Ray Fluorescence Spectroscopy , 2016 .

[34]  Chi Ma,et al.  Cation Exchange Capacity of Kaolinite , 1999 .

[35]  Keith D. Shepherd,et al.  Soil Spectroscopy: An Alternative to Wet Chemistry for Soil Monitoring , 2015 .

[36]  Viacheslav I. Adamchuk,et al.  A global spectral library to characterize the world’s soil , 2016 .

[37]  David L. Bish,et al.  FULLPAT: a full-pattern quantitative analysis program for X-ray powder diffraction using measured and calculated patterns , 2002 .

[38]  T. Doe,et al.  Nature of feldspar-grain size relations in some quartz-rich sandstones , 1976 .

[39]  B. Minasny,et al.  Digital Soil Map of the World , 2009, Science.

[40]  S. Hillier,et al.  Mineralogical budgeting of potassium in soil: A basis for understanding standard measures of reserve potassium , 2006 .

[41]  K. Shepherd,et al.  Total elemental composition of soils in Sub-Saharan Africa and relationship with soil forming factors , 2015 .

[42]  R. Barnes,et al.  Standard Normal Variate Transformation and De-Trending of Near-Infrared Diffuse Reflectance Spectra , 1989 .