A numerical procedure for curve fitting of noisy infrared spectra

A method for the detection of overfitting of noisy spectra is presented. When fitting data that contains random noise, the autocorrelation function (RL) of residuals at lag 1 (R1) approaches zero and then shows a tendency toward more negative values, while the Wald–Wolfowitz test tends to give more positive values, as the data is overfitted. Models that give residuals with a non-random ordering may be rejected. The use of these functions for the determination of the “best” fitting of several approximations (Savitzky–Golay smoothing, segmented osculating polynomials (SOPA), Fourier series, and Lorentzian bands) to a noisy synthetic spectrum, and the application to infrared spectra of coal, is shown.

[1]  P. Gans,et al.  On the Analysis of Raman Spectra by Curve Resolution , 1977 .

[2]  A. Savitzky,et al.  Smoothing and Differentiation of Data by Simplified Least Squares Procedures. , 1964 .

[3]  R. W. Snyder,et al.  Characterization of vitrinite concentrates. 1. Fourier Transform infrared studies , 1982 .

[4]  J. Douglas Faires,et al.  Numerical Analysis , 1981 .

[5]  K. H. Michaelian,et al.  Deconvolution and Curve-Fitting in the Analysis of Complex Spectra: The CH Stretching Region in Infrared Spectra of Coal , 1991 .

[6]  W. F. Maddams,et al.  The Scope and Limitations of Curve Fitting , 1980 .

[7]  W. Cheney,et al.  Numerical analysis: mathematics of scientific computing (2nd ed) , 1991 .

[8]  H. H. Madden.  Comments on the Savitzky-Golay convolution method for least-squares-fit smoothing and differentiation of digital data , 1976 .

[9]  David K. Hildebrand,et al.  Statistical thinking for managers , 1998 .

[10]  C. Alciaturi,et al.  The prediction of coal properties using compressed infrared data from osculating polynomials , 1997 .

[11]  B. Kowalski,et al.  Partial least-squares regression: a tutorial , 1986 .

[12]  R. W. Snyder,et al.  Concerning the Application of FT-IR to the Study of Coal: A Critical Assessment of Band Assignments and the Application of Spectral Analysis Programs , 1981 .

[13]  Peter R. Griffiths,et al.  Resolution enhancement of diffuse reflectance i.r. spectra of coals by Fourier self-deconvolution: 1. C-H stretching and bending modes , 1985 .

[14]  K. H. Michaelian,et al.  Photoacoustic FT-i.r. spectra of separated Western Canadian coal macerals analysis of the CH stretching region by curve-fitting and deconvolution , 1990 .

[15]  J. Lagowski,et al.  Liquid ammonia solutions. X. A Raman study of interactions in the liquid state , 1973 .

[16]  W. C. Hamilton.  Significance tests on the crystallographic R factor , 1965 .

[17]  P. A. Gorry.  General least-squares smoothing and differentiation by the convolution (Savitzky-Golay) method , 1990 .

[18]  Stephen E. Bialkowski,et al.  Generalized digital smoothing filters made easy by matrix calculations , 1989 .

[19]  R. Reyment,et al.  Statistics and Data Analysis in Geology. , 1988 .

[20]  Paul Geladi,et al.  An example of 2-block predictive partial least-squares regression with simulated data , 1986 .