A new hybrid strategy for constructing a robust calibration model for near-infrared spectral analysis

A new hybrid algorithm is proposed for construction of a high-quality calibration model for near-infrared (NIR) spectra that is robust against both spectral interference (including background and noise) and multiple outliers. The algorithm is a combination of continuous wavelet transform (CWT) and a modified iterative reweighted PLS (mIRPLS) procedure. In the proposed algorithm the spectral interference is filtered by CWT at the first stage then mIRPLS is proposed to detect the multiple outliers in the CWT domain. Compared with the original IRPLS method, mIRPLS does not need to adjust variable parameters to achieve optimum calibration results, which makes it very convenient to perform in practice. The final PLS model is constructed robustly because both the spectral interference and multiple outliers are eliminated. In order to validate the effectiveness and universality of the algorithm, it was applied to two different sets of NIR spectra. The results indicate that the proposed strategy can greatly enhance the robustness and predictive ability of NIR spectral analysis.

[1]  P. Rousseeuw,et al.  Least median of squares: a robust method for outlier and model error detection in regression and calibration , 1986 .

[2]  Xueguang Shao,et al.  Removal of major interference sources in aqueous near-infrared spectroscopy techniques , 2004, Analytical and bioanalytical chemistry.

[3]  Stuart Licht,et al.  Fundamental baseline variations in aqueous near-infrared analysis , 1999 .

[4]  Qing-Song Xu,et al.  Generalized PLS regression , 2001 .

[5]  G. W. Small,et al.  Multivariate calibration standardization across instruments for the determination of glucose by Fourier transform near-infrared spectrometry. , 2003, Analytical chemistry.

[6]  X. Shao,et al.  A novel method to calculate the approximate derivative photoacoustic spectrum using continuous wavelet transform , 2000, Fresenius' journal of analytical chemistry.

[7]  H. B. Ding,et al.  Differentiation of Beef and Kangaroo Meat by Visible/Near-Infrared Reflectance Spectroscopy , 1999 .

[8]  Barbara Rasco,et al.  Nondestructive Prediction of Moisture and Sodium Chloride in Cold Smoked Atlantic Salmon (Salmo salar) , 2002 .

[9]  H. J. H. Macfie,et al.  A robust PLS procedure , 1992 .

[10]  Israel Schechter,et al.  Correction for nonlinear fluctuating background in monovariable analytical systems , 1995 .

[11]  Gregory R. Phillips,et al.  Comparison of conventional and robust regression in analysis of chemical data , 1983 .

[12]  L. Rodriguez-Saona,et al.  Rapid analysis of sugars in fruit juices by FT-NIR spectroscopy. , 2001, Carbohydrate research.

[13]  S. Rutan,et al.  Characterization of the sources of variation affecting near-infrared spectroscopy using chemometric methods. , 1998, Analytical chemistry.

[14]  T. Iwata,et al.  Elimination of the Uninformative Calibration Sample Subset in the Modified UVE(Uninformative Variable Elimination)–PLS (Partial Least Squares) Method , 2001, Analytical sciences : the international journal of the Japan Society for Analytical Chemistry.

[15]  A. Cavinato,et al.  Detection of sodium chloride in cured salmon roe by SW-NIR spectroscopy. , 2001, Journal of agricultural and food chemistry.

[16]  Desire L. Massart,et al.  MULTIPLE OUTLIER DETECTION REVISITED , 1998 .

[17]  Yi-Zeng Liang,et al.  Robust methods for multivariate analysis — a tutorial review , 1996 .

[18]  Tetsuo Iwata,et al.  Application of the Modified UVE-PLS Method for a Mid-Infrared Absorption Spectral Data Set of Water—Ethanol Mixtures , 2000 .

[19]  Ding Hb,et al.  Near-infrared spectroscopic technique for detection of beef hamburger adulteration. , 2000 .

[20]  Douglas B. Kell,et al.  The effect of heteroscedastic noise on the chemometric modelling of frequency domain data , 1998 .

[21]  Chen Da,et al.  Elimination of interference information by a new hybrid algorithm for quantitative calibration of near infrared spectra. , 2003, The Analyst.

[22]  M A Arnold,et al.  Near-infrared spectroscopic measurement of physiological glucose levels in variable matrices of protein and triglycerides. , 1996, Analytical chemistry.

[23]  Chiara Casolino,et al.  The refinement of PLS models by iterative weighting of predictor variables and objects , 2003 .

[24]  Yizeng Liang,et al.  Uniform design and its applications in chemistry and chemical engineering , 2001 .

[25]  Desire L. Massart,et al.  Comparison of semirobust and robust partial least squares procedures , 1995 .

[26]  Steven D. Brown,et al.  Robust Calibration with Respect to Background Variation , 2001 .

[27]  Xueguang Shao,et al.  A general approach to derivative calculation using wavelet transform , 2003 .

[28]  M. Hubert,et al.  Robust methods for partial least squares regression , 2003 .

[29]  Randy J. Pell,et al.  Multiple outlier detection for multivariate calibration using robust statistical techniques , 2000 .

[30]  Barbara Rasco,et al.  Near infrared spectroscopy: a new tool for studying physical and chemical properties of polysaccharide gels , 2003 .

[31]  Bernhard Lendl,et al.  Determination of oil and water content in olive pomace using near infrared and Raman spectrometry. A comparative study , 2004, Analytical and bioanalytical chemistry.

[32]  Steven D. Brown,et al.  Wavelet analysis applied to removing non‐constant, varying spectroscopic background in multivariate calibration , 2002 .

[33]  David J. Cummins,et al.  Iteratively reweighted partial least squares: A performance analysis by monte carlo simulation , 1995 .

[34]  Xueguang Shao,et al.  Continuous Wavelet Transform Applied to Removing the Fluctuating Background in Near-Infrared Spectra , 2004, J. Chem. Inf. Model..