Selective iteratively reweighted quantile regression for baseline correction

AbstractExtraction of qualitative and quantitative information from large numbers of analytical signals is difficult with drifted baselines, particularly in multivariate analysis. Baseline drift obscures and “fuzzies” signals, and even deteriorates analytical results. In order to obtain accurate and clear results, some effective methods should be proposed and implemented to perform baseline correction before conducting further data analysis. However, most of the classic methods require user intervention or are prone to variability, especially with low signal-to-noise signals. In this study, a novel baseline correction algorithm based on quantile regression and iteratively reweighting strategy is proposed. This does not require user intervention and prior information, such as peak detection. The iteratively reweighting strategy iteratively changes weights of residuals between fitted baseline and original signals. After a series of tests and comparisons with several other popular methods, using various kinds of analytical signals, the proposed method is found to be fast, flexible, robust, and easy to use both in simulated and real datasets. Figureᅟ

[1]  Vincent Mazet,et al.  Background removal from spectra by designing and minimising a non-quadratic cost function , 2005 .

[2]  Edmund Taylor Whittaker On a New Method of Graduation , 1922, Proceedings of the Edinburgh Mathematical Society.

[3]  Joseph L. Zinnes,et al.  Theory and Methods of Scaling. , 1958 .

[4]  S. Jun Weak identification robust tests in an instrumental quantile model , 2008 .

[5]  Alan G. Ryder,et al.  Comparison of Derivative Preprocessing and Automated Polynomial Baseline Correction Method for Classification and Quantification of Narcotics in Solid Mixtures , 2006, Applied spectroscopy.

[6]  W. Dietrich,et al.  Fast and precise automatic baseline correction of one- and two-dimensional nmr spectra , 1991 .

[7]  A. Mahadevan-Jansen,et al.  Automated Method for Subtraction of Fluorescence from Biological Raman Spectra , 2003, Applied spectroscopy.

[8]  R. Koenker,et al.  Regression Quantiles , 2007 .

[9]  R. Koenker Quantile regression for longitudinal data , 2004 .

[10]  Qing-Song Xu,et al.  Morphological weighted penalized least squares for background correction. , 2013, The Analyst.

[11]  M. Morhác,et al.  Peak Clipping Algorithms for Background Estimation in Spectroscopic Data , 2008, Applied spectroscopy.

[12]  John R. Gilbert,et al.  Sparse Matrices in MATLAB: Design and Implementation , 1992, SIAM J. Matrix Anal. Appl..

[13]  P. Green Iteratively reweighted least squares for maximum likelihood estimation , 1984 .

[14]  P. Eilers,et al.  New background correction method for liquid chromatography with diode array detection, infrared spectroscopic detection and Raman spectroscopic detection. , 2004, Journal of chromatography. A.

[15]  Israel Schechter,et al.  Correction for nonlinear fluctuating background in monovariable analytical systems , 1995 .

[16]  J. Jorgenson,et al.  Median filtering for removal of low-frequency background drift. , 1993, Analytical chemistry.

[17]  J. Kruskal Multidimensional scaling by optimizing goodness of fit to a nonmetric hypothesis , 1964 .

[18]  Sirish L Shah,et al.  Robust baseline correction algorithm for signal dense NMR spectra. , 2007, Journal of magnetic resonance.

[19]  S. Jun Local structural quantile effects in a model with a nonseparable control variable , 2009 .

[20]  W. Cai,et al.  Intelligent background correction using an adaptive lifting wavelet , 2013 .

[21]  Robert W. Field,et al.  Baseline subtraction using robust local regression estimation , 2001 .

[22]  H. Goldstein Restricted unbiased iterative generalized least-squares estimation , 1989 .

[23]  Harold J. Annegarn,et al.  Algorithm for fitting XRF, SEM and PIXE X-ray spectra backgrounds , 1996 .

[24]  Jeffrey S. Morris,et al.  Quality control and peak finding for proteomics data collected from nipple aspirate fluid by surface-enhanced laser desorption and ionization. , 2003, Clinical chemistry.

[25]  Yi-Zeng Liang,et al.  Baseline correction using adaptive iteratively reweighted penalized least squares. , 2010, The Analyst.

[26]  Y. X. Wang,et al.  Nuclear Instruments and Methods in Physics Research Section B : Beam Interactions with Materials and Atoms , 2018 .

[27]  Dor Ben-Amotz,et al.  Enhanced Chemical Classification of Raman Images in the Presence of Strong Fluorescence Interference , 2000 .

[28]  Yi-Zeng Liang,et al.  A roughness penalty approach and its application to noisy hyphenated chromatographic two‐way data , 1999 .

[29]  J. Grindlay,et al.  New spectral classification technique for x-ray sources: Quantile analysis , 2004, astro-ph/0406463.

[30]  Yizeng Liang,et al.  [Raman spectroscopy fluorescence background correction and its application in clustering analysis of medicines]. , 2010, Guang pu xue yu guang pu fen xi = Guang pu.

[31]  Paul H. C. Eilers,et al.  Quantile smoothing of array CGH data , 2005, Bioinform..

[32]  T. Wunderli Total variation time flow with quantile regression for image restoration , 2013 .

[33]  R. Koenker,et al.  The Gaussian hare and the Laplacian tortoise: computability of squared-error versus absolute-error estimators , 1997 .

[34]  Donald B. Rubin Iteratively Reweighted Least Squares , 2006 .

[35]  Four (Pathological) Examples in Asymptotic Statistics , 1984 .

[36]  Kristian Hovde Liland,et al.  Customized baseline correction , 2011 .

[37]  Ł. Komsta,et al.  Comparison of Several Methods of Chromatographic Baseline Removal with a New Approach Based on Quantile Regression , 2011, Chromatographia.

[38]  Stephen J. Callister,et al.  Normalization approaches for removing systematic biases associated with mass spectrometry and label-free proteomics. , 2006, Journal of proteome research.

[39]  P. Eilers A perfect smoother. , 2003, Analytical chemistry.

[40]  P. Green Marketing Applications of MDS: Assessment and Outlook , 1975 .

[41]  Yu Ryan Yue,et al.  For a list of recent papers see the backpages of this paper. Bayesian Semiparametric Additive Quantile Regression , 2022 .

[42]  Golotvin,et al.  Improved baseline recognition and modeling of FT NMR spectra , 2000, Journal of magnetic resonance.

[43]  V. Chernozhukov,et al.  Instrumental variable quantile regression: A robust inference approach , 2008 .

[44]  Manuel Martín-Pastor,et al.  A new general-purpose fully automatic baseline-correction procedure for 1D and 2D NMR data. , 2006, Journal of magnetic resonance.

[45]  David Budgen,et al.  Software design , 2020, International computer science series.

[46]  R. Koenker Quantile Regression: Name Index , 2005 .

[47]  P. Holland,et al.  Robust regression using iteratively reweighted least-squares , 1977 .

[48]  A Jirasek,et al.  Accuracy and Precision of Manual Baseline Determination , 2004, Applied spectroscopy.

[49]  Royston Goodacre,et al.  Discrimination of Bacteria Using Pyrolysis-gas Chromatography-differential Mobility Spectrometry (py-gc-dms) and Chemometrics , 2008 .

[50]  Yi-Zeng Liang,et al.  Comments on the Baseline Removal Method Based on Quantile Regression and Comparison of Several Methods , 2012, Chromatographia.

[51]  Hwa-Lung Yu,et al.  Quantile-based Bayesian maximum entropy approach for spatiotemporal modeling of ambient air quality levels. , 2013, Environmental science & technology.

[52]  Yi-Zeng Liang,et al.  An intelligent background-correction algorithm for highly fluorescent samples in Raman spectroscopy , 2010 .

[53]  P. Eilers Parametric time warping. , 2004, Analytical chemistry.

[54]  Wensheng Cai,et al.  Wavelet transform and its applications in high performance liquid chromatography (HPLC) analysis , 1999 .