Outlier detection in chemical data by fractal analysis

A new outlier detection technique has been created which functions effectively with non‐bilinear data, a situation in which more common detection techniques have difficulty. The method involves the calculation of the data set's fractal dimension in the latent variable score space. For batch data sets a full cross‐validation is used to find outliers, while process data are analyzed using a training set. If a spectrum in either case significantly changes the fractal dimension statistically, then it is identified as an outlier. The technique shows good results for surface plasmon resonance data sets and has been shown to be effective in detecting even subtle outliers. Copyright © 2004 John Wiley & Sons, Ltd.

[1]  R. Dluhy,et al.  Pressure-Dependent Changes in the Infrared C—H Vibrations of Monolayer Films at the Air/Water Interface Revealed by Two-Dimensional Infrared Correlation Spectroscopy , 2000 .

[2]  Leo H. Chiang,et al.  Exploring process data with the use of robust outlier detection algorithms , 2003 .

[3]  Jan Frøyland,et al.  Introduction to Chaos and Coherence , 1992 .

[4]  S. Morgan,et al.  Outlier detection in multivariate analytical chemical data. , 1998, Analytical chemistry.

[5]  Peter J. Rousseeuw,et al.  Robust regression and outlier detection , 1987 .

[6]  H. Raether.  Surface Plasmons on Smooth and Rough Surfaces and on Gratings , 1988 .

[7]  K. Booksh,et al.  Influence of Wavelength-Shifted Calibration Spectra on Multivariate Calibration Models , 2004, Applied spectroscopy.

[8]  M. A. Czarnecki.  Two-Dimensional Correlation Spectroscopy: Effect of Band Position, Width, and Intensity Changes on Correlation Intensities , 2000 .

[9]  Arun S. Mujumdar,et al.  Introduction to Surface Chemistry and Catalysis , 1994 .

[10]  J. E. Jackson.  A User's Guide to Principal Components , 1991 .

[11]  Zbigniew R. Struzik,et al.  Wavelet transform based multifractal formalism in outlier detection and localisation for financial time series , 2002 .

[12]  E. Bacry,et al.  Wavelets and multifractal formalism for singular signals: Application to turbulence data. , 1991, Physical review letters.

[13]  Randall D. Tobias,et al.  Chemometrics: A Practical Guide , 1998, Technometrics.

[14]  Karl S. Booksh,et al.  Calibration of Surface Plasmon Resonance Refractometers Using Locally Weighted Parametric Regression , 1997 .

[15]  Peter J. Rousseeuw,et al.  Robust Regression and Outlier Detection , 2005, Wiley Series in Probability and Statistics.

[16]  Heinz-Otto Peitgen,et al.  The science of fractal images , 2011 .

[17]  M. Hubert,et al.  A robust PCR method for high‐dimensional regressors , 2003 .

[18]  W. Dixon,et al.  Simplified Statistics for Small Numbers of Observations , 1951 .

[19]  J. Edward Jackson,et al.  A User's Guide to Principal Components: Jackson/User's Guide to Principal Components , 2004 .

[20]  Y. Ozaki,et al.  Pattern Recognitions of Band Shifting, Overlapping, and Broadening Using Global Phase Description Derived from Generalized Two-Dimensional Correlation Spectroscopy , 2002 .

[21]  Richard G. Brereton,et al.  Chemometrics: Data Analysis for the Laboratory and Chemical Plant , 2003 .

[22]  James Gleick.  Chaos: Making a New Science , 1987 .