A new automatic threshold selecting criteria for spectroscopy data processing

Abstract A nonparametric and unsupervised method of automatic threshold selection to eliminate noise for spectroscopy data processing is described in this paper. A detecting scheme, named bi-trapezoid criteria, is devised where the threshold is selected according to the turning corner present on the uprising intensity trace. This selecting procedure is very simple, utilizing only basic summation on the sorted intensity sequence to optimize threshold for distinguishing between noises and signals. This approach is effective in selecting appropriate value to filter noise when the distribution of noises is not preconditioned or the gap between signals and noises is not obvious. Testing on both artificial and authentic data under specific quantitative evaluation condition shows that this new method performs better than previous ones.

[1]  G. Zack,et al.  Automatic measurement of sister chromatid exchange frequency. , 1977, The journal of histochemistry and cytochemistry : official journal of the Histochemistry Society.

[2]  I. Johnstone,et al.  Ideal spatial adaptation by wavelet shrinkage , 1994 .

[3]  P. Eilers.  A perfect smoother. , 2003, Analytical chemistry.

[4]  Xiaoguang Zhou,et al.  A concise iterative method using the Bezier technique for baseline construction. , 2015, The Analyst.

[5]  M L Mendelsohn,et al.  THE ANALYSIS OF CELL IMAGES * , 1966, Annals of the New York Academy of Sciences.

[6]  I. Johnstone,et al.  Wavelet Threshold Estimators for Data with Correlated Noise , 1997 .

[7]  I. Johnstone,et al.  Adapting to Unknown Smoothness via Wavelet Shrinkage , 1995 .

[8]  N. Otsu.  A threshold selection method from gray level histograms , 1979 .

[9]  Makoto Nagao,et al.  Line extraction and pattern detection in a photograph , 1969, Pattern Recognit..

[10]  A K Wagner,et al.  Segmented regression analysis of interrupted time series studies in medication use research , 2002, Journal of clinical pharmacy and therapeutics.

[11]  Robert T. Downs,et al.  The power of databases: The RRUFF project , 2016 .

[12]  Paul H. C. Eilers,et al.  Penalized regression in action: Estimating pollution roses from daily averages , 1991 .

[13]  Sylvain Sardy.  Minimax threshold for denoising complex signals with Waveshrink , 2000, IEEE Trans. Signal Process..

[14]  Paul H. C. Eilers,et al.  Generalized linear regression on sampled signals and curves: a P -spline approach , 1999 .