Two preprocessing algorithms for climate time series

We propose two preprocessing algorithms suitable for climate time series. The first algorithm detects outliers based on an autoregressive cost update mechanism. The second one is based on the wavelet transform, a method from pattern recognition. In order to benchmark the algorithms' performance we compare them to existing methods based on a synthetic data set. Eventually, for exemplary purposes, the proposed methods are applied to a data set of high-frequent temperature measurements from Novi Sad, Serbia. The results show that both methods together form a powerful tool for signal preprocessing: In case of solitary outliers the autoregressive cost update mechanism prevails, whereas the wavelet-based mechanism is the method of choice in the presence of multiple consecutive outliers.

[1]  N. Davies,et al.  An influence method for outlier detection applied to time series traffic data , 1995 .

[2]  Michael Unser,et al.  Splines: a perfect fit for signal and image processing , 1999, IEEE Signal Process. Mag..

[3]  Anazida Zainal,et al.  Adaptive and online data anomaly detection for wireless sensor systems , 2014, Knowl. Based Syst..

[4]  Haydar Demirhan,et al.  Missing value imputation for short to mid-term horizontal solar irradiance data , 2018, Applied Energy.

[5]  Witold Pedrycz,et al.  A Piecewise Aggregate pattern representation approach for anomaly detection in time series , 2017, Knowl. Based Syst..

[6]  R. Gencay,et al.  An Introduction to Wavelets and Other Filtering Methods in Finance and Economics , 2001 .

[7]  Maryam Ahmadi,et al.  Automatic denoising of single-trial evoked potentials , 2013, NeuroImage.

[8]  Michael Unser,et al.  Ten good reasons for using spline wavelets , 1997, Optics & Photonics.

[9]  Nirmal K. Bose,et al.  Properties determining choice of mother wavelet , 2005 .

[10]  Reda Alhajj,et al.  Graph-based approach for outlier detection in sequential data and its application on stock market and weather data , 2014, Knowl. Based Syst..

[11]  Eamonn J. Keogh,et al.  Finding surprising patterns in a time series database in linear time and space , 2002, KDD.

[12]  Stephan Schlüter,et al.  Wavelet-based forecasting of ARIMA time series - an empirical comparison of different methods , 2014 .

[13]  M. Mudelsee.  Climate Time Series Analysis: Classical Statistical and Bootstrap Methods , 2010 .

[14]  Ying Wah Teh,et al.  Time-series clustering - A decade review , 2015, Inf. Syst..

[15]  Ka-Ming Lau,et al.  Climate Signal Detection Using Wavelet Transform: How to Make a Time Series Sing , 1995 .

[16]  Canan Bilen,et al.  Wavelet-Based Detection of Outliers in Time Series , 2002 .

[17]  David Salesin,et al.  Wavelets for computer graphics: a primer.1 , 1995, IEEE Computer Graphics and Applications.

[18]  H SalesinDavid,et al.  Wavelets for Computer Graphics , 1995 .

[19]  Martin Meckesheimer,et al.  Automatic outlier detection for time series: an application to sensor data , 2007, Knowledge and Information Systems.

[20]  William W. Hager,et al.  A Nonmonotone Line Search Technique and Its Application to Unconstrained Optimization , 2004, SIAM J. Optim..

[21]  Stephan Schlueter,et al.  A long-term/short-term model for daily electricity prices with dynamic volatility , 2010 .

[22]  Xiao-yun Chen,et al.  Multi-scale anomaly detection algorithm based on infrequent pattern of time series , 2008 .

[23]  H. Herwartz,et al.  On the Predictive Information of Futures' Prices: A Wavelet‐Based Assessment , 2016 .

[24]  Charu C. Aggarwal,et al.  Outlier Detection for Temporal Data , 2014, Outlier Detection for Temporal Data.

[25]  Helena Veiga,et al.  Wavelet-based detection of outliers in financial time series , 2010, Comput. Stat. Data Anal..

[26]  I. Johnstone,et al.  Ideal spatial adaptation by wavelet shrinkage , 1994 .

[27]  W. Kilmer.  A Friendly Guide To Wavelets , 1998, Proceedings of the IEEE.

[28]  E. J. Stollnitz,et al.  Wavelets for Computer Graphics: A Primer Part 2 , 1995 .

[29]  S. Mallat.  A wavelet tour of signal processing , 1998 .

[30]  David J. Parsons,et al.  An improved wavelet–ARIMA approach for forecasting metal prices , 2014 .

[31]  Claude E. Duchon,et al.  Time Series Analysis in Meteorology and Climatology: An Introduction , 2011 .

[32]  D. L. Donoho,et al.  Ideal spacial adaptation via wavelet shrinkage , 1994 .

[33]  Shian-Shyong Tseng,et al.  Two-phase clustering process for outliers detection , 2001, Pattern Recognit. Lett..

[34]  Michael R. Chernick,et al.  Wavelet Methods for Time Series Analysis , 2001, Technometrics.

[35]  C. Torrence,et al.  A Practical Guide to Wavelet Analysis. , 1998 .

[36]  Basav Roychoudhury,et al.  Handling missing values: A study of popular imputation packages in R , 2018, Knowl. Based Syst..