Improving Quality of Service Parameter Prediction with Preliminary Outlier Detection and Elimination

Wide-spread real-time applications make it necessary for service providers to guarantee QoS parameters. This requires precise forecast of network traffic. A possible method of the forecast is measuring traffic then analyzing it and fitting model to the measured data, finally predicting the observed parameter using the fitted model. The efficiency of the prediction is decreased by outlying samples (so called outliers) found in the time series data. We developed a new tool that is able to detect and eliminate the outliers from time series data. This tool is capable to handle large sets of time series data fast and efficiently. We also propose a method to predict QoS parameters using the ARIMA (Auto-Regressive Integrated Moving Average) model, which is based on a preliminary detection and elimination of outliers. We have proven that this method increases the efficiency of the prediction significantly by forecasting real measurement data.

[1]  Anil K. Jain,et al.  Data clustering: a review , 1999, CSUR.

[2]  Hans-Peter Kriegel,et al.  LOF: identifying density-based local outliers , 2000, SIGMOD '00.

[3]  J. R. Koehler,et al.  Modern Applied Statistics with S-Plus. , 1996 .

[4]  P. Young,et al.  Time series analysis, forecasting and control , 1972, IEEE Transactions on Automatic Control.

[5]  Raymond T. Ng,et al.  Algorithms for Mining Distance-Based Outliers in Large Datasets , 1998, VLDB.

[6]  Christos Faloutsos,et al.  LOCI: fast outlier detection using the local correlation integral , 2003, Proceedings 19th International Conference on Data Engineering (Cat. No.03CH37405).

[7]  Theodore Johnson,et al.  Fast Computation of 2-Dimensional Depth Contours , 1998, KDD.

[8]  Víctor Gómez,et al.  Program TRAMO 'Time Series Regression with Arima Noise, Missing Observations, and Outliers'. Instructions for the User , 1994 .

[9]  Agustín Maravall,et al.  A tool for quality control of time series data. Program TERROR , 2003 .

[10]  Douglas M. Hawkins.  Identification of Outliers , 1980, Monographs on Applied Probability and Statistics.

[11]  Peter J. Rousseeuw,et al.  Robust Regression and Outlier Detection , 2005, Wiley Series in Probability and Statistics.

[12]  Calvin L. Williams,et al.  Modern Applied Statistics with S-Plus , 1997 .

[13]  Ilka Miloucheva,et al.  A practical approach to forecast Quality of Service parameters considering outliers , 2003 .

[14]  Richard A. Davis,et al.  Introduction to time series and forecasting , 1998 .

[15]  Raymond T. Ng,et al.  Distance-based outliers: algorithms and applications , 2000, The VLDB Journal.