PSOM: Periodic Self-Organizing Maps for unsupervised anomaly detection in periodic time series

Nowadays, systems providing user-oriented services often demonstrate periodic patterns due to the repetitive behaviors from people's daily routines. The monitoring data of such systems are time series of observations that record observed system status at sampled times during each day. The periodic feature and multidimensional character of such monitoring data can be well utilized by anomaly detection algorithms to enhance their detection capability. The data periodicity can be used to provide proactive anomaly prediction capability and the correlation among multidimensional series can provide more accurate results than processing the observations separately. However, existing anomaly detection methods only handle one dimensional series and do not consider the data periodicity. In addition, they often require sufficient labelled data to train the models before they can be used. In this paper, we present an unsupervised anomaly detection algorithm called Periodic Self-Organizing Maps (PSOM) to detect anomalies in periodic time series. PSOMs can be used to detect anomalies in multidimensional periodic series as well as one dimensional periodic series and aperiodic series. Our real data evaluation shows that the PSOM outperforms other supervised methods such as SARIMA and Holt-Winters method.

[1]  David Brumley,et al.  SplitScreen: Enabling efficient, distributed malware detection , 2010, Journal of Communications and Networks.

[2]  George Athanasopoulos,et al.  Forecasting: principles and practice , 2013 .

[3]  VARUN CHANDOLA,et al.  Anomaly detection: A survey , 2009, CSUR.

[4]  Armando Fox,et al.  Capturing, indexing, clustering, and retrieving system history , 2005, SOSP '05.

[5]  Parag Kulkarni,et al.  Intrusion Detection System using Self Organizing Maps , 2009, 2009 International Conference on Intelligent Agent & Multi-Agent Systems.

[6]  Network behaviour anomaly detection using Holt-Winters algorithm , 2011, 2011 International Conference for Internet Technology and Secured Transactions.

[7]  Erik Elmroth,et al.  Real-time detection of performance anomalies for cloud services , 2016, 2016 IEEE/ACM 24th International Symposium on Quality of Service (IWQoS).

[8]  Everette S. Gardner,et al.  Exponential smoothing: The state of the art , 1985 .

[9]  Fred Spiring,et al.  Introduction to Statistical Quality Control , 2007, Technometrics.

[10]  Teuvo Kohonen,et al.  Self-Organizing Maps , 2010 .

[11]  Erik Elmroth,et al.  Apex Lake: A Framework for Enabling Smart Orchestration , 2015, Middleware Industry.

[12]  Gwilym M. Jenkins,et al.  Time series analysis, forecasting and control , 1972 .

[13]  Thomas Hill Statistics: Methods and Applications , 2005 .

[14]  Gwilym M. Jenkins,et al.  Time series analysis, forecasting and control , 1971 .

[15]  Maciej Szmit,et al.  Usage of Modified Holt-Winters Method in the Anomaly Detection of Network Traffic: Case Studies , 2012, J. Comput. Networks Commun..

[16]  Xiaohui Gu,et al.  UBL: unsupervised behavior learning for predicting performance anomalies in virtualized cloud systems , 2012, ICAC '12.

[17]  Oliver W. W. Yang,et al.  Wireless traffic modeling and prediction using seasonal ARIMA models , 2003, IEEE International Conference on Communications, 2003. ICC '03..

[18]  Paul Goodwin,et al.  The Holt-Winters Approach to Exponential Smoothing: 50 Years Old and Going Strong , 2010 .

[19]  K. Piromsopa,et al.  SARIMA based network bandwidth anomaly detection , 2012, 2012 Ninth International Conference on Computer Science and Software Engineering (JCSSE).

[20]  Martin Chovanec,et al.  INTRUSION DETECTION SYSTEM USING SELF ORGANIZING MAP , 2006 .