A Streaming Data Anomaly Detection Analytic Engine for Mobile Network Management

With emerging Network Functions Virtualization (NFV), Software Defined Networking (SDN) paradigms in Network Management (NM), new network devices, features can immediately become available. Available network resources, services can be altered, optimized in real time to gain the maximum benefit. However, this requires real time analytics information sent to SDN controllers rather than traditional manual offline or batch analytics which deliver outputs in hourly or monthly in NM Systems. As a result, real time analytic is becoming a critical element for NM. In this paper, we describe a configuration free (i.e., non-parametric) streaming data anomaly detection analytic engine for automatic NM system development. We describe the design principles, innovative algorithm design, architecture, implementation of the engine in relation to streaming data, mobile NM. Finally, we present use cases, evaluation results.

[1]  Gianmarco De Francisci Morales,et al.  Big Data Stream Learning with SAMOA , 2014, 2014 IEEE International Conference on Data Mining Workshop.

[2]  Sidath Handurukande,et al.  RPig: Concise Programming Framework by Integrating R with Pig for Big Data Analytics , 2015 .

[3]  Rob J Hyndman,et al.  Rainbow Plots, Bagplots, and Boxplots for Functional Data , 2010 .

[4]  Yannis Manolopoulos,et al.  Continuous outlier detection in data streams: an extensible framework and state-of-the-art algorithms , 2013, SIGMOD '13.

[5]  Haifeng Chen,et al.  Correlating real-time monitoring data for mobile network management , 2008, 2008 International Symposium on a World of Wireless, Mobile and Multimedia Networks.

[6]  Jianmin Jiang,et al.  Network Anomaly Detection Using One Class Support Vector Machine , 2008 .

[7]  Geoff Holmes,et al.  MOA: Massive Online Analysis , 2010, J. Mach. Learn. Res..

[8]  Lon-Mu Liu,et al.  Joint Estimation of Model Parameters and Outlier Effects in Time Series , 1993 .

[9]  Yannis Manolopoulos,et al.  Continuous monitoring of distance-based outliers over data streams , 2011, 2011 IEEE 27th International Conference on Data Engineering.

[10]  Michiaki Hayashi,et al.  In-network real-time performance monitoring with distributed event processing , 2014, 2014 IEEE Network Operations and Management Symposium (NOMS).

[11]  João Gama,et al.  Distributed Adaptive Model Rules for mining big data streams , 2014, 2014 IEEE International Conference on Big Data (Big Data).

[12]  Arun Kejariwal,et al.  A Novel Technique for Long-Term Anomaly Detection in the Cloud , 2014, HotCloud.

[13]  Albert Bifet,et al.  Massive Online Analysis , 2009 .

[14]  Suresh Sood.  Book Review: Behavior Computing: Modeling, Analysis, Mining and Decision , 2012, IEEE Intell. Informatics Bull..

[15]  Ulf Lindqvist,et al.  DCAD: Dynamic Cell Anomaly Detection for operational cellular networks , 2014, 2014 IEEE Network Operations and Management Symposium (NOMS).

[16]  VARUN CHANDOLA,et al.  Anomaly detection: A survey , 2009, CSUR.

[17]  Ulf Lindqvist,et al.  Detecting anomalies in cellular networks using an ensemble method , 2013, Proceedings of the 9th International Conference on Network and Service Management (CNSM 2013).

[18]  Irma J. Terpenning,et al.  STL : A Seasonal-Trend Decomposition Procedure Based on Loess , 1990 .

[19]  Thomas F. La Porta,et al.  On cellular botnets: measuring the impact of malicious devices on a cellular network core , 2009, CCS.

[20]  Stephen L. R. Ellison,et al.  Practical Statistics for the Analytical Scientist: A Bench Guide , 2009 .

[21]  Fan Tongke.  Hadoop-Based Data Analysis System Architecture for Telecom Enterprises , 2013, 2013 International Conference on Computational and Information Sciences.