Big Data Driven Anomaly Detection for Cellular Networks

Anomaly detection for large scale cellular networks can be used by network operators to optimize network performance and enhance mobile user experience. This paper aims at detecting user anomalies from spatio-temporal cell phone activity data. We design an approach combining time series analysis and machine learning to extract the traffic patterns of areal units. This approach can cluster areal units with similar traffic patterns and segment a city into distinct groups. Then, in grouped-areas, we use a clustering technique to detect anomalous behaviors of the cellular network and verify the accuracy of the results using ground truth information collected from online sources. The results indicate that anomalies are associated with abruptly high or unexpected traffic demand at a specific location and time. In addition, we obtain anomaly-free data by removing anomalous data and train a decomposed traffic prediction model. It is observed that the prediction model trained with anomaly-free data can achieve lower normalized mean square error (NMSE), i.e., higher prediction accuracy, than the model trained with anomalous data.

[1]  Qiang Liu,et al.  Acquisition of channel state information in heterogeneous cloud radio access networks: challenges and research directions , 2015, IEEE Wireless Communications.

[2]  Zhongshan Zhang,et al.  Call Detail Records Driven Anomaly Detection and Traffic Prediction in Mobile Cellular Networks , 2018, IEEE Access.

[3]  Engin Zeydan,et al.  Anomaly Detection In Cellular Network Data Using Big Data Analytics , 2014 .

[4]  Jian Gong,et al.  A Time-Series Decomposed Model of Network Traffic , 2005, ICNC.

[5]  Joel J. P. C. Rodrigues,et al.  Anomaly detection using baseline and K-means clustering , 2010, SoftCOM 2010, 18th International Conference on Software, Telecommunications and Computer Networks.

[6]  Anil K. Jain,et al.  Data clustering: a review , 1999, CSUR.

[7]  Xinlei Chen,et al.  Characterizing and Predicting Individual Traffic Usage of Mobile Application in Cellular Network , 2018, UbiComp/ISWC Adjunct.

[8]  R. Nagaraj,et al.  Anomaly Detection via Online Oversampling Principal Component Analysis , 2014 .

[9]  Yannis A. Dimitriadis,et al.  Anomaly Detection in Network Traffic Based on Statistical Inference and \alpha-Stable Modeling , 2011, IEEE Transactions on Dependable and Secure Computing.

[10]  Jiannong Cao,et al.  On-Line Anomaly Detection With High Accuracy , 2018, IEEE/ACM Transactions on Networking.

[11]  David J. Ketchen,et al.  THE APPLICATION OF CLUSTER ANALYSIS IN STRATEGIC MANAGEMENT RESEARCH: AN ANALYSIS AND CRITIQUE , 1996 .

[12]  Zhi-Quan Luo,et al.  Bilinear Factor Matrix Norm Minimization for Robust PCA: Algorithms and Applications , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[13]  Moses Garuba,et al.  Big Data Analytics for User-Activity Analysis and User-Anomaly Detection in Mobile Wireless Network , 2017, IEEE Transactions on Industrial Informatics.

[14]  Konstantina Papagiannaki,et al.  Structural analysis of network traffic flows , 2004, SIGMETRICS '04/Performance '04.

[15]  Mohiuddin Ahmed,et al.  A survey of network anomaly detection techniques , 2016, J. Netw. Comput. Appl..

[16]  Li Ling Ko,et al.  Anomaly Detection and Attribution in Networks With Temporally Correlated Traffic , 2018, IEEE/ACM Transactions on Networking.

[17]  Jun Yang,et al.  Improved traffic detection with support vector machine based on restricted Boltzmann machine , 2017, Soft Comput..

[18]  Z. Sun,et al.  Traffic predictability based on ARIMA/GARCH model , 2006, 2006 2nd Conference on Next Generation Internet Design and Engineering, 2006. NGI '06..

[19]  Jennifer Rexford,et al.  Sensitivity of PCA for traffic anomaly detection , 2007, SIGMETRICS '07.

[20]  Yuefei Zhu,et al.  A Deep Learning Approach for Intrusion Detection Using Recurrent Neural Networks , 2017, IEEE Access.

[21]  Stefan Savage,et al.  California fault lines: understanding the causes and impact of network failures , 2010, SIGCOMM '10.

[22]  Mia Hubert,et al.  Anomaly detection by robust statistics , 2017, WIREs Data Mining Knowl. Discov..

[23]  BASIL AsSADHAN,et al.  Anomaly Detection Based on LRD Behavior Analysis of Decomposed Control and Data Planes Network Traffic Using SOSS and FARIMA Models , 2017, IEEE Access.

[24]  Zied Elouedi,et al.  Naive Bayes vs decision trees in intrusion detection systems , 2004, SAC '04.

[25]  Victor C. M. Leung,et al.  Deep-Reinforcement-Learning-Based Optimization for Cache-Enabled Opportunistic Interference Alignment Wireless Networks , 2017, IEEE Transactions on Vehicular Technology.

[26]  Jing Wang,et al.  Spatiotemporal modeling and prediction in cellular networks: A big data enabled deep learning approach , 2017, IEEE INFOCOM 2017 - IEEE Conference on Computer Communications.

[27]  J. A. Hartigan,et al.  A k-means clustering algorithm , 1979 .

[28]  Marco Fiore,et al.  Classifying call profiles in large-scale mobile traffic datasets , 2014, IEEE INFOCOM 2014 - IEEE Conference on Computer Communications.

[29]  Minas Gjoka,et al.  On the Decomposition of Cell Phone Activity Patterns and their Connection with Urban Ecology , 2015, MobiHoc.

[30]  Ioannis Ch. Paschalidis,et al.  Statistical Anomaly Detection via Composite Hypothesis Testing for Markov Models , 2017, IEEE Transactions on Signal Processing.