Mobile Network Failure Event Detection and Forecasting With Multiple User Activity Data Sets

As the demand for mobile network services increases, immediate detection and forecasting of network failure events have become important problems for service providers. Several event detection approaches have been proposed to tackle these problems by utilizing social data. However, these approaches have not tried to solve event detection and forecasting problems from multiple data sets, such as web access logs and search queries. In this paper, we propose a machine learning approach that incorporates multiple user activity data into detecting and forecasting failure events. Our approach is based on a two-level procedure. First, we introduce a novel feature construction method that treats both the imbalanced label problem and the data sparsity problem of user activity data. Second, we propose a model ensemble method that combines outputs of supervised and unsupervised learning models for each data set and gives accurate predictions of network service outage. We demonstrate the effectiveness of the proposed models by extensive experiments with realworld failure events occurred at a network service provider in Japan and three user activity data sets.

[1]  Mark Dredze,et al.  Combining Search, Social Media, and Traditional Data Sources to Improve Influenza Surveillance , 2015, PLoS Comput. Biol..

[2]  Akihiro Nakao,et al.  Development of Failure Detection System for Network Control using Collective Intelligence of Social Networking Service in Large-Scale Disasters , 2016, HT.

[3]  Wael Khreich,et al.  A Survey of Techniques for Event Detection in Twitter , 2015, Comput. Intell..

[4]  Navendu Jain,et al.  Understanding network failures in data centers: measurement, analysis, and implications , 2011, SIGCOMM.

[5]  Zihui Ge,et al.  Listen to me if you can: tracking user experience of mobile network on social media , 2010, IMC '10.

[6]  Chang-Tien Lu,et al.  Determining Relative Airport Threats from News and Social Media , 2017, AAAI.

[7]  Michael McGill,et al.  Introduction to Modern Information Retrieval , 1983 .

[8]  Jieping Ye,et al.  Multi-Task Learning for Spatio-Temporal Event Forecasting , 2015, KDD.

[9]  VARUN CHANDOLA,et al.  Anomaly detection: A survey , 2009, CSUR.

[10]  Gaël Varoquaux,et al.  Scikit-learn: Machine Learning in Python , 2011, J. Mach. Learn. Res..

[11]  Jake D. Brutlag,et al.  Aberrant Behavior Detection in Time Series for Network Monitoring , 2000, LISA.

[12]  J. Keilwagen,et al.  Area under Precision-Recall Curves for Weighted and Unweighted Data , 2014, PloS one.

[13]  Ken Nishimatsu,et al.  Early network failure detection system by analyzing Twitter data , 2015, 2015 IFIP/IEEE International Symposium on Integrated Network Management (IM).

[14]  Aravind Srinivasan,et al.  'Beating the news' with EMBERS: forecasting civil unrest using open source indicators , 2014, KDD.

[15]  Jasmina Bogojeska,et al.  Predicting Disk Replacement towards Reliable Data Centers , 2016, KDD.

[16]  Durga Toshniwal,et al.  Sub-event detection during natural hazards using features of social media data , 2013, WWW.

[17]  George Forman,et al.  An Extensive Empirical Study of Feature Selection Metrics for Text Classification , 2003, J. Mach. Learn. Res..

[18]  Nathan Kallus,et al.  Predicting crowd behavior with big public data , 2014, WWW.

[19]  Hila Becker,et al.  Identifying content for planned events across social media sites , 2012, WSDM '12.

[20]  Jieping Ye,et al.  Hierarchical Incomplete Multi-source Feature Learning for Spatiotemporal Event Forecasting , 2016, KDD.