DARN: Dynamic Baselines for Real-time Network Monitoring

Network monitoring is necessary so as to ensure high reliability and availability in telecom networks. One of the main challenges posed by state-of-the-art monitoring tools is the creation of network baselines. Such baselines include thresholds that can be used to determine whether monitored values (with a given context, e.g. time) represent normal network operation or not. The size and complexity of current (and future) networks makes it infeasible to manually determine and set baselines for each network operator and metric, let alone adapting the thresholds to changes in network conditions. This leads to the use of default baselines and/or setting baselines only once and never changing them throughout the lifetime of network elements. This does not only cause inefficient operation, but could have implications for network reliability and availability. In this paper, we present the design, implementation, and evaluation of DARN: a collection of analytics and machine learning-based algorithms aimed at ensuring that network baselines are automatically adapted to different metric evolution. DARN has been comprehensively evaluated on a deployment with real traffic to confirm accuracy of generated baselines, a 22% improvement in accuracy due to baseline adaptation, and a 72% reduction in false alarms.

[1]  Raouf Boutaba,et al.  Topology-Aware Prediction of Virtual Network Function Resource Requirements , 2017, IEEE Transactions on Network and Service Management.

[2]  Gerhard Münz,et al.  Flexible Flow Aggregation for Adaptive Network Monitoring , 2006, Proceedings. 2006 31st IEEE Conference on Local Computer Networks.

[3]  Steven C. H. Hoi,et al.  Online ARIMA Algorithms for Time Series Prediction , 2016, AAAI.

[4]  Xueyi Ye,et al.  A New Dynamic Network Monitoring Based on IA , 2008 .

[5]  Michael J Demetsky,et al.  TRAFFIC FLOW FORECASTING: COMPARISON OF MODELING APPROACHES , 1997 .

[6]  Jose M. Alcaraz Calero,et al.  MonPaaS: An Adaptive Monitoring Platformas a Service for Cloud Computing Infrastructures and Services , 2015, IEEE Trans. Serv. Comput..

[7]  Mu-Chen Chen,et al.  Forecasting the short-term metro passenger flow with empirical mode decomposition and neural networks , 2012 .

[8]  Joan Serrat,et al.  Self-managed resources in network virtualisation environments , 2015, 2015 IFIP/IEEE International Symposium on Integrated Network Management (IM).

[9]  Joan Serrat,et al.  Management and orchestration challenges in network functions virtualization , 2016, IEEE Communications Magazine.

[10]  Jonathan D. Cryer,et al.  Time Series Analysis , 1986, Encyclopedia of Big Data.

[11]  J. Scargle.  Studies in astronomical time series analysis. II - Statistical aspects of spectral analysis of unevenly spaced data , 1982 .

[12]  Villy Bæk Iversen,et al.  TELETRAFFIC ENGINEERING HANDBOOK , 2001 .

[13]  F. Dressler,et al.  Simulative Analysis of Adaptive Network Monitoring Methodologies for Attack Detection , 2005, EUROCON 2005 - The International Conference on "Computer as a Tool".

[14]  Georgina Gallizo,et al.  Evaluation of monitoring tools for cloud computing environments , 2012, 2012 XXXVIII Conferencia Latinoamericana En Informatica (CLEI).

[15]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[16]  Geoffrey I. Webb,et al.  Dynamic Time Warping Averaging of Time Series Allows Faster and More Accurate Classification , 2014, 2014 IEEE International Conference on Data Mining.

[17]  Peter C. Y. Chen,et al.  LSTM network: a deep learning approach for short-term traffic forecast , 2017 .

[18]  K. Shimizu,et al.  Adaptive Network Monitoring System for Large-Volume Streaming Services in Multi-Domain Networks , 2012, 2012 World Telecommunications Congress.

[19]  J. Faraway.  Extending the Linear Model with R: Generalized Linear, Mixed Effects and Nonparametric Regression Models , 2005 .

[20]  Joan Serrat,et al.  Contributions to Efficient Resource Management in Virtual Networks , 2014, AIMS.